There isn't a clear concensus as to whether
or not IFF_RUNNING means the link is up
or not.

-seanb

On Tue, Feb 16, 2010 at 03:56:06PM -0500, Andrew Boyd wrote:
> 
> If you look at the end of any devnp driver _init()
> function, you will see it set the IFF_RUNNING flag,
> which is how it communicates to the stack that it's
> up.
> 
> However, given that the PHY has just been reset in
> the phy init function just above, that almost certainly
> restarts the auto-negotiation which is going to take
> 2 or 3 seconds to complete.
> 
> If you look at the ep93xx driver (which I wrote not
> too long ago) I tried to address this in the _start
> transmit packet function:
> 
> http://svn.ott.qnx.com/view/product/trunk/lib/io-pkt/sys/dev_qnx/ep93xx/
> transmit.c?revision=234104&view=markup
> 
> If the IFF_RUNNING flag wasn't set, or the NIC_FLAG_LINK_DOWN
> bit was set in the (legacy) cfg->flags, no transmission of
> packets was attempted - the packet was left queued.
> 
> --
> aboyd
> 
> 
> 
> 
> _______________________________________________
> 
> Networking Drivers
> http://community.qnx.com/sf/go/post47456
> 

On 16/02/10 03:56 PM, Andrew Boyd wrote:
> If you look at the end of any devnp driver _init()
> function, you will see it set the IFF_RUNNING flag,
> which is how it communicates to the stack that it's
> up.
>    

What Sean said. "There isn't a clear concensus as to whether or not 
IFF_RUNNING means the link is up or not."

Herein lies the problem IMHO. The stack uses IFF_RUNNING & IFF_UP to 
determine that the link is operational. Otherwise it'll return ENETDOWN. 
And the stack and the apps are designed to handle that error code (at 
least where I've looked). Once, that test passes, the packet is enqueued 
and there's no error code coming back (there can't be because it's a 
queue and transmission is asynchronous from that point forward).

There is also the if_link_state_change() mechanism (which is currently 
used to trigger routing sockets). Unfortunately it flip-flops 
LINK_STATE_UP/DOWN during the autonegotiation, and there isn't just ONE 
final "I'm done now" from the driver. Also, this mechanism would be 
asynchronous to the packet transmission. So there is nothing in the 
stack that is hooked into it and utilizing it requires more work.

The stack really needs to know when an interface is ready to function 
(has completed all initialization including autonegotiation). Because, 
although IP/Ethernet is unreliable by nature, as things start up stack 
(and some apps like DHCP) that needs to start their work as soon as the 
link is ready (and has normal drop probabilities).

I.e. this should really be synchronized better. What can we do to make 
it so?

The best option IMHO is for driver to set IFF_RUNNING only once link has 
been stabilized as completely operational. Then the stack and apps would 
"just work" because they're designed to handle ENETDOWN. It also follows 
the intention of the NetBSD stack which we're following with the stack. 
The driver is the best way to do this because only it has knowledge of 
what's going on re autoneg.

> If the IFF_RUNNING flag wasn't set, or the NIC_FLAG_LINK_DOWN

> bit was set in the (legacy) cfg->flags, no transmission of
> packets was attempted - the packet was left queued.
>    
Queuing the packet is certainly better.

Problem is that this is just one driver doing that. Not all of our 
drivers. And not all of customer's know that they should do this. (Do 
the BSD drivers do this?)

Unfortunately this can also mean that the packets are severely delayed. 
While I believe this is ok for most packets, a minority will have 
problems because they have timing constraints.

E.g. if an IPv6 address is assigned it'll be subject to Duplicate 
Address Detection (DAD). And the stack sends Neighbor Solicitations 
(NSs) to see if any other node has the same address configured. It then 
waits a few seconds, giving the other nodes a chance to respond. If the 
packet is queued during that time, because as you say autoneg can take a 
few seconds, then the other nodes may not get a chance to respond at 
all! Or, equally bad, the link goes up, driver sends out the NS packet, 
autoneg brings link down, other node responds, but it doesn't get 
received because link is down!

The consequences are severe because DAD is mandatory and pretty critical 
for IPv6 operation. There is a similar mechanism in IPv4, using ARP 
probes which would have the same problems. Although it isn't mandatory 
in IPv4 so the consequences may be lesser.

Out of all the packets that most commonly will be sent during the time 
when Ethernet autonegotiation is in progress the IPv4/IPv6 DAD is 
probably the only example which is time critical. Anybody know of others?

Maybe we could somehow special case the ones that are?

/P