|
Re: io-pkt thread context nightmare
|
04/30/2019 9:50 AM
post119686
|
Re: io-pkt thread context nightmare
All driver callbacks expect to run in stack context with the exception of receive and start (transmit), so the following
expect to be in stack context:
Driver entry
Attach
Detach
Init
Stop
Ioctl
The driver receive function runs in an interrupt worker context. While the driver start function is often called from
stack context, it must also be capable of running from interrupt worker context - in the case of bridging or fast
forwarding then the Rx on the ingress interface will directly call the Tx on the egress interface without passing in to
stack context.
You may be lucky and have some driver callbacks not fail when called from a context other than stack context, but this
is just luck with that particular driver.
Stack context is single threaded but with multiple coroutines in a run to completion model, but the coroutines may
choose to yield to avoid blocking stack context (which could block traffic on other interfaces that needs stack context)
. The stack context coroutines yield by calling ltsleep(), and nic_delay() is a wrapper for this. The e1000 driver (and
most drivers) use this when waiting for hardware. The nic_mutex() call is used to serialise operations to the hardware -
think of it as a standard mutex that works across stack context coroutines, but note that it introduces a yielding
point as it may not be able to acquire the lock.
To answer your other questions:
The memcpy() isn't going to yield stack context or access the hardware so it doesn't need to be inside the mutex,
nic_mutex() is taken around the update stats call because that is going to read hardware and we need to ensure that
other operations are not happening on the hardware at the same time.
ifq_enqueue(ifp,m) is potentially going to call the driver start routine. While this doesn't need to be in stack context
, it does need to be an mbuf handling thread - so an nw_pthread and not just a pthread.
mbufs should only be touched from nw_pthreads and not plain pthreads, but note that an nw_pthread needs to have a
quiesce function for when io-pkt is updating certain structures. Ensuring that the quiesce can happen in a multi-
threaded scenario takes careful planning to avoid deadlocking.
Please name any threads that you do create - by default they get an io-pkt#0x01 style name which makes people think that
they are io-pkt threads and not that they belong to an lsm!
Regards,
Nick
|
|
|