Thanks, Sean.

You know, I ported openssh over two years ago (and have done it several times since then with new releases), and this 
issue never came up.  Now I'm worried that it's broken -- tempting to just use your port instead!  (Although we had to 
add the ability to specify the password from the command line so that it can work like a typical router.  It's pretty 
simple and I could send you the diffs if you're interested.)

We run everything as root on our target, so the permissions issue doesn't happen in our environment.  We do chroot(), 
but we're just passing the fd of an open socket, so not a problem.

We were wondering whether this behavior is unique to QNX or if other implementations of sendmsg()/recvmsg() have a 
similar issue?

Thanks,
lew

> On Mon, Jul 20, 2009 at 10:22:36PM -0400, Lewis Donzis wrote:
> > We have an application where we use sendmsg() & recvmsg() to pass fds across
>  processes, and it works generally well.  However, we recently 
encounted an 
> error condition where it appears as if the sender closed the handle just 
> before the receiver duplicated it.
> > 
> > Searching through source, it appears that sendmsg() uses ConnectServerInfo()
>  to pass the information about its open fd, and that recvmsg() receives the 
> information, attaches to the sender's resource and sends a "dup" command to 
> create a copy of the handle.  But this doesn't explain how the sender knows 
> that the receiver has finished dup'ing the handles before it closes them.
> > 
> > In addition, in core_networking/trunk/sys/kern/uipc_usrreq.c, I find the 
> following comment:
> > 
> >          * Idealy, we should also "dup" the fd to ours here, so even if the
> >          * sender close the fd after send them out, we still have a proper
> >          * fd and won't confuse receiver...
> > 
> > but then no code to actually implement the "ideal" case!  And then on the 
> receive side:
> > 
> >          * Do nothing under QNX6. All the io_dup_t is passed
> >          * back to receiver, and receiver will dup it as they want.
> >          * Note if we did a "dup" in unp_internalize (to prevent sender
> >          * close fds), we need to close the duped fds after receiver
> >          * has completed its dup work.
> > 
> > So, I have two questions:
> > 
> > 1. Am I reading this correctly, i.e., that there truly is a race condition 
> where sendmsg() just sends the fd information and returns, and recvmsg() dups 
> the file handles it received?  In other other words, that there is nothing 
> stopping the sender from closing the file handle that it just sent before the 
> receiver has a chance to duplicate the handle it just received?
> > 
> > 2. If #1 is true, what can be done, short of creating a bunch of 
> synchronization logic to close the loop externally?  (which would really seem 
> to be covering a bug in the stack.)
> > 
> > Hopefully, I'm reading the code incorrectly and missing some other mechanism
>  that closes this hole.
> > 
> 
> Your analysis is correct.  It's not just a
> synchronization issue, the other end also
> needs perms to do the dup.  For example, grep
> for QNXNTO in the fdpassing code in our port
> of openssh:
> 
> http://community.qnx.com/integration/viewvc/viewvc.cgi/trunk/crypto/external/
> bsd/openssh/dist/monitor.c?root=core_networking&system=exsy1001&view=log
> http://community.qnx.com/integration/viewvc/viewvc.cgi/trunk/crypto/
external/
> bsd/openssh/dist/monitor_wrap.c?root=core_networking&system=exsy1001&view=log
> 
> Regards,
> 
> -seanb


While I can understand the value of adding this functionality, I can't totally understand the point of implementing it 
as an unreliable delivery system such that every application ported needs to be edited to be closed loop (as you have 
done in the openssh code)  Permissions aside, at least making sure that the sender can close the fd after calling 
sendmsg() seems like a necessity.  Since it is entirely library code that implements the fd sending, why not code the 
library such that sendmsg() does not complete until the receiving end has accepted the fd or gotten an error?

Henry