Project Home
Project Home
Documents
Documents
Wiki
Wiki
Discussion Forums
Discussions
Project Information
Project Info
Forum Topic - posix_spawn() arguments issue: (4 Items)
   
posix_spawn() arguments issue  
Good day,

I have been experiencing issues when using posix_spawn. The behavior I'm
seeing is that the argc argument list passed from the parent to the child
somehow gets corrupted in some situations. 

So far, my understanding of the situation is that the problem occurs when
the parent process' environment is "large". For example, if I run a simple
test of a parent creating a child process using posix_spawn with the default
LD_LIBRARY_PATH defined on QNX, it succeeds. Using the same test, it fails
if I modify the LD_LIBRARY_PATH in the parent's environment to something
quite large. For example, in my tests the default and the large
LD_LIBRARY_PATH were:

Default (QNX 6.4 self hosted): 
/proc/boot:/lib:/usr/lib:/lib/dll:/opt/lib:/usr/photon/lib:/usr/photon/dll:/
usr/local/lib:/opt/X11R6/lib:/usr/X11R6/lib

Large: 
/dir1a/dir2a/dir3a/dir4a/dir5a/dir6a/dir7a/dir8a/dir9a:/dir1b/dir2b/dir3b/di
r4b/dir5b/dir6b/dir7b/dir8b/dir9b:/dir1c/dir2c/dir3c/dir4c/dir5c/dir6c/dir7c
/dir8c/dir9c:/dir1d/dir2d/dir3d/dir4d/dir5d/dir6d/dir7d/dir8d/dir9d:/dir1e/d
ir2e/dir3e/dir4e/dir5e/dir6e/dir7e/dir8e/dir9e:/proc/boot:/lib:/usr/lib:/lib
/dll:/opt/lib:/usr/photon/lib:/usr/photon/dll:/usr/local/lib:/opt/X11R6/lib:
/usr/X11R6/lib

The observable behavior of the issue is that the argc parameters received in
the child process seem like they have been moved position. For example, the
parent passes the following argc list:

char* childArgv[] =
{"/root/PosixSpawnTest/Child","PARAM_1_NAME","PARAM_1_VALUE",  0};


When the test succeeds (with the small environment), the child prints the
argument list it received normally:

Child Param 0 is : /root/PosixSpawnTest/Child
Child Param 1 is : PARAM_1_NAME
Child Param 2 is : PARAM_1_VALUE
Child Param 3 is NULL

When the test fails, the child prints the following:

Child Param 0 is : SpawnTest/Child
Child Param 1 is : /root/PosixSpawnTest/Child
Child Param 2 is : PARAM_1_NAME
Child Param 3 is NULL
Child Param 4 is : PARAM_1_VALUE

So it looks like an extra parameter (derived from the file name?) was added
which shifted down the rest of the parameters, including the value which in
now passed the terminating null.

I would appreciate your help in determining if I'm doing something wrong, or
if I'm actually exceeding a limit on the size of the environment that can be
passed via posix_spawn().

Thank you,
Charles Auger

Re: posix_spawn() arguments issue  
Hi Charles,

I'm afraid that this sounds very much like a known problem (PR 75606).
If I remember correctly, the problem is somewhat ameliorated by some
changes I made in 6.5 (it should return an error rather than corrupting
anything) but there is still a (relatively small) upper limit on the
environment.

In summary, you are correct: there is a smallish upper limit on the
environment size.  Unfortunately the only workaround I can think of is
to make the environment smaller.  Perhaps you can use hard links,
symlinks, or union mounts to coalesce all of your libraries into a
single directory (perhaps one used exclusively for this purpose)? 

Regards,
Neil

On Thu, 2010-08-05 at 10:50 -0400, Charles Auger wrote:
> Good day,
> 
> I have been experiencing issues when using posix_spawn. The behavior I'm
> seeing is that the argc argument list passed from the parent to the child
> somehow gets corrupted in some situations. 
> 
> So far, my understanding of the situation is that the problem occurs when
> the parent process' environment is "large". For example, if I run a simple
> test of a parent creating a child process using posix_spawn with the default
> LD_LIBRARY_PATH defined on QNX, it succeeds. Using the same test, it fails
> if I modify the LD_LIBRARY_PATH in the parent's environment to something
> quite large. For example, in my tests the default and the large
> LD_LIBRARY_PATH were:
> 
> Default (QNX 6.4 self hosted): 
> /proc/boot:/lib:/usr/lib:/lib/dll:/opt/lib:/usr/photon/lib:/usr/photon/dll:/
> usr/local/lib:/opt/X11R6/lib:/usr/X11R6/lib
> 
> Large: 
> /dir1a/dir2a/dir3a/dir4a/dir5a/dir6a/dir7a/dir8a/dir9a:/dir1b/dir2b/dir3b/di
> r4b/dir5b/dir6b/dir7b/dir8b/dir9b:/dir1c/dir2c/dir3c/dir4c/dir5c/dir6c/dir7c
> /dir8c/dir9c:/dir1d/dir2d/dir3d/dir4d/dir5d/dir6d/dir7d/dir8d/dir9d:/dir1e/d
> ir2e/dir3e/dir4e/dir5e/dir6e/dir7e/dir8e/dir9e:/proc/boot:/lib:/usr/lib:/lib
> /dll:/opt/lib:/usr/photon/lib:/usr/photon/dll:/usr/local/lib:/opt/X11R6/lib:
> /usr/X11R6/lib
> 
> The observable behavior of the issue is that the argc parameters received in
> the child process seem like they have been moved position. For example, the
> parent passes the following argc list:
> 
> char* childArgv[] =
> {"/root/PosixSpawnTest/Child","PARAM_1_NAME","PARAM_1_VALUE",  0};
> 
> 
> When the test succeeds (with the small environment), the child prints the
> argument list it received normally:
> 
> Child Param 0 is : /root/PosixSpawnTest/Child
> Child Param 1 is : PARAM_1_NAME
> Child Param 2 is : PARAM_1_VALUE
> Child Param 3 is NULL
> 
> When the test fails, the child prints the following:
> 
> Child Param 0 is : SpawnTest/Child
> Child Param 1 is : /root/PosixSpawnTest/Child
> Child Param 2 is : PARAM_1_NAME
> Child Param 3 is NULL
> Child Param 4 is : PARAM_1_VALUE
> 
> So it looks like an extra parameter (derived from the file name?) was added
> which shifted down the rest of the parameters, including the value which in
> now passed the terminating null.
> 
> I would appreciate your help in determining if I'm doing something wrong, or
> if I'm actually exceeding a limit on the size of the environment that can be
> passed via posix_spawn().
> 
> Thank you,
> Charles Auger
> 
> 
> 
> 
> 
> _______________________________________________
> 
> OSTech
> http://community.qnx.com/sf/go/post61756
> 
Re: posix_spawn() arguments issue  
Hello Neil,

Thank you for the quick answer on this. Just two follow-up questions:

- Would you know what the actual limit for the size (or number of characters) of the environment is? 

- I don't think I can reproduce the issue using spawn() rather then posix_spawn(). Is the issue specific to 
posix_spawn()?

Thanks again,
Charles
Re: posix_spawn() arguments issue  
Hi Charles,

Yes, the problem is specific to posix_spawn() and has to do with an
obscure implementation detail.  Unfortunately the limit isn't a
hardcoded constant: it's a side effect of the somewhat odd way in which
posix_spawn() is implemented as a translation into a spawn request.  If
you have a source code license, the relevant bits under 6.4 are
lib/c/1d/posix_spawn.c and
services/system/procmgr/procmgr_posix_spawn.c.

Just FYI, there is another PR 77603 against 6.5 which indicates that we
may have a POSIX discrepancy in that we do not search the cwd for the
executable unless it is explicitly on the PATH.  The path search logic
in 6.4 is probably even less POSIX compliant.

Regards,
Neil

On Thu, 2010-08-05 at 11:48 -0400, Charles Auger wrote:
> Hello Neil,
> 
> Thank you for the quick answer on this. Just two follow-up questions:
> 
> - Would you know what the actual limit for the size (or number of characters) of the environment is? 
> 
> - I don't think I can reproduce the issue using spawn() rather then posix_spawn(). Is the issue specific to 
posix_spawn()?
> 
> Thanks again,
> Charles
> 
> 
> 
> _______________________________________________
> 
> OSTech
> http://community.qnx.com/sf/go/post61791
>