Project Home
Project Home
Wiki
Wiki
Discussion Forums
Discussions
Project Information
Project Info
Forum Topic - io-pkt crash when shutting down pppds: (3 Items)
   
io-pkt crash when shutting down pppds  
Hi,
we’re (again) having an issue with io-pkt crashing when disconnecting ppp connections (e.g. start multiple pppds + „ 
slay pppd“). Target is x86.
With our original support case (CaseID00100180) we could reproduce the problem with at least 3  ppp connections. 
Shutting down all three connections would always cause io-pkt-v4 to crash (NULL pointer exception).

We closed the support case (and I guess PR#73053) since we could no longer reproduce the problem with revision 1139 (io-
pkt main trunk, fixing PR71330 + PR71018). We now noticed the problem still exists – but it takes at least 6 pppd 
connections to crash io-pkt now. The crash always happens at the same moment, e.g. when shutting down the final 
connection – so it’s obviously not a rare timing issue.

We have also tested the latest revision from the 6.4.1 branch (io-pkt revision 149) – same result.

When using a debug build we see an assertion failing:
In function receive_loop_uni  Z:/io-pkt-rev149/6.4.1/trunk/sys/receive.c:1012 ((wpt))->wt_critical == 0 -- assertion 
failed

We started to dig a little deeper and found the assertion is triggered by two consecutive calls to „NW_SIGUNHOLD_P“ 
(which causes wt_critical == -1).
We logged the calls to „NW_SIGHOLD_P“ and „NW_SIGUNHOLD“ and get:

NW_SIGHOLD_P:   wpt->wt_critical=0  wpt: 0x80e7b00, m_ext_free:830
NW_SIGUNHOLD_P: wpt->wt_critical=1  wpt: 0x80e7b00, pool_cache_put_header:2764
NW_SIGHOLD_P:   wpt->wt_critical=0  wpt: 0x80e7b00, m_ext_free:830
NW_SIGUNHOLD_P: wpt->wt_critical=1  wpt: 0x80e7b00, pool_cache_put_header:2764
NW_SIGHOLD_P:   wpt->wt_critical=0  wpt: 0x80e7b00, m_ext_free:830
NW_SIGUNHOLD_P: wpt->wt_critical=1  wpt: 0x80e7b00, pool_cache_put_header:2764
NW_SIGUNHOLD_P: wpt->wt_critical=0  wpt: 0x80e7b00, pool_cache_get_header:2627
NW_SIGHOLD_P:   wpt->wt_critical=-1 wpt: 0x80e7b00, m_ext_free:830
NW_SIGUNHOLD_P: wpt->wt_critical=0  wpt: 0x80e7b00, pool_cache_put_header:2764
NW_SIGHOLD_P:   wpt->wt_critical=-1 wpt: 0x80e7b00, process_pkts:265
NW_SIGUNHOLD_P: wpt->wt_critical=0  wpt: 0x80e7b00, receive_loop_uni:1012
Z:/io-pkt-rev149/6.4.1/trunk/sys/receive.c:1012 ((wpt))->wt_critical == 0 -- assertion failed

Line numbers may be off by 1-2 lines (compared to rev 149) since we added some lines for logging.

Before we start spending any time to dig even deeper – any ideas/hints on the issue?

Regards,
Thorsten
Re: io-pkt crash when shutting down pppds  
On Tue, Mar 09, 2010 at 12:47:01PM -0500, Thorsten Brehm wrote:
> Hi,
> we???re (again) having an issue with io-pkt crashing when disconnecting ppp connections (e.g. start multiple pppds + ?
?? slay pppd???). Target is x86.
> With our original support case (CaseID00100180) we could reproduce the problem with at least 3  ppp connections. 
Shutting down all three connections would always cause io-pkt-v4 to crash (NULL pointer exception).
> 
> We closed the support case (and I guess PR#73053) since we could no longer reproduce the problem with revision 1139 
(io-pkt main trunk, fixing PR71330 + PR71018). We now noticed the problem still exists ??? but it takes at least 6 pppd 
connections to crash io-pkt now. The crash always happens at the same moment, e.g. when shutting down the final 
connection ??? so it???s obviously not a rare timing issue.
> 
> We have also tested the latest revision from the 6.4.1 branch (io-pkt revision 149) ??? same result.
> 
> When using a debug build we see an assertion failing:
> In function receive_loop_uni  Z:/io-pkt-rev149/6.4.1/trunk/sys/receive.c:1012 ((wpt))->wt_critical == 0 -- assertion 
failed
> 
> We started to dig a little deeper and found the assertion is triggered by two consecutive calls to ???NW_SIGUNHOLD_P??
? (which causes wt_critical == -1).
> We logged the calls to ???NW_SIGHOLD_P??? and ???NW_SIGUNHOLD??? and get:
> 
> NW_SIGHOLD_P:   wpt->wt_critical=0  wpt: 0x80e7b00, m_ext_free:830
> NW_SIGUNHOLD_P: wpt->wt_critical=1  wpt: 0x80e7b00, pool_cache_put_header:2764
> NW_SIGHOLD_P:   wpt->wt_critical=0  wpt: 0x80e7b00, m_ext_free:830
> NW_SIGUNHOLD_P: wpt->wt_critical=1  wpt: 0x80e7b00, pool_cache_put_header:2764
> NW_SIGHOLD_P:   wpt->wt_critical=0  wpt: 0x80e7b00, m_ext_free:830
> NW_SIGUNHOLD_P: wpt->wt_critical=1  wpt: 0x80e7b00, pool_cache_put_header:2764
> NW_SIGUNHOLD_P: wpt->wt_critical=0  wpt: 0x80e7b00, pool_cache_get_header:2627
> NW_SIGHOLD_P:   wpt->wt_critical=-1 wpt: 0x80e7b00, m_ext_free:830
> NW_SIGUNHOLD_P: wpt->wt_critical=0  wpt: 0x80e7b00, pool_cache_put_header:2764
> NW_SIGHOLD_P:   wpt->wt_critical=-1 wpt: 0x80e7b00, process_pkts:265
> NW_SIGUNHOLD_P: wpt->wt_critical=0  wpt: 0x80e7b00, receive_loop_uni:1012
> Z:/io-pkt-rev149/6.4.1/trunk/sys/receive.c:1012 ((wpt))->wt_critical == 0 -- assertion failed
> 
> Line numbers may be off by 1-2 lines (compared to rev 149) since we added some lines for logging.
> 
> Before we start spending any time to dig even deeper ??? any ideas/hints on the issue?

Yes, that particular count is off; however, I don't think
that in itself would cause an issue other than the assert
in the debug build as the underlying lock looks to be
handled in a consistent manner.  At any rate I've made a PR
on this lock issue.

Regards,

-seanb
Re: io-pkt crash when shutting down pppds  
> Yes, that particular count is off; however, I don't think
> that in itself would cause an issue other than the assert
> in the debug build as the underlying lock looks to be
> handled in a consistent manner.  At any rate I've made a PR
> on this lock issue.
> 
> Regards,
> 
> -seanb

Ok, so the assertion *may* be unrelated to the actual crash (in the standard build) and it may be worth to just ignore 
it for now and go after the cause of the actual crash (NULL pointer exception)?

Regards,
Thorsten