|
Mario Charest
08/18/2008 2:31 PM
post11967
|
Why would sendto() on a udp socket set in non blocking mode takes 10 ms about every 1 per second while the other calls
to sendto() are very fast. If the buffer is full shouldn't it return with -1. That's with io-net 6.3.2.
I can see the call being block for 10ms via the profiler ( a write to io-net ). Yet the CPU is almost idle during this
10ms.
- Mario
|
|
|
|
|
|
Sean Boudreau(deleted)
08/18/2008 2:35 PM
post11968
|
On Mon, Aug 18, 2008 at 02:31:54PM -0400, Mario Charest wrote:
>
> Why would sendto() on a udp socket set in non blocking mode takes 10 ms about every 1 per second while the other calls
to sendto() are very fast. If the buffer is full shouldn't it return with -1. That's with io-net 6.3.2.
>
> I can see the call being block for 10ms via the profiler ( a write to io-net ). Yet the CPU is almost idle during
this 10ms.
Is it SEND blocked or REPLY blocked?
-seanb
|
|
|
|
|
|
Mario Charest
08/18/2008 3:01 PM
post11970
|
> On Mon, Aug 18, 2008 at 02:31:54PM -0400, Mario Charest wrote:
> >
> > Why would sendto() on a udp socket set in non blocking mode takes 10 ms
> about every 1 per second while the other calls to sendto() are very fast. If
> the buffer is full shouldn't it return with -1. That's with io-net 6.3.2.
> >
> > I can see the call being block for 10ms via the profiler ( a write to io-
> net ). Yet the CPU is almost idle during this 10ms.
>
> Is it SEND blocked or REPLY blocked?
>
> -seanb
Sorry, it's REPLY blocked, then I can see io-net receiving the message and becoming block on a MUTEX almost right away
and unblocking 10ms latter.
The unblocking occurs after thread 9 from io-net return from a 10ms nanosleep... That 10ms nanosleep occurs every 500ms
.
|
|
|
|
|
|
Robert Craig
08/18/2008 3:37 PM
post11976
|
Hmmmm... I wonder if you're getting hit by the network card doing a
probe of the PHY chip? What driver are you using? Does it have a
"probe_phy" option available?
Robert.
-----Original Message-----
From: Mario Charest [mailto:community-noreply@qnx.com]
Sent: Monday, August 18, 2008 3:01 PM
To: technology-networking
Subject: Re: Slow UDP
> On Mon, Aug 18, 2008 at 02:31:54PM -0400, Mario Charest wrote:
> >
> > Why would sendto() on a udp socket set in non blocking mode takes 10
ms
> about every 1 per second while the other calls to sendto() are very
fast. If
> the buffer is full shouldn't it return with -1. That's with io-net
6.3.2.
> >
> > I can see the call being block for 10ms via the profiler ( a write
to io-
> net ). Yet the CPU is almost idle during this 10ms.
>
> Is it SEND blocked or REPLY blocked?
>
> -seanb
Sorry, it's REPLY blocked, then I can see io-net receiving the message
and becoming block on a MUTEX almost right away and unblocking 10ms
latter.
The unblocking occurs after thread 9 from io-net return from a 10ms
nanosleep... That 10ms nanosleep occurs every 500ms.
_______________________________________________
Technology
http://community.qnx.com/sf/go/post11970
|
|
|
|
|
|
Mario Charest
08/18/2008 4:11 PM
post11982
|
> Hmmmm... I wonder if you're getting hit by the network card doing a
> probe of the PHY chip? What driver are you using? Does it have a
> "probe_phy" option available?
>
It's 100Mbits Intel Card with the speedo driver.
|
|
|
|
|
|
Robert Craig
08/18/2008 4:27 PM
post11984
|
I'm not sure what version of the driver has actually shipped (do you
have a date on it), but I'm guessing that the problem is here inside the
get stats routine.
speedo_send_bcmd(speedo, B_CUC_STATS, 1);
delay(10);
if (ENDIAN_LE32(dstats->completion)) {
Newer code has replaced the delay(10) with a polling loop instead.
Robert.
-----Original Message-----
From: Mario Charest [mailto:community-noreply@qnx.com]
Sent: Monday, August 18, 2008 4:11 PM
To: technology-networking
Subject: Re: RE: Slow UDP
> Hmmmm... I wonder if you're getting hit by the network card doing a
> probe of the PHY chip? What driver are you using? Does it have a
> "probe_phy" option available?
>
It's 100Mbits Intel Card with the speedo driver.
_______________________________________________
Technology
http://community.qnx.com/sf/go/post11982
|
|
|
|
|
|
Mario Charest
08/18/2008 4:43 PM
post11987
|
> I'm not sure what version of the driver has actually shipped (do you
> have a date on it), but I'm guessing that the problem is here inside the
> get stats routine.
>
April 30 2004, TAGID=193
> speedo_send_bcmd(speedo, B_CUC_STATS, 1);
>
> delay(10);
>
> if (ENDIAN_LE32(dstats->completion)) {
>
>
> Newer code has replaced the delay(10) with a polling loop instead.
The stats are coming from the hardware?
>
> Robert.
>
>
>
>
> -----Original Message-----
> From: Mario Charest [mailto:community-noreply@qnx.com]
> Sent: Monday, August 18, 2008 4:11 PM
> To: technology-networking
> Subject: Re: RE: Slow UDP
>
> > Hmmmm... I wonder if you're getting hit by the network card doing a
> > probe of the PHY chip? What driver are you using? Does it have a
> > "probe_phy" option available?
> >
>
> It's 100Mbits Intel Card with the speedo driver.
>
>
>
> _______________________________________________
> Technology
> http://community.qnx.com/sf/go/post11982
|
|
|
|
|
|
Robert Craig
08/18/2008 4:48 PM
post11989
|
The removal of the delay was made on Mon Jan 17 15:56:44 2005 UTC, so
that's definitely the problem.
In terms of the stats, some are maintained in the driver, others come
straight from the hardware.
(e.g.
stats->txed_ok = ENDIAN_LE32(dstats->tx_good_frames);
stats->rxed_ok = ENDIAN_LE32(dstats->rx_good_frames);
stats->octets_txed_ok = speedo->txed_bytes;
stats->octets_rxed_ok = speedo->rxed_bytes;
estats->tx_deferred = ENDIAN_LE32(dstats->tx_deferred);
estats->xcoll_aborted =
ENDIAN_LE32(dstats->tx_coll16_errs);
estats->late_collisions =
ENDIAN_LE32(dstats->tx_late_colls);
estats->single_collisions =
ENDIAN_LE32(dstats->tx_one_colls);
estats->multi_collisions =
ENDIAN_LE32(dstats->tx_multi_colls);
estats->total_collision_frames = estats->late_collisions
+
estats->single_collisions +
estats->multi_collisions;
)
-----Original Message-----
From: Mario Charest [mailto:community-noreply@qnx.com]
Sent: Monday, August 18, 2008 4:43 PM
To: technology-networking
Subject: Re: RE: RE: Slow UDP
> I'm not sure what version of the driver has actually shipped (do you
> have a date on it), but I'm guessing that the problem is here inside
the
> get stats routine.
>
April 30 2004, TAGID=193
> speedo_send_bcmd(speedo, B_CUC_STATS, 1);
>
> delay(10);
>
> if (ENDIAN_LE32(dstats->completion)) {
>
>
> Newer code has replaced the delay(10) with a polling loop instead.
The stats are coming from the hardware?
>
> Robert.
>
>
>
>
> -----Original Message-----
> From: Mario Charest [mailto:community-noreply@qnx.com]
> Sent: Monday, August 18, 2008 4:11 PM
> To: technology-networking
> Subject: Re: RE: Slow UDP
>
> > Hmmmm... I wonder if you're getting hit by the network card doing a
> > probe of the PHY chip? What driver are you using? Does it have a
> > "probe_phy" option available?
> >
>
> It's 100Mbits Intel Card with the speedo driver.
>
>
>
> _______________________________________________
> Technology
> http://community.qnx.com/sf/go/post11982
_______________________________________________
Technology
http://community.qnx.com/sf/go/post11987
|
|
|
|
|
|
Thomas Fletcher
08/20/2008 3:24 PM
post12116
|
Robert Craig wrote:
> The removal of the delay was made on Mon Jan 17 15:56:44 2005 UTC, so
> that's definitely the problem.
>
Since this holds off other clients (the mutex locking) would it not be
possible to make the statistics
a non-blocking call. Statistics are only a relative snapshot in time
... so as long as they are trending
in the proper direction, the actual detailed values are likely less
important (since they are immediately
out of date) compared to a performant connection.
Thomas
--
Thomas Fletcher
Crank Software Inc.
www.cranksoftware.com
|
|
|
|
|
|
Robert Craig
08/20/2008 3:32 PM
post12121
|
Hi Thomas:
Not quite sure what you mean by making it non-blocking? Do you
mean the tx_mutex locking inside of the devctl or are you thinking about
something else?
Robert.
-----Original Message-----
From: Thomas Fletcher [mailto:community-noreply@qnx.com]
Sent: Wednesday, August 20, 2008 3:25 PM
To: technology-networking
Subject: Re: Slow UDP
Robert Craig wrote:
> The removal of the delay was made on Mon Jan 17 15:56:44 2005 UTC, so
> that's definitely the problem.
>
Since this holds off other clients (the mutex locking) would it not be
possible to make the statistics
a non-blocking call. Statistics are only a relative snapshot in time
... so as long as they are trending
in the proper direction, the actual detailed values are likely less
important (since they are immediately
out of date) compared to a performant connection.
Thomas
--
Thomas Fletcher
Crank Software Inc.
www.cranksoftware.com
_______________________________________________
Technology
http://community.qnx.com/sf/go/post12116
|
|
|
|
|
|
Thomas Fletcher
08/20/2008 3:45 PM
post12123
|
Robert Craig wrote:
> Hi Thomas:
> Not quite sure what you mean by making it non-blocking? Do you
> mean the tx_mutex locking inside of the devctl or are you thinking about
> something else?
>
I haven't looked at the code, but it seems that it was either
serializing on that mutex and/or waiting
for the HW to provide the stat information. If the stat information
could be provided to clients in
a non-blocking way (ie read only with an atomic stat structure swap)
then the HW data could be
updated during idle periods and/or inline with other HW accesses.
Of course, I have no idea how long it takes to pull that information
from the HW, so perhaps it
isn't feasible at all ... but it seems that a request for stats
shouldn't lock out clients trying to actually
send or receive data.
Thomas
> -----Original Message-----
> From: Thomas Fletcher [mailto:community-noreply@qnx.com]
> Sent: Wednesday, August 20, 2008 3:25 PM
> To: technology-networking
> Subject: Re: Slow UDP
>
> Robert Craig wrote:
>
>> The removal of the delay was made on Mon Jan 17 15:56:44 2005 UTC, so
>> that's definitely the problem.
>>
>>
>
> Since this holds off other clients (the mutex locking) would it not be
> possible to make the statistics
> a non-blocking call. Statistics are only a relative snapshot in time
> ... so as long as they are trending
> in the proper direction, the actual detailed values are likely less
> important (since they are immediately
> out of date) compared to a performant connection.
>
> Thomas
>
>
--
Thomas Fletcher
Crank Software Inc.
www.cranksoftware.com
|
|
|
|
|
|
Robert Craig
08/20/2008 3:57 PM
post12126
|
Yup... so you were thinking about the mutex lock. This is probably a
moot point in that the newer code removed the delay. There's still some
question as to how long the hardware takes to retrieve the statistics,
but I wouldn't expect that to be significant.
I agree that a request for stats shouldn't prevent data from being sent
/ received though. Most cards have registers that are directly
accessible without having to go through some sort of request to the
hardware to update them.
Robert.
-----Original Message-----
From: Thomas Fletcher [mailto:community-noreply@qnx.com]
Sent: Wednesday, August 20, 2008 3:45 PM
To: technology-networking
Subject: Re: Slow UDP
Robert Craig wrote:
> Hi Thomas:
> Not quite sure what you mean by making it non-blocking? Do you
> mean the tx_mutex locking inside of the devctl or are you thinking
about
> something else?
>
I haven't looked at the code, but it seems that it was either
serializing on that mutex and/or waiting
for the HW to provide the stat information. If the stat information
could be provided to clients in
a non-blocking way (ie read only with an atomic stat structure swap)
then the HW data could be
updated during idle periods and/or inline with other HW accesses.
Of course, I have no idea how long it takes to pull that information
from the HW, so perhaps it
isn't feasible at all ... but it seems that a request for stats
shouldn't lock out clients trying to actually
send or receive data.
Thomas
> -----Original Message-----
> From: Thomas Fletcher [mailto:community-noreply@qnx.com]
> Sent: Wednesday, August 20, 2008 3:25 PM
> To: technology-networking
> Subject: Re: Slow UDP
>
> Robert Craig wrote:
>
>> The removal of the delay was made on Mon Jan 17 15:56:44 2005 UTC, so
>> that's definitely the problem.
>>
>>
>
> Since this holds off other clients (the mutex locking) would it not be
> possible to make the statistics
> a non-blocking call. Statistics are only a relative snapshot in time
> ... so as long as they are trending
> in the proper direction, the actual detailed values are likely less
> important (since they are immediately
> out of date) compared to a performant connection.
>
> Thomas
>
>
--
Thomas Fletcher
Crank Software Inc.
www.cranksoftware.com
_______________________________________________
Technology
http://community.qnx.com/sf/go/post12123
|
|
|
|
|
|
Mario Charest
08/18/2008 3:38 PM
post11977
|
> > On Mon, Aug 18, 2008 at 02:31:54PM -0400, Mario Charest wrote:
> > >
> > > Why would sendto() on a udp socket set in non blocking mode takes 10 ms
> > about every 1 per second while the other calls to sendto() are very fast.
> If
> > the buffer is full shouldn't it return with -1. That's with io-net 6.3.2.
> > >
> > > I can see the call being block for 10ms via the profiler ( a write to io-
>
> > net ). Yet the CPU is almost idle during this 10ms.
> >
> > Is it SEND blocked or REPLY blocked?
> >
> > -seanb
>
> Sorry, it's REPLY blocked, then I can see io-net receiving the message and
> becoming block on a MUTEX almost right away and unblocking 10ms latter.
>
> The unblocking occurs after thread 9 from io-net return from a 10ms nanosleep.
> .. That 10ms nanosleep occurs every 500ms.
>
The nanosleep of io-net thread 9 seems to occurs following a message send from shelf:
Event, Time, Owner, Type, Data
233600, 4s 216ms 758us, shelf Thread 2, MsgSendv Enter, coid 0x8 sparts 1 rparts 2 msg0 0x100106 msg1 0x44c00416 msg2
0x4c0 function devctl().
I got rid of the shelf plugin ( System monitor ) and the problem went away.
|
|
|
|
|
|
Robert Craig
08/18/2008 3:40 PM
post11978
|
That would be the network monitor utility querying the driver for
statistics to display the usage graph.
-----Original Message-----
From: Mario Charest [mailto:community-noreply@qnx.com]
Sent: Monday, August 18, 2008 3:39 PM
To: technology-networking
Subject: Re: Slow UDP
> > On Mon, Aug 18, 2008 at 02:31:54PM -0400, Mario Charest wrote:
> > >
> > > Why would sendto() on a udp socket set in non blocking mode takes
10 ms
> > about every 1 per second while the other calls to sendto() are very
fast.
> If
> > the buffer is full shouldn't it return with -1. That's with io-net
6.3.2.
> > >
> > > I can see the call being block for 10ms via the profiler ( a
write to io-
>
> > net ). Yet the CPU is almost idle during this 10ms.
> >
> > Is it SEND blocked or REPLY blocked?
> >
> > -seanb
>
> Sorry, it's REPLY blocked, then I can see io-net receiving the message
and
> becoming block on a MUTEX almost right away and unblocking 10ms
latter.
>
> The unblocking occurs after thread 9 from io-net return from a 10ms
nanosleep.
> .. That 10ms nanosleep occurs every 500ms.
>
The nanosleep of io-net thread 9 seems to occurs following a message
send from shelf:
Event, Time, Owner, Type, Data
233600, 4s 216ms 758us, shelf Thread 2, MsgSendv Enter, coid 0x8 sparts
1 rparts 2 msg0 0x100106 msg1 0x44c00416 msg2 0x4c0 function devctl().
I got rid of the shelf plugin ( System monitor ) and the problem went
away.
_______________________________________________
Technology
http://community.qnx.com/sf/go/post11977
|
|
|
|
|
|
Mario Charest
08/18/2008 4:08 PM
post11980
|
> That would be the network monitor utility querying the driver for
> statistics to display the usage graph.
>
Yeah that I figured, but you wouldn't expect the network monitoring utility to disrupt network operation of a program
running at priority 100 ;)
|
|
|
|
|
|
