Shannon Gu
01/03/2008 12:16 PM
post4036
|
Hi all,
We have recently started a project based on io-pkt infrastructure, how ever we've observed following symptoms
regarding network stack, I'd appreciate if anyone could shed some light on where this problem might come from? (network
stack setup/configuration or build)
============================================
#1 . ping test with bigger buffer (ping out / ping in has same symptom ):
[/]# ping -c 100 -s 4432 172.16.1.48
PING 172.16.1.48 (172.16.1.48): 4432 data bytes
4440 bytes from 172.16.1.48: icmp_seq=0 ttl=128 time=9 ms
4440 bytes from 172.16.1.48: icmp_seq=1 ttl=128 time=9 ms
4440 bytes from 172.16.1.48: icmp_seq=2 ttl=128 time=8 ms
4440 bytes from 172.16.1.48: icmp_seq=3 ttl=128 time=8 ms
4440 bytes from 172.16.1.48: icmp_seq=4 ttl=128 time=725 ms
----172.16.1.48 PING Statistics----
5 packets transmitted, 5 packets received, 0% packet loss
round-trip min/avg/max = 8/151/725 ms variance = 102977 ms^2
[/]# ping -c 100 -s 4433 172.16.1.48
PING 172.16.1.48 (172.16.1.48): 4433 data bytes
----172.16.1.48 PING Statistics----
100 packets transmitted, 0 packets received, 100% packet loss
[/]#
ping test with bigger buffer is OK within LAN , not cross LAN, and same behavior happens to both wired and wireless
interfaces (wireless interface using wpa_supplicant provided on the board to associate with AP in an open mode) , so I
suspect that there is an buffer issue at layer that is higher than layer2 (tcp/ip possible);
[/etc]# ifconfig
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> mtu 33192
inet 127.0.0.1 netmask 0xff000000
ath0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500
ssid AcmeWifi
powersave off
bssid 00:03:52:e0:46:f0 chan 7
address: 00:06:80:00:a1:ed
media: IEEE802.11 autoselect (OFDM12 mode 11g)
status: active
inet 192.168.10.87 netmask 0xffffff00 broadcast 192.168.10.255
en0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500
address: 00:55:44:33:22:11
inet 172.16.3.54 netmask 0xffff0000 broadcast 172.16.255.255
============= Ethernet interface en0 ==============
[/etc]# ping -s 5000 172.16.3.1
PING 172.16.3.1 (172.16.3.1): 5000 data bytes
5008 bytes from 172.16.3.1: icmp_seq=0 ttl=64 time=11 ms
5008 bytes from 172.16.3.1: icmp_seq=1 ttl=64 time=12 ms
----172.16.3.1 PING Statistics----
2 packets transmitted, 2 packets received, 0% packet loss
round-trip min/avg/max = 11/11/12 ms variance = 23 ms^2
[/etc]# ping -s 5000 172.16.1.48
PING 172.16.1.48 (172.16.1.48): 5000 data bytes
----172.16.1.48 PING Statistics----
221 packets transmitted, 0 packets received, 100% packet loss
[/etc]# ping 172.16.1.48
PING 172.16.1.48 (172.16.1.48): 56 data bytes
64 bytes from 172.16.1.48: icmp_seq=0 ttl=128 time=1 ms
64 bytes from 172.16.1.48: icmp_seq=1 ttl=128 time=1 ms
----172.16.1.48 PING Statistics----
2 packets transmitted, 2 packets received, 0% packet loss
round-trip min/avg/max = 1/1/1 ms variance = 0 ms^2
============== wireless interface ath0 =========
/etc]# ifconfig en0 down
[/etc]# ifconfig
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> mtu 33192
inet 127.0.0.1 netmask 0xff000000
ath0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500
ssid Devicescape-Guest
powersave off
bssid 00:14:c2:a5:4c:41 chan 10
address: 00:06:80:00:a1:ed
media: IEEE802.11 autoselect (OFDM54 mode 11g)
status: active
inet 192.168.30.10 netmask 0xffffff00 broadcast 192.168.30.255
en0: flags=8802<BROADCAST,SIMPLEX,MULTICAST> mtu 1500
address: 00:55:44:33:22:11
inet 172.16.3.54 netmask 0xffff0000 broadcast 172.16.255.255
[/etc]# ping...
View Full Message
|
|
|
Sean Boudreau(deleted)
|
Re: network buffer issues
|
Sean Boudreau(deleted)
01/05/2008 8:59 AM
post4066
|
Re: network buffer issues
On Thu, Jan 03, 2008 at 12:16:07PM -0500, Shannon Gu wrote:
> Hi all,
>
> We have recently started a project based on io-pkt infrastructure, how ever we've observed following symptoms
regarding network stack, I'd appreciate if anyone could shed some light on where this problem might come from? (network
stack setup/configuration or build)
>
>
> ============================================
> #1 . ping test with bigger buffer (ping out / ping in has same symptom ):
>
>
> [/]# ping -c 100 -s 4432 172.16.1.48
>
> PING 172.16.1.48 (172.16.1.48): 4432 data bytes
>
> 4440 bytes from 172.16.1.48: icmp_seq=0 ttl=128 time=9 ms
>
> 4440 bytes from 172.16.1.48: icmp_seq=1 ttl=128 time=9 ms
>
> 4440 bytes from 172.16.1.48: icmp_seq=2 ttl=128 time=8 ms
>
> 4440 bytes from 172.16.1.48: icmp_seq=3 ttl=128 time=8 ms
>
> 4440 bytes from 172.16.1.48: icmp_seq=4 ttl=128 time=725 ms
>
>
>
> ----172.16.1.48 PING Statistics----
>
> 5 packets transmitted, 5 packets received, 0% packet loss
>
> round-trip min/avg/max = 8/151/725 ms variance = 102977 ms^2
>
> [/]# ping -c 100 -s 4433 172.16.1.48
>
> PING 172.16.1.48 (172.16.1.48): 4433 data bytes
>
>
>
> ----172.16.1.48 PING Statistics----
>
> 100 packets transmitted, 0 packets received, 100% packet loss
>
> [/]#
>
Something must have changed between the two runs. It's hard
to say with the providfed info but it may be a routing issue.
Make sure 172.16.1.48 has a route back to you.
>
>
> ping test with bigger buffer is OK within LAN , not cross LAN, and same behavior happens to both wired and wireless
interfaces (wireless interface using wpa_supplicant provided on the board to associate with AP in an open mode) , so I
suspect that there is an buffer issue at layer that is higher than layer2 (tcp/ip possible);
> [/etc]# ifconfig
>
> lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> mtu 33192
>
> inet 127.0.0.1 netmask 0xff000000
>
> ath0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500
>
> ssid AcmeWifi
>
> powersave off
>
> bssid 00:03:52:e0:46:f0 chan 7
>
> address: 00:06:80:00:a1:ed
>
> media: IEEE802.11 autoselect (OFDM12 mode 11g)
>
> status: active
>
> inet 192.168.10.87 netmask 0xffffff00 broadcast 192.168.10.255
>
> en0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500
>
> address: 00:55:44:33:22:11
>
> inet 172.16.3.54 netmask 0xffff0000 broadcast 172.16.255.255
>
>
>
>
>
> ============= Ethernet interface en0 ==============
>
>
>
> [/etc]# ping -s 5000 172.16.3.1
>
> PING 172.16.3.1 (172.16.3.1): 5000 data bytes
>
> 5008 bytes from 172.16.3.1: icmp_seq=0 ttl=64 time=11 ms
>
> 5008 bytes from 172.16.3.1: icmp_seq=1 ttl=64 time=12 ms
>
>
>
> ----172.16.3.1 PING Statistics----
>
> 2 packets transmitted, 2 packets received, 0% packet loss
>
> round-trip min/avg/max = 11/11/12 ms variance = 23 ms^2
>
>
>
> [/etc]# ping -s 5000 172.16.1.48
>
> PING 172.16.1.48 (172.16.1.48): 5000 data bytes
>
>
>
> ----172.16.1.48 PING Statistics----
>
> 221 packets transmitted, 0 packets received, 100% packet loss
>
>
>
>
>
> [/etc]# ping 172.16.1.48
>
> PING 172.16.1.48 (172.16.1.48): 56 data bytes
>
> 64 bytes from 172.16.1.48: icmp_seq=0 ttl=128 time=1 ms
>
> 64 bytes from 172.16.1.48: icmp_seq=1 ttl=128 time=1 ms
>
>
>
> ----172.16.1.48 PING Statistics----
>
> 2 packets transmitted, 2 packets...
View Full Message
|
|
|
Shannon Gu
|
Re: network buffer issues
|
Shannon Gu
01/07/2008 1:59 PM
post4099
|
Re: network buffer issues
============================================
> > #1 . ping test with bigger buffer (ping out / ping in has same symptom ):
> >
> >
> > [/]# ping -c 100 -s 4432 172.16.1.48
> >
> > PING 172.16.1.48 (172.16.1.48): 4432 data bytes
> >
> > 4440 bytes from 172.16.1.48: icmp_seq=0 ttl=128 time=9 ms
> >
> > 4440 bytes from 172.16.1.48: icmp_seq=1 ttl=128 time=9 ms
> >
> > 4440 bytes from 172.16.1.48: icmp_seq=2 ttl=128 time=8 ms
> >
> > 4440 bytes from 172.16.1.48: icmp_seq=3 ttl=128 time=8 ms
> >
> > 4440 bytes from 172.16.1.48: icmp_seq=4 ttl=128 time=725 ms
> >
> >
> >
> > ----172.16.1.48 PING Statistics----
> >
> > 5 packets transmitted, 5 packets received, 0% packet loss
> >
> > round-trip min/avg/max = 8/151/725 ms variance = 102977 ms^2
> >
> > [/]# ping -c 100 -s 4433 172.16.1.48
> >
> > PING 172.16.1.48 (172.16.1.48): 4433 data bytes
> >
> >
> >
> > ----172.16.1.48 PING Statistics----
> >
> > 100 packets transmitted, 0 packets received, 100% packet loss
> >
> > [/]#
> >
>
> Something must have changed between the two runs. It's hard
> to say with the providfed info but it may be a routing issue.
> Make sure 172.16.1.48 has a route back to you.
>
Yes 172.16.1.48 has a route back, but expeirence the same size issues to the target.
Another observation is that the packet size is destination dependent, for every host I pinged, each one fixes at certain
buffer size like shreshhold, some at couple of kilo bytes, some at several, I have not seen any one exeeded 10 Kb ...
Thanks a lot for your prompt response!
Regards,
Shannon
|
|
|
Robert Craig
|
RE: network buffer issues
|
Robert Craig
01/07/2008 2:11 PM
post4101
|
RE: network buffer issues
Is the router between the subnet not letting larger ICMP packets through?
Is the truncation tied to a subnet or to particular hosts on the subnet?
Is it possible to put a sniffer on one of the other hosts to see if the
packets are being truncated? Are the other hosts all running the same OS?
Does the reverse path (pinging from the host to QNX work)?
Problem here is that we can't really tell where things are falling apart. It
would seem (:->) that if everything on the same subnet is working with large
packets then the QNX side of things is working properly. What you really
need to do is follow the larger packets through the network and try and see
where they're getting lost or discarded.
Robert.
-----Original Message-----
From: Shannon Gu [mailto:shannon@devicescape.com]
Sent: Monday, January 07, 2008 1:59 PM
To: ionetmig-networking
Subject: Re: network buffer issues
============================================
> > #1 . ping test with bigger buffer (ping out / ping in has same symptom
):
> >
> >
> > [/]# ping -c 100 -s 4432 172.16.1.48
> >
> > PING 172.16.1.48 (172.16.1.48): 4432 data bytes
> >
> > 4440 bytes from 172.16.1.48: icmp_seq=0 ttl=128 time=9 ms
> >
> > 4440 bytes from 172.16.1.48: icmp_seq=1 ttl=128 time=9 ms
> >
> > 4440 bytes from 172.16.1.48: icmp_seq=2 ttl=128 time=8 ms
> >
> > 4440 bytes from 172.16.1.48: icmp_seq=3 ttl=128 time=8 ms
> >
> > 4440 bytes from 172.16.1.48: icmp_seq=4 ttl=128 time=725 ms
> >
> >
> >
> > ----172.16.1.48 PING Statistics----
> >
> > 5 packets transmitted, 5 packets received, 0% packet loss
> >
> > round-trip min/avg/max = 8/151/725 ms variance = 102977 ms^2
> >
> > [/]# ping -c 100 -s 4433 172.16.1.48
> >
> > PING 172.16.1.48 (172.16.1.48): 4433 data bytes
> >
> >
> >
> > ----172.16.1.48 PING Statistics----
> >
> > 100 packets transmitted, 0 packets received, 100% packet loss
> >
> > [/]#
> >
>
> Something must have changed between the two runs. It's hard
> to say with the providfed info but it may be a routing issue.
> Make sure 172.16.1.48 has a route back to you.
>
Yes 172.16.1.48 has a route back, but expeirence the same size issues to the
target.
Another observation is that the packet size is destination dependent, for
every host I pinged, each one fixes at certain buffer size like shreshhold,
some at couple of kilo bytes, some at several, I have not seen any one
exeeded 10 Kb ...
Thanks a lot for your prompt response!
Regards,
Shannon
_______________________________________________
io-net migration
http://community.qnx.com/sf/go/post4099
|
|
|
Shannon Gu
|
Re: network buffer issues
|
Shannon Gu
01/07/2008 4:08 PM
post4107
|
Re: network buffer issues
============================================
> > #2. wpa_supplicnt in PSK mode stopped and network on board is basically
> crashed:
> >
> > a. wpa_supplicant stopped during 4-way handshake after receiving Key;
> sniffer shows that the client(the board) never response to key exchange;
> >
> > b. AP shows that WPA pairwise key handshake timed out ( no reply was
> received for the first massage)
> >
> > c. On board , the shell still working, except ifconfig command hangs; but it
> is able to quit after inputting Ctrl-C;
> >
> > d. supplicant involves pcap lib when using security mode to transfer packets
> while in open mode does not;
> > So if I put a stop at calling pcap_inject before let it continue, then
> network system on board wont crash, the program will go on and on, the
> buffered packets will be eventually sent out;
> > but this is only at a slow speed, if I hit the continue right away then
> the system will hang right away, just like what a~c described; the result is
> that the board will never get authenticated with any AP as it always times out
> ???
>
> Some more detailed info on how to reproduce this would be
> appreciated: config files etc.
>
1. wpa_supplicant command is :
[/]#wpa_supplicant -Dbsd -iath0 -c/etc/shxy.conf -ddd
2. [/]# cat etc/shxy.conf
ctrl_interface=/var/run/wpa_supplicant
network={
disabled=0
ssid="tomato-DLS"
proto=WPA2
key_mgmt=WPA-PSK
psk="secureDLS"
}
3. I attached some reference files here:
-- An Ethereal file that captured all the related packets;
-- A wpa_supplicant log file; (next post)
Thanks a lot,
Shannon
|
|
|
Shannon Gu
|
Re: network buffer issues
|
Shannon Gu
01/07/2008 4:10 PM
post4108
|
Re: network buffer issues
============================================
> > #2. wpa_supplicnt in PSK mode stopped and network on board is basically
> crashed:
This is to attach the log file
Regards,
Shannon
|
|
|
Robert Craig
|
RE: network buffer issues
|
Robert Craig
01/08/2008 4:12 PM
post4131
|
RE: network buffer issues
Hi Shannon:
I've got someone looking into this to see if we can reproduce it.
Sorry for not getting back to you sooner. Most people have been away for
the last couple of weeks on Christmas break.
Robert.
-----Original Message-----
From: Shannon Gu [mailto:shannon@devicescape.com]
Sent: Monday, January 07, 2008 4:11 PM
To: ionetmig-networking
Subject: Re: network buffer issues
============================================
> > #2. wpa_supplicnt in PSK mode stopped and network on board is basically
> crashed:
This is to attach the log file
Regards,
Shannon
_______________________________________________
io-net migration
http://community.qnx.com/sf/go/post4108
|
|
|
Weijie Zhang(deleted)
|
Re: RE: network buffer issues
|
Weijie Zhang(deleted)
01/09/2008 1:44 PM
post4153
|
Re: RE: network buffer issues
Hi, I have tried to reproduce the problem (WPA2 4-way handshake) but I can't reproduce it here. I have tried to
associate to an access point with the short preamble ability (as the one you are using has this ability) and also
another access point without the short preamble ability. Also besides the io-pkt-v4-hc (that is what you are running), I
have also tried another flavor like io-pkt-v6. They all work fine.
So I think it would be better to solve/understand the following two suspectable points. I also listed my wpa conf file
at the end and attached my log of accessing an WPA2+psk access point. Hope it be helpful.
1) the ifconfig got blocked (although ^C can make it free). I can't see it on my side either. Can you check the pidin to
see what is the status of the io-pkt? I noticed in you very first email, there is a script, (?) bt_detect? where you
have one io-net and two io-pkt instance running? If you intended to run more than one instances of io-net and/or io-pkt,
you have to use the prefix option, otherwise, neither of them work as expected. Anyway, please use pidin to see if any
suspectable point. I think we'd better to solve it first (ifconfig blocks) and go further onto such as wpa2.
==== the section in your script ?? ===============
/usr/sbin/random
#/proc/boot/io-net -ptcpip -ppppmgr &
/proc/boot/waitfor /dev/random 4
#/sbin/io-pkt-v4-hc random -ptcpip -pppmgr &
/platform/bin/wd &
/platform/bin/pm &
/platform/bin/doc &
/platform/bin/adi &
/proc/boot/waitfor /dev/pm/wifi/controls/enable 4
/sbin/pci-lge5200b
/proc/boot/waitfor /dev/pci 4
/sbin/io-pkt-v4-hc random -dath -ptcpip -pppmgr &
====================
2) I noticed there is more than 1 second between the broadcast and the first message sending to your access point. It
might indicate a problem of difficulty of accessing your access point. You may run "ifconfig ath0 scan" to see if you
can get the list of wireless devices around you and please check if the signal of your access point is too weak. Also
please check your access configuration settings and make sure it is correct. (I blindly suspect the signal is too weak
on your side and if it is the case, move closer to your access point and try again ...)
I list below the wpa.conf that I am using and the attached is the log file that the io-pkt+ath successfully associated
to my access point (Linksys Wireless-G WRT54GS).
=== my wpa2.conf ===
ctrl_interface=/var/run/wpa_supplicant
ctrl_interface_group=0
update_config=1
ap_scan=1
network={
ssid="linksys"
# proto=RSN
proto=WPA2
key_mgmt=WPA-PSK
# pairwise=CCMP TKIP
# group=CCMP TKIP
psk="LINKSYSWPAKEY"
}
===================
|
Attachment: |
mylog
10.41 KB
|
|
|
Shannon Gu
|
Re: RE: network buffer issues
|
Shannon Gu
01/09/2008 8:33 PM
post4164
|
Re: RE: network buffer issues
Hi Robert,
I think I need to give more background of this project. The nature of it is to port our wpa_supplicant to QNX platform
on top of io-pkt stack.
The hardware and the kernel come with it are from our customer; these network issues has been found on this particular
target, therefore I dont think they can be reproduced on a different type of target.
The network subnet envrionment is pretty standard, and works fine with all other devices that has been connected with --
including different type board/cpu/wireless driver, for example, we have put a PPC board running QNX 6 with io-net
network stack and a Broadcom chip on the same subnet, it can shoot big buffer packets no problem at all (up to the limit
65K); so I wouldn't suspect the subnet too much, (let me know if this makes sense).
I have requested our customer to reproduce the problem on same configuration target, but I have not gotten results back;
I suspect the problem is related to network configuration while building the kernel or some memory issues specific to
this board -- any advice regarding this perspective? Or have you encountered similar problems before?
Thanks a lot,
Regards,
Shannon
|
|
|
Dooeui Hong(deleted)
|
Re: RE: network buffer issues
|
Dooeui Hong(deleted)
01/10/2008 7:43 AM
post4168
|
Re: RE: network buffer issues
Dear team
I'm LGE engineer. I performed same test by using h/w which are same with the one Shannon used for his test.
And found that the board stop running during wpa_supplicant execution.
Following is my configuration and I'm attaching execution log.
[/etc]# cat shxy.conf
ctrl_interface=/usr/sbin/wpa_supplicant
network={
disabled=0
ssid="FCP_dooeui"
proto=WPA
key_mgmt=WPA-PSK
psk="secureDLS"
}
[/etc]#
[/]#
[/]# ifconfig ath0 ssid FCP_dooeui
[/]# ifconfig ath0 192.168.1.50 netmask 255.255.255.0
[/]# route add default 192.168.1.245
add net default: gateway 192.168.1.245
[/]#
[/]# wpa_supplicant -Dbsd -iath0 -c/etc/shxy.conf -ddd
|
|
|
|