Project Home
Project Home
Wiki
Wiki
Discussion Forums
Discussions
Project Information
Project Info
Forum Topic - reference filter for 6.4.0 ?: (13 Items)
   
reference filter for 6.4.0 ?  
Customer is using PPC w. 6.4.0

He wants to create a simple filter to receive a packet from a specific IP addres  and 'reflect' it back as fast as 
possible. Initially it is to compare performance with other OSes, but ultimately it is to forward (modified) packets.

His current implementation is a process that receives a packet (from Windows), tweaks it and resends it - via sockets. 
It is too slow - he's hoping to do a roundtrip in under 1ms, and he's seeing just over 2ms right now, which I understand
, given it is going up and down the entire stack, copy out, copy in etc.

The question is, is there a good piece of filter reference code (with documentation!) to start from, that he can use to 
create his very modest requirement ? 

Any pointers greatly appreciated...

Dave
RE: reference filter for 6.4.0 ?  
Checkout the "nraw" source
(http://community.qnx.com/integration/viewcvs/viewcvs.cgi/trunk/sys/lsm/
nraw/?root=core_networking&system=exsy1001)

He may not need the resmgr porting, but the nraw_hook.c probably
explained how it is done.

-xtang

> -----Original Message-----
> From: Dave Bott [mailto:community-noreply@qnx.com]
> Sent: March 10, 2009 12:00 PM
> To: technology-networking
> Subject: reference filter for 6.4.0 ?
> 
> Customer is using PPC w. 6.4.0
> 
> He wants to create a simple filter to receive a packet from a specific
IP
> addres  and 'reflect' it back as fast as possible. Initially it is to
> compare performance with other OSes, but ultimately it is to forward
> (modified) packets.
> 
> His current implementation is a process that receives a packet (from
> Windows), tweaks it and resends it - via sockets. It is too slow -
he's
> hoping to do a roundtrip in under 1ms, and he's seeing just over 2ms
right
> now, which I understand, given it is going up and down the entire
stack,
> copy out, copy in etc.
> 
> The question is, is there a good piece of filter reference code (with
> documentation!) to start from, that he can use to create his very
modest
> requirement ?
> 
> Any pointers greatly appreciated...
> 
> Dave
> 
> _______________________________________________
> Technology
> http://community.qnx.com/sf/go/post23990
RE: reference filter for 6.4.0 ?  
> receive a packet from a specific IP addres  and 'reflect' 
> it back as fast as possible. 

We always hurt in benchmarking small packets, because
of the overhead of the message passing and context
switch between the server (stack) and the client
(eg ttcp).  We can do gige wire rate with ttcp, we 
just require a larger payload size to amortize the 
overhead.

In the past, we have talked about creating a ttcp.so
dll application which would load into the stack, and
get rid of the context switch and message pass.  Because
there is no message pass through the kernel or context
switch, it would be very fast.  Think "fast-forwarding"
ttcp.

I think that's what you want.

--
aboyd
Re: reference filter for 6.4.0 ?  
On Tue, Mar 10, 2009 at 09:50:22AM -0400, Andrew Boyd wrote:
> 
> > receive a packet from a specific IP addres  and 'reflect' 
> > it back as fast as possible. 
> 
> We always hurt in benchmarking small packets, because
> of the overhead of the message passing and context
> switch between the server (stack) and the client
> (eg ttcp).  We can do gige wire rate with ttcp, we 
> just require a larger payload size to amortize the 
> overhead.
> 
> In the past, we have talked about creating a ttcp.so
> dll application which would load into the stack, and
> get rid of the context switch and message pass.  Because
> there is no message pass through the kernel or context
> switch, it would be very fast.  Think "fast-forwarding"
> ttcp.
> 
> I think that's what you want.

But simple reflection should be possible already via
the hooks provided.  As Xiaodan mentioned nraw is
probably a good starting point.

-seanb
Re: reference filter for 6.4.0 ?  
That sounds like a good idea - how soon can we get it done ?

Dave

Andrew Boyd wrote:
>
> > receive a packet from a specific IP addres  and 'reflect'
> > it back as fast as possible.
>
> We always hurt in benchmarking small packets, because
> of the overhead of the message passing and context
> switch between the server (stack) and the client
> (eg ttcp).  We can do gige wire rate with ttcp, we
> just require a larger payload size to amortize the
> overhead.
>
> In the past, we have talked about creating a ttcp.so
> dll application which would load into the stack, and
> get rid of the context switch and message pass.  Because
> there is no message pass through the kernel or context
> switch, it would be very fast.  Think "fast-forwarding"
> ttcp.
>
> I think that's what you want.
>
> --
> aboyd
>
>
> _______________________________________________
> Technology
> http://community.qnx.com/sf/go/post24013
>


-- 
Dave Bott (dbott@qnx.com)	Field Applications Engineer 
QNX Software Systems, Inc.	Cell:408 391-3535
San Jose CA

Join Foundry27 <http://community.qnx.com>;  - the new QNX developer forum. 

RE: reference filter for 6.4.0 ?  

> -----Original Message-----
> From: Dave Bott [mailto:community-noreply@qnx.com]
> Sent: March-10-09 11:28 AM
> To: technology-networking
> Subject: Re: reference filter for 6.4.0 ?
> 
> That sounds like a good idea - how soon can we get it done ?
> 

Yes great idea.  I`ll make sure everyone here reads it!

> Dave
> 
> Andrew Boyd wrote:
> >
> > > receive a packet from a specific IP addres  and 'reflect'
> > > it back as fast as possible.
> >
> > We always hurt in benchmarking small packets, because
> > of the overhead of the message passing and context
> > switch between the server (stack) and the client
> > (eg ttcp).  We can do gige wire rate with ttcp, we
> > just require a larger payload size to amortize the
> > overhead.
> >
> > In the past, we have talked about creating a ttcp.so
> > dll application which would load into the stack, and
> > get rid of the context switch and message pass.  Because
> > there is no message pass through the kernel or context
> > switch, it would be very fast.  Think "fast-forwarding"
> > ttcp.
> >
> > I think that's what you want.
> >
> > --
> > aboyd
> >
> >
> > _______________________________________________
> > Technology
> > http://community.qnx.com/sf/go/post24013
> >
> 
> 
> --
> Dave Bott (dbott@qnx.com)	Field Applications Engineer
> QNX Software Systems, Inc.	Cell:408 391-3535
> San Jose CA
> 
> Join Foundry27 <http://community.qnx.com>;  - the new QNX developer
> forum.
> 
> 
> 
> _______________________________________________
> Technology
> http://community.qnx.com/sf/go/post24029
> 
RE: reference filter for 6.4.0 ?  
There's code here
http://community.qnx.com/sf/wiki/do/viewPage/projects.networking/wiki/Fi
ltering_wiki_page

Showing how to connect into the packet filter infrastructure.  The only
problem is that it doesn't show how to turn the packet around and send
it out again (that's where the nraw code comes in).  This code gives you
full access to the mbuf so you can modify the packet and also indicate
(via the return code) whether or not you want the packet sent up the
stack. There are a few different places that you can hook into the stack
and, I'd suspect for this case, you want to be in the lowest level.
Sean added an input pfil_hook at the ether_input level (the last thing
that the driver calls before passing the packet up the stack).  
(see here
http://community.qnx.com/integration/viewcvs/viewcvs.cgi/trunk/sys/net/i
f_ethersubr.c?root=core_networking&rev=659&system=exsy1001&view=markup
for how the hook was added.)

This hook isn't retrievable through the normal "pfil_head_get" function.
Instead, you have to "know" where the hook is (it's a global define).

.
.
.
	pfh_inet = ð_input_pfil_hook;
	pfil_add_hook(input_hook, NULL, PFIL_IN | PFIL_WAITOK,
pfh_inet);
.
.
.

  Note that this hook entry isn't interface specific.  It will be called
by any interface which uses ether_input (and I can't think of any
interface which we currently support which doesn't call ether_input
right now).  You can check which interface the packet came in on in the
filter, though, since the interface pointer is passed as one of the
arguments.

	Robert.

-----Original Message-----
From: Dave Bott [mailto:community-noreply@qnx.com] 
Sent: Tuesday, March 10, 2009 12:00 AM
To: technology-networking
Subject: reference filter for 6.4.0 ?

Customer is using PPC w. 6.4.0

He wants to create a simple filter to receive a packet from a specific
IP addres  and 'reflect' it back as fast as possible. Initially it is to
compare performance with other OSes, but ultimately it is to forward
(modified) packets.

His current implementation is a process that receives a packet (from
Windows), tweaks it and resends it - via sockets. It is too slow - he's
hoping to do a roundtrip in under 1ms, and he's seeing just over 2ms
right now, which I understand, given it is going up and down the entire
stack, copy out, copy in etc.

The question is, is there a good piece of filter reference code (with
documentation!) to start from, that he can use to create his very modest
requirement ? 

Any pointers greatly appreciated...

Dave

_______________________________________________
Technology
http://community.qnx.com/sf/go/post23990
Re: RE: reference filter for 6.4.0 ?  
Thanks for the replies and the information.

I'm not sure I understand enough to use the info you sent Robert.
Neither I, nor my customer, is familiar  enoughwith this code to know what to do with the input hook that you decscribe.


However, looking at nraw, there is a sample app called 'pingd' which I built, and ran, after first installing the nraw 
filter. It seems to claim to do ping replies, as well as notice arps and IP packets. It only has an X86 build, so I 
don't know if it is endian-safe - I built a PPC and an X86 version of it anyway.

It certainly detects IP and ARP packets, but it did not seem to ping reply correctly - leastways, pinging the box (X86 
or PPC) did not get a good reply once this was enabled. It also seems to consume all packets, regardless of the 
originating IP address.

Did we write the pingd code, or test it ? Am I using it incorrectly ? It seems like this would be a nicer way to get 
what the customer wants - it must be faster via this interface surely, yet still living in a separate process... Or is 
this still a bad idea ? Writing a filter looks to be a tricky proposition...

Dave
RE: RE: reference filter for 6.4.0 ?  
Using the nraw interface will be faster since it doesn't involve sending
the packet up the stack.  It should also be somewhat faster and less
prone to jitter.

It delivers raw ethernet packets so the customer would have to set up
the appropriate byte pattern (or customize the code) to pick off the
packets that are required.  Hopefully xtang can comment on the pingd
example code (it is, I believe, code that has been written internally).

This will definitely be slower than implementing everything in the
filter since it is crossing the stack process boundary. The filter code
that's provided on the wiki covers the connection into the stack.  If
the nraw code doesn't work out, I'd say just start off using the code
example as is.  The filter point that I talked about in the previous
posting can be easily modifed later. The input / modify portion are
straight forward.  Unfortunately a "simple" example showing how to
create a transmit thread doesn't currently exist.  This is something
that is worthwhile adding to the example, but I can't commit to a time
for doing it right now.  The nraw code does have it in there, but it
obviously needs to be dug out.

	I'll also look at getting that sample code into the source
repository so that it can be checked out rather than having to do a copy
/ paste / create build environment thing.

		Robert.


-----Original Message-----
From: Dave Bott [mailto:community-noreply@qnx.com] 
Sent: Tuesday, March 10, 2009 4:14 PM
To: technology-networking
Subject: Re: RE: reference filter for 6.4.0 ?

Thanks for the replies and the information.

I'm not sure I understand enough to use the info you sent Robert.
Neither I, nor my customer, is familiar  enoughwith this code to know
what to do with the input hook that you decscribe.

However, looking at nraw, there is a sample app called 'pingd' which I
built, and ran, after first installing the nraw filter. It seems to
claim to do ping replies, as well as notice arps and IP packets. It only
has an X86 build, so I don't know if it is endian-safe - I built a PPC
and an X86 version of it anyway.

It certainly detects IP and ARP packets, but it did not seem to ping
reply correctly - leastways, pinging the box (X86 or PPC) did not get a
good reply once this was enabled. It also seems to consume all packets,
regardless of the originating IP address.

Did we write the pingd code, or test it ? Am I using it incorrectly ? It
seems like this would be a nicer way to get what the customer wants - it
must be faster via this interface surely, yet still living in a separate
process... Or is this still a bad idea ? Writing a filter looks to be a
tricky proposition...

Dave

_______________________________________________
Technology
http://community.qnx.com/sf/go/post24077
RE: RE: reference filter for 6.4.0 ?  
The "pingd" works for me, I double checked.

The "ip address" you passed to pingd, must be on same subnet, otherwise,
there won't be an ARP send out if it is pinged from some other node.

The "consume all packets", is also intentional. It suppose to show how
to exclusively access the packet. You can change the "open()" in pingd,
*not to* open(O_EXCL), (just O_RDWR), that way, nraw will let the packet
pass through.

If you customer also want performance, he should use a filter module
instead of nraw. 

-xtang

> -----Original Message-----
> From: Dave Bott [mailto:community-noreply@qnx.com]
> Sent: March 11, 2009 4:14 AM
> To: technology-networking
> Subject: Re: RE: reference filter for 6.4.0 ?
> 
> Thanks for the replies and the information.
> 
> I'm not sure I understand enough to use the info you sent Robert.
> Neither I, nor my customer, is familiar  enoughwith this code to know
what
> to do with the input hook that you decscribe.
> 
> However, looking at nraw, there is a sample app called 'pingd' which I
> built, and ran, after first installing the nraw filter. It seems to
claim
> to do ping replies, as well as notice arps and IP packets. It only has
an
> X86 build, so I don't know if it is endian-safe - I built a PPC and an
X86
> version of it anyway.
> 
> It certainly detects IP and ARP packets, but it did not seem to ping
reply
> correctly - leastways, pinging the box (X86 or PPC) did not get a good
> reply once this was enabled. It also seems to consume all packets,
> regardless of the originating IP address.
> 
> Did we write the pingd code, or test it ? Am I using it incorrectly ?
It
> seems like this would be a nicer way to get what the customer wants -
it
> must be faster via this interface surely, yet still living in a
separate
> process... Or is this still a bad idea ? Writing a filter looks to be
a
> tricky proposition...
> 
> Dave
> 
> _______________________________________________
> Technology
> http://community.qnx.com/sf/go/post24077
RE: RE: reference filter for 6.4.0 ?  
Thanks Xiaodan,
 
Yes, I realized that the IP address is the one it will respond to, not the one to expect packets from - that was not 
clear to me.
Thanks for the tip on how to pass packets - good stuff to document.
 
Dave

________________________________

From: Xiaodan Tang [mailto:community-noreply@qnx.com]
Sent: Wed 11/03/2009 8:29 AM
To: technology-networking
Subject: RE: RE: reference filter for 6.4.0 ?



The "pingd" works for me, I double checked.

The "ip address" you passed to pingd, must be on same subnet, otherwise,
there won't be an ARP send out if it is pinged from some other node.

The "consume all packets", is also intentional. It suppose to show how
to exclusively access the packet. You can change the "open()" in pingd,
*not to* open(O_EXCL), (just O_RDWR), that way, nraw will let the packet
pass through.

If you customer also want performance, he should use a filter module
instead of nraw.

-xtang

> -----Original Message-----
> From: Dave Bott [mailto:community-noreply@qnx.com]
> Sent: March 11, 2009 4:14 AM
> To: technology-networking
> Subject: Re: RE: reference filter for 6.4.0 ?
>
> Thanks for the replies and the information.
>
> I'm not sure I understand enough to use the info you sent Robert.
> Neither I, nor my customer, is familiar  enoughwith this code to know
what
> to do with the input hook that you decscribe.
>
> However, looking at nraw, there is a sample app called 'pingd' which I
> built, and ran, after first installing the nraw filter. It seems to
claim
> to do ping replies, as well as notice arps and IP packets. It only has
an
> X86 build, so I don't know if it is endian-safe - I built a PPC and an
X86
> version of it anyway.
>
> It certainly detects IP and ARP packets, but it did not seem to ping
reply
> correctly - leastways, pinging the box (X86 or PPC) did not get a good
> reply once this was enabled. It also seems to consume all packets,
> regardless of the originating IP address.
>
> Did we write the pingd code, or test it ? Am I using it incorrectly ?
It
> seems like this would be a nicer way to get what the customer wants -
it
> must be faster via this interface surely, yet still living in a
separate
> process... Or is this still a bad idea ? Writing a filter looks to be
a
> tricky proposition...
>
> Dave
>
> _______________________________________________
> Technology
> http://community.qnx.com/sf/go/post24077


_______________________________________________
Technology
http://community.qnx.com/sf/go/post24143



Attachment: Text winmail.dat 5.4 KB
Re: reference filter for 6.4.0 ?  
I've nearly got things working right - currently using nraw (it would 
help a lot if it were *documented*...)

I can bounce a UDP packet ck bain my process, but only if I set the 
return checksum to 0 (i.e. disable checksums) - I don't seem to be able 
to calculate it correctly.
I'm trying to use the cksum() code in the pingd example code, but that's 
for icmp/arp stuff - so is that the wrong sort of checksum ?

I'm also a bit unsure of exactly *what* to checksum - I think it's 
everything after the ethernet header - so is that everything after the 
first 14 bytes, setting the checksum field to 0, or is it more 
complicated than that ? I tried checksumming the incoming UDP packets 
(from some packet generator code the customer gave me) but I can't get 
it to match...

nraw is cool, but writing some notes on how to use patterns (I'd like to 
specify 2 to use together to restrict to a given port and IP, but don't 
know how), and some examples of how to create each packet type would 
make it really *useful*... Otherwise, we're back to the old adage : "a 
week in the lab can often save an hour in the library" :-)

Any help available ?

Thanks !

Dave

Xiaodan Tang wrote:
>
> The "pingd" works for me, I double checked.
>
> The "ip address" you passed to pingd, must be on same subnet, otherwise,
> there won't be an ARP send out if it is pinged from some other node.
>
> The "consume all packets", is also intentional. It suppose to show how
> to exclusively access the packet. You can change the "open()" in pingd,
> *not to* open(O_EXCL), (just O_RDWR), that way, nraw will let the packet
> pass through.
>
> If you customer also want performance, he should use a filter module
> instead of nraw.
>
> -xtang
>
> > -----Original Message-----
> > From: Dave Bott [mailto:community-noreply@qnx.com]
> > Sent: March 11, 2009 4:14 AM
> > To: technology-networking
> > Subject: Re: RE: reference filter for 6.4.0 ?
> >
> > Thanks for the replies and the information.
> >
> > I'm not sure I understand enough to use the info you sent Robert.
> > Neither I, nor my customer, is familiar  enoughwith this code to know
> what
> > to do with the input hook that you decscribe.
> >
> > However, looking at nraw, there is a sample app called 'pingd' which I
> > built, and ran, after first installing the nraw filter. It seems to
> claim
> > to do ping replies, as well as notice arps and IP packets. It only has
> an
> > X86 build, so I don't know if it is endian-safe - I built a PPC and an
> X86
> > version of it anyway.
> >
> > It certainly detects IP and ARP packets, but it did not seem to ping
> reply
> > correctly - leastways, pinging the box (X86 or PPC) did not get a good
> > reply once this was enabled. It also seems to consume all packets,
> > regardless of the originating IP address.
> >
> > Did we write the pingd code, or test it ? Am I using it incorrectly ?
> It
> > seems like this would be a nicer way to get what the customer wants -
> it
> > must be faster via this interface surely, yet still living in a
> separate
> > process... Or is this still a bad idea ? Writing a filter looks to be
> a
> > tricky proposition...
> >
> > Dave
> >
> > _______________________________________________
> > Technology
> > http://community.qnx.com/sf/go/post24077
>
>
> _______________________________________________
> Technology
> http://community.qnx.com/sf/go/post24143
>


-- 
Dave Bott (dbott@qnx.com)	Field Applications Engineer 
QNX Software Systems, Inc.	Cell:408 391-3535
San Jose CA

Join Foundry27 <http://community.qnx.com>;  - the new QNX developer forum. 

RE: reference filter for 6.4.0 ?  
Hi Dave:

UDP packets include checksums for both the IP header and the UDP packet

http://www.networksorcery.com/enp/protocol/ip.htm
http://www.networksorcery.com/enp/protocol/udp.htm

Nraw is brand new so hasn't been documented yet.  We should add a quick
blurb on it in the Networking wiki.  As a side note (and for emphasis),
there is no intent to ship it with product in binary format.  It is
intended to be used as a source code example for people interested in
how to connect into the stack and provide a resource manager interface.

		R.


-----Original Message-----
From: Dave Bott [mailto:community-noreply@qnx.com] 
Sent: Thursday, March 19, 2009 12:32 AM
To: technology-networking
Subject: Re: reference filter for 6.4.0 ?

I've nearly got things working right - currently using nraw (it would
help a lot if it were *documented*...)

I can bounce a UDP packet ck bain my process, but only if I set the
return checksum to 0 (i.e. disable checksums) - I don't seem to be able
to calculate it correctly.
I'm trying to use the cksum() code in the pingd example code, but that's
for icmp/arp stuff - so is that the wrong sort of checksum ?

I'm also a bit unsure of exactly *what* to checksum - I think it's
everything after the ethernet header - so is that everything after the
first 14 bytes, setting the checksum field to 0, or is it more
complicated than that ? I tried checksumming the incoming UDP packets
(from some packet generator code the customer gave me) but I can't get
it to match...

nraw is cool, but writing some notes on how to use patterns (I'd like to
specify 2 to use together to restrict to a given port and IP, but don't
know how), and some examples of how to create each packet type would
make it really *useful*... Otherwise, we're back to the old adage : "a
week in the lab can often save an hour in the library" :-)

Any help available ?

Thanks !

Dave

Xiaodan Tang wrote:
>
> The "pingd" works for me, I double checked.
>
> The "ip address" you passed to pingd, must be on same subnet, 
> otherwise, there won't be an ARP send out if it is pinged from some
other node.
>
> The "consume all packets", is also intentional. It suppose to show how

> to exclusively access the packet. You can change the "open()" in 
> pingd, *not to* open(O_EXCL), (just O_RDWR), that way, nraw will let 
> the packet pass through.
>
> If you customer also want performance, he should use a filter module 
> instead of nraw.
>
> -xtang
>
> > -----Original Message-----
> > From: Dave Bott [mailto:community-noreply@qnx.com]
> > Sent: March 11, 2009 4:14 AM
> > To: technology-networking
> > Subject: Re: RE: reference filter for 6.4.0 ?
> >
> > Thanks for the replies and the information.
> >
> > I'm not sure I understand enough to use the info you sent Robert.
> > Neither I, nor my customer, is familiar  enoughwith this code to 
> > know
> what
> > to do with the input hook that you decscribe.
> >
> > However, looking at nraw, there is a sample app called 'pingd' which

> > I built, and ran, after first installing the nraw filter. It seems 
> > to
> claim
> > to do ping replies, as well as notice arps and IP packets. It only 
> > has
> an
> > X86 build, so I don't know if it is endian-safe - I built a PPC and 
> > an
> X86
> > version of it anyway.
> >
> > It certainly detects IP and ARP packets, but it did not seem to ping
> reply
> > correctly - leastways, pinging the box (X86 or PPC) did not get a 
> > good reply once this was enabled. It also seems to consume all 
> > packets, regardless of the originating IP address.
> >
> > Did we write the pingd code, or test it ? Am I using it incorrectly
?
> It
> > seems like...
View Full Message