Project Home
Project Home
Wiki
Wiki
Discussion Forums
Discussions
Project Information
Project Info
Forum Topic - QNET-over-IP vs. QNET-over-RAW: (5 Items)
   
QNET-over-IP vs. QNET-over-RAW  
I am planning to use QNET-over-IP in our product. Are there any problems with this way of using QNET? Is this fully 
supported configuration (e.g. not experimental)? And what disadvantages it would have in comparison with QNET-over-RAW (
"normal" QNET, connected directly to the Networking drivers)?

I assume, performance would be one of them. But, it is not big issue for us regarding this transport. The primary 
concern is reliability of message delivery (when IP links are re-established) and stability (failure recovery) of the 
configuration in the environment where failures (including communication paths) may happen randomly and often.

The operational runtime environment:
QNX-6.4.1 running on SMP mode (8 CPU cores) with two parallel networking stacks (only one of them uses QNET). The 
Networking Stacks operates under heavy load (UDP/TCP streams).

Thank you.
RE: QNET-over-IP vs. QNET-over-RAW  
I suspect that at least 99% of our customers use 
qnet over "raw" ethernet (default transport mode).

The only operational reason to run qnet with IP 
encapsulation is if you have to route the qnet 
packets between subnets - ie you have to "go through"
a router.

There is some performance hit with qnet over IP.  The
name resolution process is different, too.  However 
the protocol for session connection establishment and
maintenance and the protocol for data transfer are
the same - the packets are just wrapped with an IP
header.

SMP will make no difference to the above.

--
aboyd
Re: RE: QNET-over-IP vs. QNET-over-RAW  
Thank you, Andrew.
So, what are you saying -- you do not expect that we would face major problems if we use QNET-over-IP?

We, like vast majority of QNX customers, were using QNX-over-Ethernet for years. This was a logical choice for the 
network configuration we had in the previous products -- single (as seen by the Networking Stack and the Application 
code) network interface on top of which both TCP/IP and QNET were riding. The Networking driver was modified to perform 
link switching between multiple NICs. Thus, for any software, which was running above the driver, there was only one 
physical link. All others were hidden and disabled by the driver.

In the last product (for certain reasons) we are using different configuration -- redundant and all active links. The 
problem with this -- when we are using two different transports (TCP/IP and QNET), they require different mechanism of 
link switching and synchronization. TCP/IP is sitting on different subnets (one subnet per physical link) and switching 
is performed by applying the third (virtual) subnet (in form of IP alias) to one of the underlying subnets. QNET, from 
the other hands, uses a link grouping. My experiments showed that even though QNET suppose to do load balancing over all
 available links, it operates only with one of them at a time. When that link fails, it requires some time (20 seconds 
or something) to detect this failure and switch to the different link. Thus, there are time windows when both TCP/IP and
 QNET transport may not be in sync (even if timeout is reduced). The logical solution for this problem (as I see it) -- 
to use QNET-over-IP. This allows to treat QNET as a "regular" Stack User, thus if we monitor and deal with IP link 
failures, the problem with QNET is resolved automatically.

Of course, we could monitor and switch IP and QNET independently (that was the original plan). But, this doesn't seem as
 a neat solution.
RE: RE: QNET-over-IP vs. QNET-over-RAW  
> even though QNET suppose to do load balancing over 
> all available links, it operates only with one of 
> them at a time

I'm going to guess that you plugged all the links into
the same hub.  In this case, qnet notices that there 
are network loops - packets that go out interface en0
are received on en1 and en2.  In this configuration
qnet just uses en0, because it only detects one network.
Other nodes on the network will learn of this node
via the mac address of en0 - not en1, or en2, or some
combination of them.  In the this configuration, you 
really only have one network.

Now, if you plugged en0 into one hub, and en1 into
another hub, of two completely separate networks,
then if there is another QNX node which is also 
connected to both networks with two interfaces, 
then load balancing will occur, because you really 
have two separate networks, which can carry two 
different packets simultaneously.

All of the above is what occurs with multiple
interfaces with qnet-over-ethernet.

If you don't like the above behaviour, then 
qnet-over-ip should allow you to control what 
happens with multiple links via ip routing.

--
aboyd
Re: RE: RE: QNET-over-IP vs. QNET-over-RAW  
Andrew, thank you, again.
You wrote: "I'm going to guess that you plugged all the links into the same hub."
That is true. All links are connected to the same switch which creates "ugly" loops. Also, whatever you described is 
absolutely correct -- if links do not have loops (connected to different networks) then all of them are used (load 
balancing mode) by QNET.

Unfortunately, the network topology we currently have, cannot be changed. For several reasons, including the fact, that 
such network configuration allows much better to handle System partitioning.

Anyway, so far QNET-over-IP works for me. I am trying to use it during last month or so. I just wanted to be sure that 
that is "legal" configuration which QNX supports and does not discourage to use, when it is appropriate.