Project Home
Project Home
Wiki
Wiki
Discussion Forums
Discussions
Project Information
Project Info
Forum Topic - route cache / socket route cache issue on io-net: (7 Items)
   
route cache / socket route cache issue on io-net  
Hi All

I am facing a problem where in case a UDP socket application is sending traffic and packets are using a routing entry to
 select the next hop. In case i choose to change the gateway by deleting the old route entry and adding a new one, the 
traffic stops working.
However, if i use "route change" command for doing this traffic continues to work fine.

Also, in case socket application exits and starts again, the traffic works.

So, somehow socket has some binding with routing entry which is not updated when route is deleted and added again/

Is there some route cache issue here? Existing design of my SW application requires using "route delete" followed by "
route add" command for changing the gateway. 

Can someone pls shed some light on what is happening here?

Regards
Vineet
Re: route cache / socket route cache issue on io-net  
Found some reference to this issue on qnx site: 
"If a route is in use when it's deleted, the routing entry is marked down and removed from the routing table, but the 
resources associated with it won't be reclaimed until all references to it are released"

Can this be the cause of the problem?

In that case, Is there a workaround possible?

VG
Re: route cache / socket route cache issue on io-net  
I really have no time to verify/investigate, but I seem to recall that 
sockets cache routes not just for speed, but also for stability. The 
intent of the latter being to stick to the routing information while 
forward progress can be made and there is no indication that the path is 
broken (e.g. ICMP errors).

The reference you saw is just to the regular reference counting used 
when a socket has a pointer to a route.

I remember you writing something about 'route change' working and that's 
probably because the routing entry itself is actually changed in that 
case. As opposed to the delete case where it is just marked as down, 
which effectively means all who have it cached will have to re-lookup 
the route. The re-lookup is probably triggered only in some situations.

Again, sorry I don't have the time to look into it at the moment but you 
can check the source code if you want, that's the best advice I have at 
the moment...

Cheers!
/P

On 10-07-02 11:58 PM, Vineet Garg wrote:
> Found some reference to this issue on qnx site: "If a route is in use
> when it's deleted, the routing entry is marked down and removed from
> the routing table, but the resources associated with it won't be
> reclaimed until all references to it are released"
>
> Can this be the cause of the problem?
>
> In that case, Is there a workaround possible?
>
> VG
>
>
>
> _______________________________________________
>
> Technology http://community.qnx.com/sf/go/post58360
>
Re: route cache / socket route cache issue on io-net  
Hi Patrik

Thanks for replying and it does make sense

However, couple of hours ago, I figured out the issue.

My route table had following two entries:

Dest               GW
14.14.14.2     15.15.15.1
default           10.118.209.1

So, when I deleted the first entry and before adding the new entry, packets
started taking the default route. On adding a new route saying

14.14.14.2    15.15.15.2

Traffic continued to take the default route instead of choosing the now
available longest match.

I guess this means that route selection is done only when cached route is no
longer in the table.

Regarding source code, this is on io-net, so not sure if source code is
public for that too...


Regards
VG

On Tue, Jul 6, 2010 at 7:19 AM, Patrik Lahti <community-noreply@qnx.com>wrote:

> I really have no time to verify/investigate, but I seem to recall that
> sockets cache routes not just for speed, but also for stability. The
> intent of the latter being to stick to the routing information while
> forward progress can be made and there is no indication that the path is
> broken (e.g. ICMP errors).
>
> The reference you saw is just to the regular reference counting used
> when a socket has a pointer to a route.
>
> I remember you writing something about 'route change' working and that's
> probably because the routing entry itself is actually changed in that
> case. As opposed to the delete case where it is just marked as down,
> which effectively means all who have it cached will have to re-lookup
> the route. The re-lookup is probably triggered only in some situations.
>
> Again, sorry I don't have the time to look into it at the moment but you
> can check the source code if you want, that's the best advice I have at
> the moment...
>
> Cheers!
> /P
>
> On 10-07-02 11:58 PM, Vineet Garg wrote:
> > Found some reference to this issue on qnx site: "If a route is in use
> > when it's deleted, the routing entry is marked down and removed from
> > the routing table, but the resources associated with it won't be
> > reclaimed until all references to it are released"
> >
> > Can this be the cause of the problem?
> >
> > In that case, Is there a workaround possible?
> >
> > VG
> >
> >
> >
> > _______________________________________________
> >
> > Technology http://community.qnx.com/sf/go/post58360
> >
>
>
>
> _______________________________________________
>
> Technology
> http://community.qnx.com/sf/go/post58620
>
>
Re: route cache / socket route cache issue on io-net  
That is the way how route cache works.
New route cache is generated when first traffic is hit after the route is marked down.

Linux will flush route cache when this happened.
Look at io-pkt source I didn't see where it flush the flow caches when routing table changed.

Yao

> Hi Patrik
> 
> Thanks for replying and it does make sense
> 
> However, couple of hours ago, I figured out the issue.
> 
> My route table had following two entries:
> 
> Dest               GW
> 14.14.14.2     15.15.15.1
> default           10.118.209.1
> 
> So, when I deleted the first entry and before adding the new entry, packets
> started taking the default route. On adding a new route saying
> 
> 14.14.14.2    15.15.15.2
> 
> Traffic continued to take the default route instead of choosing the now
> available longest match.
> 
> I guess this means that route selection is done only when cached route is no
> longer in the table.
> 
> Regarding source code, this is on io-net, so not sure if source code is
> public for that too...
> 
> 
> Regards
> VG
> 
> On Tue, Jul 6, 2010 at 7:19 AM, Patrik Lahti <community-noreply@qnx.com>wrote:
> 
> > I really have no time to verify/investigate, but I seem to recall that
> > sockets cache routes not just for speed, but also for stability. The
> > intent of the latter being to stick to the routing information while
> > forward progress can be made and there is no indication that the path is
> > broken (e.g. ICMP errors).
> >
> > The reference you saw is just to the regular reference counting used
> > when a socket has a pointer to a route.
> >
> > I remember you writing something about 'route change' working and that's
> > probably because the routing entry itself is actually changed in that
> > case. As opposed to the delete case where it is just marked as down,
> > which effectively means all who have it cached will have to re-lookup
> > the route. The re-lookup is probably triggered only in some situations.
> >
> > Again, sorry I don't have the time to look into it at the moment but you
> > can check the source code if you want, that's the best advice I have at
> > the moment...
> >
> > Cheers!
> > /P
> >
> > On 10-07-02 11:58 PM, Vineet Garg wrote:
> > > Found some reference to this issue on qnx site: "If a route is in use
> > > when it's deleted, the routing entry is marked down and removed from
> > > the routing table, but the resources associated with it won't be
> > > reclaimed until all references to it are released"
> > >
> > > Can this be the cause of the problem?
> > >
> > > In that case, Is there a workaround possible?
> > >
> > > VG
> > >
> > >
> > >
> > > _______________________________________________
> > >
> > > Technology http://community.qnx.com/sf/go/post58360
> > >
> >
> >
> >
> > _______________________________________________
> >
> > Technology
> > http://community.qnx.com/sf/go/post58620
> >
> >


Re: route cache / socket route cache issue on io-net  
Thanks Yao for your inputs!

Another point here is that this issue is seen only for connection oriented
protocols and not connectionless ones.

VG


On Tue, Jul 6, 2010 at 7:32 PM, Yao Zhao <community-noreply@qnx.com> wrote:

> That is the way how route cache works.
> New route cache is generated when first traffic is hit after the route is
> marked down.
>
> Linux will flush route cache when this happened.
> Look at io-pkt source I didn't see where it flush the flow caches when
> routing table changed.
>
> Yao
>
> > Hi Patrik
> >
> > Thanks for replying and it does make sense
> >
> > However, couple of hours ago, I figured out the issue.
> >
> > My route table had following two entries:
> >
> > Dest               GW
> > 14.14.14.2     15.15.15.1
> > default           10.118.209.1
> >
> > So, when I deleted the first entry and before adding the new entry,
> packets
> > started taking the default route. On adding a new route saying
> >
> > 14.14.14.2    15.15.15.2
> >
> > Traffic continued to take the default route instead of choosing the now
> > available longest match.
> >
> > I guess this means that route selection is done only when cached route is
> no
> > longer in the table.
> >
> > Regarding source code, this is on io-net, so not sure if source code is
> > public for that too...
> >
> >
> > Regards
> > VG
> >
> > On Tue, Jul 6, 2010 at 7:19 AM, Patrik Lahti <community-noreply@qnx.com
> >wrote:
> >
> > > I really have no time to verify/investigate, but I seem to recall that
> > > sockets cache routes not just for speed, but also for stability. The
> > > intent of the latter being to stick to the routing information while
> > > forward progress can be made and there is no indication that the path
> is
> > > broken (e.g. ICMP errors).
> > >
> > > The reference you saw is just to the regular reference counting used
> > > when a socket has a pointer to a route.
> > >
> > > I remember you writing something about 'route change' working and
> that's
> > > probably because the routing entry itself is actually changed in that
> > > case. As opposed to the delete case where it is just marked as down,
> > > which effectively means all who have it cached will have to re-lookup
> > > the route. The re-lookup is probably triggered only in some situations.
> > >
> > > Again, sorry I don't have the time to look into it at the moment but
> you
> > > can check the source code if you want, that's the best advice I have at
> > > the moment...
> > >
> > > Cheers!
> > > /P
> > >
> > > On 10-07-02 11:58 PM, Vineet Garg wrote:
> > > > Found some reference to this issue on qnx site: "If a route is in use
> > > > when it's deleted, the routing entry is marked down and removed from
> > > > the routing table, but the resources associated with it won't be
> > > > reclaimed until all references to it are released"
> > > >
> > > > Can this be the cause of the problem?
> > > >
> > > > In that case, Is there a workaround possible?
> > > >
> > > > VG
> > > >
> > > >
> > > >
> > > > _______________________________________________
> > > >
> > > > Technology http://community.qnx.com/sf/go/post58360
> > > >
> > >
> > >
> > >
> > >...
Re: route cache / socket route cache issue on io-net  
I can't remember udp socket will cache it but you have to know that normal route cache is srcip+dstip+tos or more, and 
udp socket you can send to anywhere you want as it is connectionless so it may be useless but tcp is useful definitely.

yao
> Thanks Yao for your inputs!
> 
> Another point here is that this issue is seen only for connection oriented
> protocols and not connectionless ones.
> 
> VG
> 
> 
> On Tue, Jul 6, 2010 at 7:32 PM, Yao Zhao <community-noreply@qnx.com> wrote:
> 
> > That is the way how route cache works.
> > New route cache is generated when first traffic is hit after the route is
> > marked down.
> >
> > Linux will flush route cache when this happened.
> > Look at io-pkt source I didn't see where it flush the flow caches when
> > routing table changed.
> >
> > Yao
> >
> > > Hi Patrik
> > >
> > > Thanks for replying and it does make sense
> > >
> > > However, couple of hours ago, I figured out the issue.
> > >
> > > My route table had following two entries:
> > >
> > > Dest               GW
> > > 14.14.14.2     15.15.15.1
> > > default           10.118.209.1
> > >
> > > So, when I deleted the first entry and before adding the new entry,
> > packets
> > > started taking the default route. On adding a new route saying
> > >
> > > 14.14.14.2    15.15.15.2
> > >
> > > Traffic continued to take the default route instead of choosing the now
> > > available longest match.
> > >
> > > I guess this means that route selection is done only when cached route is
> > no
> > > longer in the table.
> > >
> > > Regarding source code, this is on io-net, so not sure if source code is
> > > public for that too...
> > >
> > >
> > > Regards
> > > VG
> > >
> > > On Tue, Jul 6, 2010 at 7:19 AM, Patrik Lahti <community-noreply@qnx.com
> > >wrote:
> > >
> > > > I really have no time to verify/investigate, but I seem to recall that
> > > > sockets cache routes not just for speed, but also for stability. The
> > > > intent of the latter being to stick to the routing information while
> > > > forward progress can be made and there is no indication that the path
> > is
> > > > broken (e.g. ICMP errors).
> > > >
> > > > The reference you saw is just to the regular reference counting used
> > > > when a socket has a pointer to a route.
> > > >
> > > > I remember you writing something about 'route change' working and
> > that's
> > > > probably because the routing entry itself is actually changed in that
> > > > case. As opposed to the delete case where it is just marked as down,
> > > > which effectively means all who have it cached will have to re-lookup
> > > > the route. The re-lookup is probably triggered only in some situations.
> > > >
> > > > Again, sorry I don't have the time to look into it at the moment but
> > you
> > > > can check the source code if you want, that's the best advice I have at
> > > > the moment...
> > > >
> > > > Cheers!
> > > > /P
> > > >
> > > > On 10-07-02 11:58 PM, Vineet Garg wrote:
> > > > > Found some reference to this issue on qnx site: "If a route is in use
> > > > > when it's deleted, the routing entry is marked down and removed from
> > > > > the routing table, but the resources...
View Full Message