Vineet Garg
|
route cache / socket route cache issue on io-net
|
Vineet Garg
07/02/2010 6:58 AM
post58315
|
route cache / socket route cache issue on io-net
Hi All
I am facing a problem where in case a UDP socket application is sending traffic and packets are using a routing entry to
select the next hop. In case i choose to change the gateway by deleting the old route entry and adding a new one, the
traffic stops working.
However, if i use "route change" command for doing this traffic continues to work fine.
Also, in case socket application exits and starts again, the traffic works.
So, somehow socket has some binding with routing entry which is not updated when route is deleted and added again/
Is there some route cache issue here? Existing design of my SW application requires using "route delete" followed by "
route add" command for changing the gateway.
Can someone pls shed some light on what is happening here?
Regards
Vineet
|
|
|
Vineet Garg
|
Re: route cache / socket route cache issue on io-net
|
Vineet Garg
07/02/2010 11:58 PM
post58360
|
Re: route cache / socket route cache issue on io-net
Found some reference to this issue on qnx site:
"If a route is in use when it's deleted, the routing entry is marked down and removed from the routing table, but the
resources associated with it won't be reclaimed until all references to it are released"
Can this be the cause of the problem?
In that case, Is there a workaround possible?
VG
|
|
|
Patrik Lahti
|
Re: route cache / socket route cache issue on io-net
|
Patrik Lahti
07/05/2010 9:49 PM
post58620
|
Re: route cache / socket route cache issue on io-net
I really have no time to verify/investigate, but I seem to recall that
sockets cache routes not just for speed, but also for stability. The
intent of the latter being to stick to the routing information while
forward progress can be made and there is no indication that the path is
broken (e.g. ICMP errors).
The reference you saw is just to the regular reference counting used
when a socket has a pointer to a route.
I remember you writing something about 'route change' working and that's
probably because the routing entry itself is actually changed in that
case. As opposed to the delete case where it is just marked as down,
which effectively means all who have it cached will have to re-lookup
the route. The re-lookup is probably triggered only in some situations.
Again, sorry I don't have the time to look into it at the moment but you
can check the source code if you want, that's the best advice I have at
the moment...
Cheers!
/P
On 10-07-02 11:58 PM, Vineet Garg wrote:
> Found some reference to this issue on qnx site: "If a route is in use
> when it's deleted, the routing entry is marked down and removed from
> the routing table, but the resources associated with it won't be
> reclaimed until all references to it are released"
>
> Can this be the cause of the problem?
>
> In that case, Is there a workaround possible?
>
> VG
>
>
>
> _______________________________________________
>
> Technology http://community.qnx.com/sf/go/post58360
>
|
|
|
Vineet Garg
|
Re: route cache / socket route cache issue on io-net
|
Vineet Garg
07/05/2010 10:16 PM
post58622
|
Re: route cache / socket route cache issue on io-net
Hi Patrik
Thanks for replying and it does make sense
However, couple of hours ago, I figured out the issue.
My route table had following two entries:
Dest GW
14.14.14.2 15.15.15.1
default 10.118.209.1
So, when I deleted the first entry and before adding the new entry, packets
started taking the default route. On adding a new route saying
14.14.14.2 15.15.15.2
Traffic continued to take the default route instead of choosing the now
available longest match.
I guess this means that route selection is done only when cached route is no
longer in the table.
Regarding source code, this is on io-net, so not sure if source code is
public for that too...
Regards
VG
On Tue, Jul 6, 2010 at 7:19 AM, Patrik Lahti <community-noreply@qnx.com>wrote:
> I really have no time to verify/investigate, but I seem to recall that
> sockets cache routes not just for speed, but also for stability. The
> intent of the latter being to stick to the routing information while
> forward progress can be made and there is no indication that the path is
> broken (e.g. ICMP errors).
>
> The reference you saw is just to the regular reference counting used
> when a socket has a pointer to a route.
>
> I remember you writing something about 'route change' working and that's
> probably because the routing entry itself is actually changed in that
> case. As opposed to the delete case where it is just marked as down,
> which effectively means all who have it cached will have to re-lookup
> the route. The re-lookup is probably triggered only in some situations.
>
> Again, sorry I don't have the time to look into it at the moment but you
> can check the source code if you want, that's the best advice I have at
> the moment...
>
> Cheers!
> /P
>
> On 10-07-02 11:58 PM, Vineet Garg wrote:
> > Found some reference to this issue on qnx site: "If a route is in use
> > when it's deleted, the routing entry is marked down and removed from
> > the routing table, but the resources associated with it won't be
> > reclaimed until all references to it are released"
> >
> > Can this be the cause of the problem?
> >
> > In that case, Is there a workaround possible?
> >
> > VG
> >
> >
> >
> > _______________________________________________
> >
> > Technology http://community.qnx.com/sf/go/post58360
> >
>
>
>
> _______________________________________________
>
> Technology
> http://community.qnx.com/sf/go/post58620
>
>
|
|
|
Yao Zhao(deleted)
|
Re: route cache / socket route cache issue on io-net
|
Yao Zhao(deleted)
07/06/2010 10:02 AM
post58673
|
Re: route cache / socket route cache issue on io-net
That is the way how route cache works.
New route cache is generated when first traffic is hit after the route is marked down.
Linux will flush route cache when this happened.
Look at io-pkt source I didn't see where it flush the flow caches when routing table changed.
Yao
> Hi Patrik
>
> Thanks for replying and it does make sense
>
> However, couple of hours ago, I figured out the issue.
>
> My route table had following two entries:
>
> Dest GW
> 14.14.14.2 15.15.15.1
> default 10.118.209.1
>
> So, when I deleted the first entry and before adding the new entry, packets
> started taking the default route. On adding a new route saying
>
> 14.14.14.2 15.15.15.2
>
> Traffic continued to take the default route instead of choosing the now
> available longest match.
>
> I guess this means that route selection is done only when cached route is no
> longer in the table.
>
> Regarding source code, this is on io-net, so not sure if source code is
> public for that too...
>
>
> Regards
> VG
>
> On Tue, Jul 6, 2010 at 7:19 AM, Patrik Lahti <community-noreply@qnx.com>wrote:
>
> > I really have no time to verify/investigate, but I seem to recall that
> > sockets cache routes not just for speed, but also for stability. The
> > intent of the latter being to stick to the routing information while
> > forward progress can be made and there is no indication that the path is
> > broken (e.g. ICMP errors).
> >
> > The reference you saw is just to the regular reference counting used
> > when a socket has a pointer to a route.
> >
> > I remember you writing something about 'route change' working and that's
> > probably because the routing entry itself is actually changed in that
> > case. As opposed to the delete case where it is just marked as down,
> > which effectively means all who have it cached will have to re-lookup
> > the route. The re-lookup is probably triggered only in some situations.
> >
> > Again, sorry I don't have the time to look into it at the moment but you
> > can check the source code if you want, that's the best advice I have at
> > the moment...
> >
> > Cheers!
> > /P
> >
> > On 10-07-02 11:58 PM, Vineet Garg wrote:
> > > Found some reference to this issue on qnx site: "If a route is in use
> > > when it's deleted, the routing entry is marked down and removed from
> > > the routing table, but the resources associated with it won't be
> > > reclaimed until all references to it are released"
> > >
> > > Can this be the cause of the problem?
> > >
> > > In that case, Is there a workaround possible?
> > >
> > > VG
> > >
> > >
> > >
> > > _______________________________________________
> > >
> > > Technology http://community.qnx.com/sf/go/post58360
> > >
> >
> >
> >
> > _______________________________________________
> >
> > Technology
> > http://community.qnx.com/sf/go/post58620
> >
> >
|
|
|
Vineet Garg
|
Re: route cache / socket route cache issue on io-net
|
Vineet Garg
07/07/2010 12:41 PM
post58850
|
Re: route cache / socket route cache issue on io-net
Thanks Yao for your inputs!
Another point here is that this issue is seen only for connection oriented
protocols and not connectionless ones.
VG
On Tue, Jul 6, 2010 at 7:32 PM, Yao Zhao <community-noreply@qnx.com> wrote:
> That is the way how route cache works.
> New route cache is generated when first traffic is hit after the route is
> marked down.
>
> Linux will flush route cache when this happened.
> Look at io-pkt source I didn't see where it flush the flow caches when
> routing table changed.
>
> Yao
>
> > Hi Patrik
> >
> > Thanks for replying and it does make sense
> >
> > However, couple of hours ago, I figured out the issue.
> >
> > My route table had following two entries:
> >
> > Dest GW
> > 14.14.14.2 15.15.15.1
> > default 10.118.209.1
> >
> > So, when I deleted the first entry and before adding the new entry,
> packets
> > started taking the default route. On adding a new route saying
> >
> > 14.14.14.2 15.15.15.2
> >
> > Traffic continued to take the default route instead of choosing the now
> > available longest match.
> >
> > I guess this means that route selection is done only when cached route is
> no
> > longer in the table.
> >
> > Regarding source code, this is on io-net, so not sure if source code is
> > public for that too...
> >
> >
> > Regards
> > VG
> >
> > On Tue, Jul 6, 2010 at 7:19 AM, Patrik Lahti <community-noreply@qnx.com
> >wrote:
> >
> > > I really have no time to verify/investigate, but I seem to recall that
> > > sockets cache routes not just for speed, but also for stability. The
> > > intent of the latter being to stick to the routing information while
> > > forward progress can be made and there is no indication that the path
> is
> > > broken (e.g. ICMP errors).
> > >
> > > The reference you saw is just to the regular reference counting used
> > > when a socket has a pointer to a route.
> > >
> > > I remember you writing something about 'route change' working and
> that's
> > > probably because the routing entry itself is actually changed in that
> > > case. As opposed to the delete case where it is just marked as down,
> > > which effectively means all who have it cached will have to re-lookup
> > > the route. The re-lookup is probably triggered only in some situations.
> > >
> > > Again, sorry I don't have the time to look into it at the moment but
> you
> > > can check the source code if you want, that's the best advice I have at
> > > the moment...
> > >
> > > Cheers!
> > > /P
> > >
> > > On 10-07-02 11:58 PM, Vineet Garg wrote:
> > > > Found some reference to this issue on qnx site: "If a route is in use
> > > > when it's deleted, the routing entry is marked down and removed from
> > > > the routing table, but the resources associated with it won't be
> > > > reclaimed until all references to it are released"
> > > >
> > > > Can this be the cause of the problem?
> > > >
> > > > In that case, Is there a workaround possible?
> > > >
> > > > VG
> > > >
> > > >
> > > >
> > > > _______________________________________________
> > > >
> > > > Technology http://community.qnx.com/sf/go/post58360
> > > >
> > >
> > >
> > >
> > >...
|
|
|
Yao Zhao(deleted)
|
Re: route cache / socket route cache issue on io-net
|
Yao Zhao(deleted)
07/07/2010 1:34 PM
post58855
|
Re: route cache / socket route cache issue on io-net
I can't remember udp socket will cache it but you have to know that normal route cache is srcip+dstip+tos or more, and
udp socket you can send to anywhere you want as it is connectionless so it may be useless but tcp is useful definitely.
yao
> Thanks Yao for your inputs!
>
> Another point here is that this issue is seen only for connection oriented
> protocols and not connectionless ones.
>
> VG
>
>
> On Tue, Jul 6, 2010 at 7:32 PM, Yao Zhao <community-noreply@qnx.com> wrote:
>
> > That is the way how route cache works.
> > New route cache is generated when first traffic is hit after the route is
> > marked down.
> >
> > Linux will flush route cache when this happened.
> > Look at io-pkt source I didn't see where it flush the flow caches when
> > routing table changed.
> >
> > Yao
> >
> > > Hi Patrik
> > >
> > > Thanks for replying and it does make sense
> > >
> > > However, couple of hours ago, I figured out the issue.
> > >
> > > My route table had following two entries:
> > >
> > > Dest GW
> > > 14.14.14.2 15.15.15.1
> > > default 10.118.209.1
> > >
> > > So, when I deleted the first entry and before adding the new entry,
> > packets
> > > started taking the default route. On adding a new route saying
> > >
> > > 14.14.14.2 15.15.15.2
> > >
> > > Traffic continued to take the default route instead of choosing the now
> > > available longest match.
> > >
> > > I guess this means that route selection is done only when cached route is
> > no
> > > longer in the table.
> > >
> > > Regarding source code, this is on io-net, so not sure if source code is
> > > public for that too...
> > >
> > >
> > > Regards
> > > VG
> > >
> > > On Tue, Jul 6, 2010 at 7:19 AM, Patrik Lahti <community-noreply@qnx.com
> > >wrote:
> > >
> > > > I really have no time to verify/investigate, but I seem to recall that
> > > > sockets cache routes not just for speed, but also for stability. The
> > > > intent of the latter being to stick to the routing information while
> > > > forward progress can be made and there is no indication that the path
> > is
> > > > broken (e.g. ICMP errors).
> > > >
> > > > The reference you saw is just to the regular reference counting used
> > > > when a socket has a pointer to a route.
> > > >
> > > > I remember you writing something about 'route change' working and
> > that's
> > > > probably because the routing entry itself is actually changed in that
> > > > case. As opposed to the delete case where it is just marked as down,
> > > > which effectively means all who have it cached will have to re-lookup
> > > > the route. The re-lookup is probably triggered only in some situations.
> > > >
> > > > Again, sorry I don't have the time to look into it at the moment but
> > you
> > > > can check the source code if you want, that's the best advice I have at
> > > > the moment...
> > > >
> > > > Cheers!
> > > > /P
> > > >
> > > > On 10-07-02 11:58 PM, Vineet Garg wrote:
> > > > > Found some reference to this issue on qnx site: "If a route is in use
> > > > > when it's deleted, the routing entry is marked down and removed from
> > > > > the routing table, but the resources...
View Full Message
|
|
|
|