Project Home
Project Home
Documents
Documents
Wiki
Wiki
Discussion Forums
Discussions
Project Information
Project Info
Forum Topic - io-graphics increasing hog of CPU: (8 Items)
   
io-graphics increasing hog of CPU  
We have a product in the field where we are seeing io-graphics become a CPU hog over time.

In this product we have a fixed number of applications that are running under Photon.
The system is "stable" in the sense that the number of applications, and what they are doing, essentially doesn't change
.

However we are seeing that over time (a period of weeks) the CPU usage of the system increases to the point where it 
becomes un-usable and needs to be re-booted. Typically the system will be stable for many days/weeks and then the CPU 
usage will begin to increase over a period of days. Once it starts increasing it takes a couple of days to become 
unusable.

Our initial analysis shows that when the CPU usage starts increasing, the main culprit is actually io-graphics. It goes 
from using negligible amounts of CPU to 10%,20%,30% of the CPU or more. None of our other processes show any increase in
 memory or CPU usage.

We are running 6.3.2A on x86 (PC Platform).

I know that we haven't given enough information for anyone to be able to actually help us track this down, but we do 
have the following questions:

- Has anyone seen anything similar to this before?
- What additional information do we need to capture to try to isolate this problem, and how do we capture it?
- Any other hints for how to get to the bottom of this?


Thanks in advance for any ideas.
Re: io-graphics increasing hog of CPU  
I've never heard of this before, but I would gather a kernel trace when the system is 'healthy' and another when the
system is starting to show symptoms.

A good line to use would be

tracelogger -S32M -M -w -v -c

make sure the system isn't so sick that you can't retrieve the logfile! :-)

Robert Rutherford wrote:
> We have a product in the field where we are seeing io-graphics become a CPU hog over time.
> 
> In this product we have a fixed number of applications that are running under Photon.
> The system is "stable" in the sense that the number of applications, and what they are doing, essentially doesn't 
change.
> 
> However we are seeing that over time (a period of weeks) the CPU usage of the system increases to the point where it 
becomes un-usable and needs to be re-booted. Typically the system will be stable for many days/weeks and then the CPU 
usage will begin to increase over a period of days. Once it starts increasing it takes a couple of days to become 
unusable.
> 
> Our initial analysis shows that when the CPU usage starts increasing, the main culprit is actually io-graphics. It 
goes from using negligible amounts of CPU to 10%,20%,30% of the CPU or more. None of our other processes show any 
increase in memory or CPU usage.
> 
> We are running 6.3.2A on x86 (PC Platform).
> 
> I know that we haven't given enough information for anyone to be able to actually help us track this down, but we do 
have the following questions:
> 
> - Has anyone seen anything similar to this before?
> - What additional information do we need to capture to try to isolate this problem, and how do we capture it?
> - Any other hints for how to get to the bottom of this?
> 
> 
> Thanks in advance for any ideas.
> 
> 
> _______________________________________________
> Photon microGUI
> http://community.qnx.com/sf/go/post32798
> 

-- 
cburgess@qnx.com
Re: io-graphics increasing hog of CPU  
> I've never heard of this before, but I would gather a kernel trace when the 
> system is 'healthy' and another when the
> system is starting to show symptoms.
> 

Thanks for the suggestion Colin.
We've captured the kernel trace from a 'healthy' system and we are now waiting for the symptoms to re-appear.
Re: io-graphics increasing hog of CPU  
I don't think io-graphics in itself is the problem - it may be the graphics driver that is loaded by io-graphics. Thus, 
you should provide info on the QNX version you are using and the graphics driver. If possible, you could try a different
 hardware with different graphics driver to see if it's really the driver, or io-graphics itself.

It may also be worthwhile to try with the latest QNX version (6.4.1), especially if you are on 6.3.x, because all 
graphics drivers have been re-worked in 6.4 to work with the new io-display server.


- Malte
Re: io-graphics increasing hog of CPU  
Thanks Malte.

We are running 6.3.2A and the graphics driver is the rage driver (devg-rage.so).

We will look at trying the flat or vesa drivers to see if the problem still occurs.

It is not possible to try different hardware.

> I don't think io-graphics in itself is the problem - it may be the graphics 
> driver that is loaded by io-graphics. Thus, you should provide info on the QNX
>  version you are using and the graphics driver. If possible, you could try a 
> different hardware with different graphics driver to see if it's really the 
> driver, or io-graphics itself.
> 
> It may also be worthwhile to try with the latest QNX version (6.4.1), 
> especially if you are on 6.3.x, because all graphics drivers have been re-
> worked in 6.4 to work with the new io-display server.
> 
> 
> - Malte


RE: io-graphics increasing hog of CPU  
Do any of these applications create/free memory as they run?

When you see this problem occur how much memory is left in the system?

Is there any type of error messages in sloginfo?

Thanks

-----Original Message-----
From: Robert Rutherford [mailto:community-noreply@qnx.com] 
Sent: June 30, 2009 3:58 AM
To: photon-graphics
Subject: io-graphics increasing hog of CPU

We have a product in the field where we are seeing io-graphics become a
CPU hog over time.

In this product we have a fixed number of applications that are running
under Photon.
The system is "stable" in the sense that the number of applications, and
what they are doing, essentially doesn't change.

However we are seeing that over time (a period of weeks) the CPU usage
of the system increases to the point where it becomes un-usable and
needs to be re-booted. Typically the system will be stable for many
days/weeks and then the CPU usage will begin to increase over a period
of days. Once it starts increasing it takes a couple of days to become
unusable.

Our initial analysis shows that when the CPU usage starts increasing,
the main culprit is actually io-graphics. It goes from using negligible
amounts of CPU to 10%,20%,30% of the CPU or more. None of our other
processes show any increase in memory or CPU usage.

We are running 6.3.2A on x86 (PC Platform).

I know that we haven't given enough information for anyone to be able to
actually help us track this down, but we do have the following
questions:

- Has anyone seen anything similar to this before?
- What additional information do we need to capture to try to isolate
this problem, and how do we capture it?
- Any other hints for how to get to the bottom of this?


Thanks in advance for any ideas.


_______________________________________________
Photon microGUI
http://community.qnx.com/sf/go/post32798
Re: RE: io-graphics increasing hog of CPU  
> 
> Do any of these applications create/free memory as they run?
> 
> When you see this problem occur how much memory is left in the system?
> 
> Is there any type of error messages in sloginfo?
> 
> Thanks
> 

There is still plenty of free memory left in the system (we are at about 50% total memory utilization in a system with 
2GB of RAM). None of our apps show any sign of a memory leak (their memory usage hasn't significantly increased over 
time).

Nothing relevant in sloginfo.

Thanks for the ideas though. Any others?

Re: RE: io-graphics increasing hog of CPU  
> Thanks for the ideas though. Any others?


It might be this:
http://community.qnx.com/sf/discussion/do/listPosts/projects.networking/discussion.technology.topc3292

I wouldn't care so much about who hogs (io-grahics or io-...), just check if you ran into PR is 39687.

Our PR for this says:
"select()/poll() with a timeout setting of 0 causes delays/ memory leak"


Hope this helps.
Jeevan