After some feedback from certain customers who shall go nameless, I've
given some more thought on how to obtain the speed up we need to flush
more efficiently without penalizing the unmap side of the equation.

To recap, we've seen regressions (starting in 6.3.2) in process startup
time on some drivers/applications (devf is a good example).  This slow
down is due to additional cache flushing we introduced to ensure the
dcache was sane when making device mappings.  The reason is mostly
because these mappings are large and our cache flushing callouts flush
the entire region regardless of the size (and device mappings can get
quite large).  In previous metrics I saw up to almost a 1 seconds worth
of time for devf just to map it's flash array (almost all of which was
cache flushing 32megs of cache line by line).

The attached diff is only a concept and not really how I'd like to
implement it in the final form.  There are comments about where I think
the change is weak or inadequate and I'm looking for feedback on how to
improve it.  We're flushing/dumping the dcache in about 50ms which is
better and more inline with the mmap() performance from 6.3.0.

Effectively, the change implements a short cut that triggers if we're
attempting to flush the dcache for a region larger than 3x the size of
the cache (arbitrary amount based on some quick numbers on an ARM).  In
this case, instead of doing the line by line flush, the kernel will
pollute the cache instead (effectively doing a flush of the original
data for the whole cache) for the size of the dcache.

As mentioned in the comments, I'd really like to use a mapping the size
of the cache which maps to one page instead of abusing RAM the way I am
now, so if anyone has a good idea (other than ones I've already
commented) I'm all ears.

Feedback in general is also welcome.  I always enjoy being told I'm
crazy or silly :)


-- 
 Cheers,
    Adam

   QNX Software Systems
   [ amallory@qnx.com ]
   ---------------------------------------------------
   With a PC, I always felt limited by the software available.
   On Unix, I am limited only by my knowledge.
       --Peter J. Schoenster