I’m posting this here (instead of the filesystem forum) since it happens on more than one filesystem type (which leads 
me to believe it’s a bug with mmap, not with the filesystems).

In my application, I open a file as read/write, mmap it in, compare it with new data, and only if the new data is 
different do I copy the changed data into the mmap’d buffer and call msync. The goal is to avoid unnecessary writes to 
our NOR flash filesystem.

However, on both our devf-generic NOR flash FFS3 and devb-eide compact flash IDE QNX4 filesystems, it appears that 
simply mmap’ing the file with PROT_WRITE is enough to force some kind of a flush out to the physical storage on munmap 
(even if msync was never called because the data didn’t change). If all I do is mmap with PROT_WRITE and munmap, the 
flash filesystem ends up reclaiming blocks (and periodically getting delayed for a couple seconds because of this).

The workaround is pretty straightforward: If I do my initial mmap with PROT_READ only, compare my contents, and then do 
an mprotect with PROT_WRITE only if the contents have changed, the unnecessary filesystem flushes don’t occur.

Interestingly, if I do my initial mmap with PROT_READ only, then do an mprotect with PROT_READ | PROT_WRITE, and then 
munmap (no msync), the filesystem data doesn’t appear to be flushed. Only if the initial mmap has the PROT_WRITE flag 
is the physical storage unnecessarily touched.

I’m attaching a sample program that runs through several different sets of mmap and mprotect flags (runs forever with 
100 runs in each set). You need to pass in a filename that it will write to, and optionally the string values that it 
will write. Here is the example output from our NOR flash filesystem (as previously mentioned, I was able to reproduce 
the relative differences on a compact flash IDE filesystem as well, though the absolute delays are much shorter):

# /tmp/z_test_mmap /persist/test_file.txt
Toggling between writing hello and hello to /persist/test_file.txt
Elapsed time 11339 msec for 100 cycles of mmap PROT_READ | PROT_WRITE, no additional mprotect
Elapsed time    63 msec for 100 cycles of mmap PROT_READ, mprotect PROT_READ | PROT_WRITE
Elapsed time    62 msec for 100 cycles of mmap PROT_READ, no additional mprotect (file is not actually updated even on a
 change)
Elapsed time 11495 msec for 100 cycles of mmap PROT_READ | PROT_WRITE, mprotect PROT_READ (file is not actually updated 
even on a change)
Elapsed time    66 msec for 100 cycles of mmap PROT_READ, mprotect PROT_READ | PROT_WRITE, but no mysnc (file is not 
actually updated even on a change)

# /tmp/z_test_mmap /persist/test_file.txt hello world
Toggling between writing hello and world to /persist/test_file.txt
Elapsed time 11594 msec for 100 cycles of mmap PROT_READ | PROT_WRITE, no additional mprotect
Elapsed time 11438 msec for 100 cycles of mmap PROT_READ, mprotect PROT_READ | PROT_WRITE
Elapsed time    78 msec for 100 cycles of mmap PROT_READ, no additional mprotect (file is not actually updated even on a
 change)
Elapsed time 13454 msec for 100 cycles of mmap PROT_READ | PROT_WRITE, mprotect PROT_READ (file is not actually updated 
even on a change)
Elapsed time    33 msec for 100 cycles of mmap PROT_READ, mprotect PROT_READ | PROT_WRITE, but no mysnc (file is not 
actually updated even on a change)