Project Home
Project Home		 Documents
Documents		 Wiki
Wiki		 Discussion Forums
Discussions		 Project Information
Project Info
QNX Operating System/Discussion/OSTech/[QNX7] x86_64 memory virtual <-> physical mapping upon allocation/List of Posts
Forum Topic - [QNX7] x86_64 memory virtual <-> physical mapping upon allocation: (5 Items)
   
Update
Davide Ancri
10/18/2017 12:08 PM
post118125
[QNX7] x86_64 memory virtual <-> physical mapping upon allocation   
hi all

I'm porting some code from qnx 6.5 to 7.0, where the kernel gets queried about the physical fragmentation of some memory
 just allocated with the following:

mmap64( 0, length, PROT_READ, MAP_SHARED | MAP_ANON | MAP_NOINIT, NOFD, 0 )

All the virtual address space obtained with the mmap64() call gets scanned, calling many times mem_offset64() to get the
 lenght of each physically contiguous chunk.

In qnx7, it seems that even if few physically contiguous blocks have been selected for serviceing a mmap64(), the phys -
> virtual mapping is performed in a reversed page-by-page order:
- first virtual page maps to last phys page
- second virtual page maps to second-last phys page
....
- last virtual page maps to first phys page

Here is the output of my mem_offset64() scanning iteration, for a 2 MB mmap64() :

   chunk#   phys_addr    len(hex)  len(dec)
        0   0001862C000       1000        4096
        1   0001862B000       1000        4096
        2   0001862A000       1000        4096
        3   00018629000       1000        4096
        4   00018628000       1000        4096
        5   00018627000       1000        4096
        6   00018626000       1000        4096
        ......
      506   00018419000       1000        4096
      507   00018418000       1000        4096
      508   00018417000       1000        4096
      509   00018416000       1000        4096
      510   00018415000       1000        4096
      511   00018414000       1000        4096

This creates some problems to me (lot of stuff to describe virt->phys mapping), while in qnx6 even large mmap64() calls 
gets satisfied with an reasonable number of physically contiguous chunks.

Is this a new memory allocation approach developed for qnx7? Or is this a symptom of problems/errors in my kernel build 
or config?

Davide
Elad Lahav
10/19/2017 9:58 PM
post118129
Re: [QNX7] x86_64 memory virtual <-> physical mapping upon allocation   
The new memory manager on 7.0 uses on-demand paging for all regular mappings. This means that a physical page is only 
allocated when the relevant virtual page is first accessed. There is also no guarantee on such mappings that the 
physical page will be the same the next time it is accessed.
To satisfy on-demand paging the physical allocator is optimized for single-page allocations, which, to reduce 
fragmentation, are carved from the top down (hence the reverse order, at least at the early stages after boot).
You must never rely on the physical addresses of normal mappings, and it is not even clear to me how you got the result 
from mem_offset(), unless you ignored the return code (should have been -1).
For memory that has stricter requirements on physical backing, such as contiguity and static assignment of physical 
addresses, we provide the MAP_PHYS flag for mmap() and the shm_ctl() API. The physical allocator uses a different 
mechanism to allocate those, and it starts from the bottom going up. This way, the domains of single-page allocations 
and multi-page allocations are kept separate, such that the former does not fragment the latter.

--Elad
Davide Ancri
10/20/2017 8:57 AM
post118133
Re: [QNX7] x86_64 memory virtual <-> physical mapping upon allocation   
hi Elad, and thanks a lot for you effective answer.

We run qnx6 with "-mL" option to procnto (= superlock all memory), and of course I kept the same option to qnx7 kernel.
This is the reason for mem_offset() calls success right after mmap(), and it also allow to share physical addresses with
 other cores on the system but running outside of the qnx kernel control.

Anyway, what you described explains clearly what I get with my experiments.

Calling many times mmap() with MAP_PHYS, splitting the total amount of memory I need into a reasonable number of 
physically contiguous chunks, will be a good workaround for me.

The reason for avoiding a single big mmap() with MAP_PHYS is trying to improve the overall availability of physically 
contiguous memory.

Davide
Elad Lahav
10/20/2017 9:48 AM
post118134
Re: [QNX7] x86_64 memory virtual <-> physical mapping upon allocation   
Take a look at shm_ctl(). With SHMCTL_ANON only (i.e., without SHMCTL_PHYS) you will get "mostly contiguous memory", 
which may be the best choice in your case. "Mostly contiguous" means that the allocator will try to use one chunk for 
the entire region, and, if it fails, will use whatever contiguous chunks are available in order of physical address, 
which results in a very fast allocation scheme.

--Elad
Davide Ancri
10/20/2017 10:18 AM
post118136
Re: [QNX7] x86_64 memory virtual <-> physical mapping upon allocation   
Very good info, I will try soon.
Thanks again, Elad.
Davide

Return


  A subsidiary of BlackBerry             All content ©2004-2013, QNX Software Systems