|
Malte Mundt(deleted)
|
Shared Memory object - map it at a fixed virtual address
|
|
Malte Mundt(deleted)
04/24/2009 5:08 AM
post27959
|
Shared Memory object - map it at a fixed virtual address
Hi,
a potential customer considers moving to QNX from an 'OS' that doesn't know virtual memory. Their code base has grown
over 20 years and threads are using lots of pointers that are passed along. There are a lot of linked lists aswell,
shared by many threads, which of course contain pointers with physical / absolute addresses.
To move to QNX, the easiest way would be to have 'one big process', so all memory is shared, like in the old OS, between
the threads. But that means not using the main benefit of Neutrino, the memory protection.
So the idea is to use shared memory. However virtual pointers are only valid in one process, not in another, so the
linked lists and other things will not work... UNLESS we could mmap the shared memory object to the same virtual address
in all processes that use it.
The first mmap parameter seems to allow this - however there is no docs I found that would explain which virtual
addresses not to use, e.g. where the OS usually puts code, data, heap, and shared libs. What can we recommend to the
customer? Can he use mmap with a fixed virtual address - and in which range can the address be?
- Malte
|
|
|
|
|
|
Brian Stecher
|
Re: Shared Memory object - map it at a fixed virtual address
|
|
Brian Stecher
04/24/2009 9:27 AM
post27988
|
Re: Shared Memory object - map it at a fixed virtual address
You have one process runs first and maps the storage without MAP_FIXED, then
pass that address to the rest who do the mapping with MAP_FIXED to get the
same address and hope like heck that the secondary processes haven't
already used those virtual addresses for something else.
On Fri, Apr 24, 2009 at 05:08:56AM -0400, Malte Mundt wrote:
> Hi,
>
> a potential customer considers moving to QNX from an 'OS' that doesn't know virtual memory. Their code base has grown
over 20 years and threads are using lots of pointers that are passed along. There are a lot of linked lists aswell,
shared by many threads, which of course contain pointers with physical / absolute addresses.
>
> To move to QNX, the easiest way would be to have 'one big process', so all memory is shared, like in the old OS,
between the threads. But that means not using the main benefit of Neutrino, the memory protection.
>
> So the idea is to use shared memory. However virtual pointers are only valid in one process, not in another, so the
linked lists and other things will not work... UNLESS we could mmap the shared memory object to the same virtual address
in all processes that use it.
>
> The first mmap parameter seems to allow this - however there is no docs I found that would explain which virtual
addresses not to use, e.g. where the OS usually puts code, data, heap, and shared libs. What can we recommend to the
customer? Can he use mmap with a fixed virtual address - and in which range can the address be?
>
>
> - Malte
>
>
> _______________________________________________
> OSTech
> http://community.qnx.com/sf/go/post27959
>
--
Brian Stecher (bstecher@qnx.com) QNX Software Systems
phone: +1 (613) 591-0931 (voice) 175 Terence Matthews Cr.
+1 (613) 591-3579 (fax) Kanata, Ontario, Canada K2M 1W8
|
|
|
|
|
|
Malte Mundt(deleted)
|
Re: Shared Memory object - map it at a fixed virtual address
|
|
Malte Mundt(deleted)
04/27/2009 10:59 AM
post28145
|
Re: Shared Memory object - map it at a fixed virtual address
Hm, sounds good, but the 2nd (etc.) process could have loaded some shared libraries. It's not under the processes
control where the shared libs get put in the virtual address space. Also I couldn't find docs that say where in the
virtual address space the heap usually is put.
On x86, is there a virtual address (range) that usually isn't used by Neutrino?
- Malte
|
|
|
|
|
|
Brian Stecher
|
Re: Shared Memory object - map it at a fixed virtual address
|
|
Brian Stecher
04/28/2009 9:41 AM
post28228
|
Re: Shared Memory object - map it at a fixed virtual address
All virtual addresses are capable of being used, but there are starting
points for where procnto looks for heap/stack/shared objects when it's
not told where the user would like them to be. By choosing a vaddr a distance
from any of those starting points, you increase the probability that
the vaddr will be free in all the processes. However, what those starting
points vary from architecture to architecture and potentially from
release to release (or patch to patch). That is, we reserve the right
to change the algorithm that allocates virtual addresses at any time, for
any reason.
The safest (ha!) bet is to allocate the vaddr range early on in all
the processes and hope for the best (and maybe have a retry algorithm
if one process can't get it)
On Mon, Apr 27, 2009 at 10:59:48AM -0400, Malte Mundt wrote:
> Hm, sounds good, but the 2nd (etc.) process could have loaded some shared libraries. It's not under the processes
control where the shared libs get put in the virtual address space. Also I couldn't find docs that say where in the
virtual address space the heap usually is put.
>
> On x86, is there a virtual address (range) that usually isn't used by Neutrino?
>
>
> - Malte
>
>
> _______________________________________________
> OSTech
> http://community.qnx.com/sf/go/post28145
>
--
Brian Stecher (bstecher@qnx.com) QNX Software Systems
phone: +1 (613) 591-0931 (voice) 175 Terence Matthews Cr.
+1 (613) 591-3579 (fax) Kanata, Ontario, Canada K2M 1W8
|
|
|
|
|
|
Michael Kisel
|
Re: Shared Memory object - map it at a fixed virtual address
|
|
Michael Kisel
04/28/2009 10:03 AM
post28233
|
Re: Shared Memory object - map it at a fixed virtual address
Brian, could typed memory be used to set aside (hide from procnto) the
memory for this purpose?
Brian Stecher wrote:
> All virtual addresses are capable of being used, but there are starting
> points for where procnto looks for heap/stack/shared objects when it's
> not told where the user would like them to be. By choosing a vaddr a distance
> from any of those starting points, you increase the probability that
> the vaddr will be free in all the processes. However, what those starting
> points vary from architecture to architecture and potentially from
> release to release (or patch to patch). That is, we reserve the right
> to change the algorithm that allocates virtual addresses at any time, for
> any reason.
>
> The safest (ha!) bet is to allocate the vaddr range early on in all
> the processes and hope for the best (and maybe have a retry algorithm
> if one process can't get it)
>
> On Mon, Apr 27, 2009 at 10:59:48AM -0400, Malte Mundt wrote:
>
>> Hm, sounds good, but the 2nd (etc.) process could have loaded some shared libraries. It's not under the processes
control where the shared libs get put in the virtual address space. Also I couldn't find docs that say where in the
virtual address space the heap usually is put.
>>
>> On x86, is there a virtual address (range) that usually isn't used by Neutrino?
>>
>>
>> - Malte
>>
>>
>> _______________________________________________
>> OSTech
>> http://community.qnx.com/sf/go/post28145
>>
>>
>
>
|
|
|
|
|
|
Brian Stecher
|
Re: Shared Memory object - map it at a fixed virtual address
|
|
Brian Stecher
04/28/2009 10:11 AM
post28240
|
Re: Shared Memory object - map it at a fixed virtual address
Nope. Typed memory is a physical address thing - it doesn't have any effect
on the allocation of the virtual addresses used to map the memory.
On Tue, Apr 28, 2009 at 10:03:09AM -0400, Michael Kisel wrote:
> Brian, could typed memory be used to set aside (hide from procnto) the
> memory for this purpose?
>
> Brian Stecher wrote:
> > All virtual addresses are capable of being used, but there are starting
> > points for where procnto looks for heap/stack/shared objects when it's
> > not told where the user would like them to be. By choosing a vaddr a distance
> > from any of those starting points, you increase the probability that
> > the vaddr will be free in all the processes. However, what those starting
> > points vary from architecture to architecture and potentially from
> > release to release (or patch to patch). That is, we reserve the right
> > to change the algorithm that allocates virtual addresses at any time, for
> > any reason.
> >
> > The safest (ha!) bet is to allocate the vaddr range early on in all
> > the processes and hope for the best (and maybe have a retry algorithm
> > if one process can't get it)
> >
> > On Mon, Apr 27, 2009 at 10:59:48AM -0400, Malte Mundt wrote:
> >
> >> Hm, sounds good, but the 2nd (etc.) process could have loaded some shared libraries. It's not under the processes
control where the shared libs get put in the virtual address space. Also I couldn't find docs that say where in the
virtual address space the heap usually is put.
> >>
> >> On x86, is there a virtual address (range) that usually isn't used by Neutrino?
> >>
> >>
> >> - Malte
> >>
> >>
> >> _______________________________________________
> >> OSTech
> >> http://community.qnx.com/sf/go/post28145
> >>
> >>
> >
> >
>
>
> _______________________________________________
> OSTech
> http://community.qnx.com/sf/go/post28233
>
--
Brian Stecher (bstecher@qnx.com) QNX Software Systems
phone: +1 (613) 591-0931 (voice) 175 Terence Matthews Cr.
+1 (613) 591-3579 (fax) Kanata, Ontario, Canada K2M 1W8
|
|
|
|
|
|
Michael Kisel
|
Re: Shared Memory object - map it at a fixed virtual address
|
|
Michael Kisel
04/28/2009 10:17 AM
post28242
|
Re: Shared Memory object - map it at a fixed virtual address
yes I know, but I had in my mind that vaddrs were influenced by the
(physical) memory pools and thought that perhaps this would increase the
chance of success ... guess not
Brian Stecher wrote:
> Nope. Typed memory is a physical address thing - it doesn't have any effect
> on the allocation of the virtual addresses used to map the memory.
>
>
> On Tue, Apr 28, 2009 at 10:03:09AM -0400, Michael Kisel wrote:
>
>> Brian, could typed memory be used to set aside (hide from procnto) the
>> memory for this purpose?
>>
>> Brian Stecher wrote:
>>
>>> All virtual addresses are capable of being used, but there are starting
>>> points for where procnto looks for heap/stack/shared objects when it's
>>> not told where the user would like them to be. By choosing a vaddr a distance
>>> from any of those starting points, you increase the probability that
>>> the vaddr will be free in all the processes. However, what those starting
>>> points vary from architecture to architecture and potentially from
>>> release to release (or patch to patch). That is, we reserve the right
>>> to change the algorithm that allocates virtual addresses at any time, for
>>> any reason.
>>>
>>> The safest (ha!) bet is to allocate the vaddr range early on in all
>>> the processes and hope for the best (and maybe have a retry algorithm
>>> if one process can't get it)
>>>
>>> On Mon, Apr 27, 2009 at 10:59:48AM -0400, Malte Mundt wrote:
>>>
>>>
>>>> Hm, sounds good, but the 2nd (etc.) process could have loaded some shared libraries. It's not under the processes
control where the shared libs get put in the virtual address space. Also I couldn't find docs that say where in the
virtual address space the heap usually is put.
>>>>
>>>> On x86, is there a virtual address (range) that usually isn't used by Neutrino?
>>>>
>>>>
>>>> - Malte
>>>>
>>>>
>>>> _______________________________________________
>>>> OSTech
>>>> http://community.qnx.com/sf/go/post28145
>>>>
>>>>
>>>>
>>>
>>>
>> _______________________________________________
>> OSTech
>> http://community.qnx.com/sf/go/post28233
>>
>>
>
>
|
|
|
|
|
|
