Lewis Donzis
|
Can't use more than 3.5GB of RAM
|
Lewis Donzis
02/25/2011 8:56 AM
post83499
|
Can't use more than 3.5GB of RAM
We have systems (x86) with 8GB total RAM, using "startup-apic -x" to enable PAE.
"pidin info" shows 8GB total.
However, as soon as 3.5GB is allocated in total, no more is available. This is not per-process, this is the total
across all processes. For example, if one process allocates 2.5GB, then another process can allocate 1GB, and then no
more is available, even though there should be 4.5GB free.
I've tried allocating with malloc(), and with mmap(), and there's no significant difference.
Does this make any sense?
Thanks,
lew
|
|
|
Jerry Sui
|
RE: Can't use more than 3.5GB of RAM
|
Jerry Sui
02/25/2011 9:24 AM
post83502
|
RE: Can't use more than 3.5GB of RAM
Start procnto with -mP option.
http://www.qnx.com/developers/docs/6.4.1/neutrino/utilities/p/procnto.ht
ml
Jerry
-----Original Message-----
From: Lewis Donzis [mailto:community-noreply@qnx.com]
Sent: February 25, 2011 8:57 AM
To: ostech-core_os
Subject: Can't use more than 3.5GB of RAM
We have systems (x86) with 8GB total RAM, using "startup-apic -x" to
enable PAE.
"pidin info" shows 8GB total.
However, as soon as 3.5GB is allocated in total, no more is available.
This is not per-process, this is the total across all processes. For
example, if one process allocates 2.5GB, then another process can
allocate 1GB, and then no more is available, even though there should be
4.5GB free.
I've tried allocating with malloc(), and with mmap(), and there's no
significant difference.
Does this make any sense?
Thanks,
lew
_______________________________________________
OSTech
http://community.qnx.com/sf/go/post83499
|
|
|
Adam Mallory
|
Re: Can't use more than 3.5GB of RAM
|
Adam Mallory
02/25/2011 9:41 AM
post83504
|
Re: Can't use more than 3.5GB of RAM
You may rather use the spawn flag SPAWN_PADDR64_SAFE or the 'on' utility
with -P to allow a particular process to map from >4g addresses (like the
test apps). Letting all processes do this might break drivers or
applications interacting with hardware as some hw doesn't take well to 64bit
paddrs.
-Adam
On 11-02-25 9:24 AM, "Jerry Sui" <community-noreply@qnx.com> wrote:
> Start procnto with -mP option.
>
> http://www.qnx.com/developers/docs/6.4.1/neutrino/utilities/p/procnto.ht
> ml
>
> Jerry
>
> -----Original Message-----
> From: Lewis Donzis [mailto:community-noreply@qnx.com]
> Sent: February 25, 2011 8:57 AM
> To: ostech-core_os
> Subject: Can't use more than 3.5GB of RAM
>
> We have systems (x86) with 8GB total RAM, using "startup-apic -x" to
> enable PAE.
>
> "pidin info" shows 8GB total.
>
> However, as soon as 3.5GB is allocated in total, no more is available.
> This is not per-process, this is the total across all processes. For
> example, if one process allocates 2.5GB, then another process can
> allocate 1GB, and then no more is available, even though there should be
> 4.5GB free.
>
> I've tried allocating with malloc(), and with mmap(), and there's no
> significant difference.
>
> Does this make any sense?
>
> Thanks,
> lew
>
>
>
> _______________________________________________
>
> OSTech
> http://community.qnx.com/sf/go/post83499
>
>
>
>
> _______________________________________________
>
> OSTech
> http://community.qnx.com/sf/go/post83502
>
--
Adam Mallory (amallory@qnx.com)
Manager, Support Services Phone: +1.613.271.9115
QNX Software Systems Mobile: +1.613.614.6612
|
|
|
Lewis Donzis
|
Re: Can't use more than 3.5GB of RAM
|
Lewis Donzis
02/25/2011 9:51 AM
post83505
|
Re: Can't use more than 3.5GB of RAM
Thank you both for the suggestions. I guess I was slightly taken taken aback with -mP being documented as being useful
only for testing.
So now I'm wondering what might not work with procnto -mP? The only systems we have with more than 4GB are "modern", i.
e., with PCI Express and hardware that understands large addresses.
I was under the impression that beginning with 6.4, all of the QNX-supplied drivers were supposed to be 64-bit physaddr
compliant?
We're pretty familiar with the io-pkt drivers, and they appear to be correctly setting 64-bit physical addresses. I
haven't looked into the devb-eide or devb-ahci drivers, but if they're compliant, I can't think of much else that would
be doing DMA and care about high addresses.
Also, as a practical matter, devb-xxx and io-pkt get loaded pretty early at boot time, so wouldn't they tend to get
physical memory below 4GB anyway?
Thanks,
lew
|
|
|
Jerry Sui
|
RE: Can't use more than 3.5GB of RAM
|
Jerry Sui
02/25/2011 10:03 AM
post83507
|
RE: Can't use more than 3.5GB of RAM
You can give it a try. Do some thorough tests on your system. It might
be ok.
>> I was under the impression that beginning with 6.4, all of the
QNX-supplied drivers were supposed to be 64-bit physaddr compliant?
Not necessary. The driver just use mmap with default parameters, it is
the kernel who decides which physical memory to map with the default
flags. However, you can control which part of physical memory you like
to get with posix_typed_mem_open().
>>Also, as a practical matter, devb-xxx and io-pkt get loaded pretty
early at boot time, so wouldn't they tend to get physical memory below
4GB anyway?
No, when -mP is given, the system tries memory beyond 4G first, then
below 4G, then the first 256M.
Jerry Sui | Software Developer | jsui@qnx.com
O:613-591-0836 x 2904 C:613-240-9239 | www.qnx.com | QNX Software
Systems
Cisco Support Team email: cisco_support@qnx.com
-----Original Message-----
From: Lewis Donzis [mailto:community-noreply@qnx.com]
Sent: February 25, 2011 9:51 AM
To: ostech-core_os
Subject: Re: Can't use more than 3.5GB of RAM
Thank you both for the suggestions. I guess I was slightly taken taken
aback with -mP being documented as being useful only for testing.
So now I'm wondering what might not work with procnto -mP? The only
systems we have with more than 4GB are "modern", i.e., with PCI Express
and hardware that understands large addresses.
I was under the impression that beginning with 6.4, all of the
QNX-supplied drivers were supposed to be 64-bit physaddr compliant?
We're pretty familiar with the io-pkt drivers, and they appear to be
correctly setting 64-bit physical addresses. I haven't looked into the
devb-eide or devb-ahci drivers, but if they're compliant, I can't think
of much else that would be doing DMA and care about high addresses.
Also, as a practical matter, devb-xxx and io-pkt get loaded pretty early
at boot time, so wouldn't they tend to get physical memory below 4GB
anyway?
Thanks,
lew
_______________________________________________
OSTech
http://community.qnx.com/sf/go/post83505
|
|
|
Lewis Donzis
|
Re: RE: Can't use more than 3.5GB of RAM
|
Lewis Donzis
02/25/2011 10:14 AM
post83509
|
Re: RE: Can't use more than 3.5GB of RAM
> >> I was under the impression that beginning with 6.4, all of the
> QNX-supplied drivers were supposed to be 64-bit physaddr compliant?
>
> Not necessary. The driver just use mmap with default parameters, it is
> the kernel who decides which physical memory to map with the default
> flags. However, you can control which part of physical memory you like
> to get with posix_typed_mem_open().
I guess what I meant was that all of the QNX-supplied drivers are designed to be aware of 64-bit physical addresses,
wherever possible.
For example, in io-net, the way it was compiled in 6.3.2, an npkt could only handle a 32-bit physaddr, no matter whether
the driver could do better or not. In io-pkt, everything is capable of 64-bit physaddrs, and the drivers, at least the
ones we use, are properly setting the entire 64-bit address in the hardware for DMA.
So what I was trying to say was that I was under the impression that in 6.4/6.5, all of the QNX-supplied drivers,
wherever the hardware supports it, properly handle memory above 4GB.
Do you know of any instance where that is not true?
Thanks,
lew
|
|
|
Jerry Sui
|
RE: RE: Can't use more than 3.5GB of RAM
|
Jerry Sui
02/25/2011 10:21 AM
post83511
|
RE: RE: Can't use more than 3.5GB of RAM
>>Do you know of any instance where that is not true?
I am not sure of that. I did not search all the drivers to find out if
they are all capable of 64bit physaddrs.
Jerry Sui | Software Developer | jsui@qnx.com
O:613-591-0836 x 2904 C:613-240-9239 | www.qnx.com | QNX Software
Systems
Cisco Support Team email: cisco_support@qnx.com
-----Original Message-----
From: Lewis Donzis [mailto:community-noreply@qnx.com]
Sent: February 25, 2011 10:14 AM
To: ostech-core_os
Subject: Re: RE: Can't use more than 3.5GB of RAM
> >> I was under the impression that beginning with 6.4, all of the
> QNX-supplied drivers were supposed to be 64-bit physaddr compliant?
>
> Not necessary. The driver just use mmap with default parameters, it is
> the kernel who decides which physical memory to map with the default
> flags. However, you can control which part of physical memory you like
> to get with posix_typed_mem_open().
I guess what I meant was that all of the QNX-supplied drivers are
designed to be aware of 64-bit physical addresses, wherever possible.
For example, in io-net, the way it was compiled in 6.3.2, an npkt could
only handle a 32-bit physaddr, no matter whether the driver could do
better or not. In io-pkt, everything is capable of 64-bit physaddrs,
and the drivers, at least the ones we use, are properly setting the
entire 64-bit address in the hardware for DMA.
So what I was trying to say was that I was under the impression that in
6.4/6.5, all of the QNX-supplied drivers, wherever the hardware supports
it, properly handle memory above 4GB.
Do you know of any instance where that is not true?
Thanks,
lew
_______________________________________________
OSTech
http://community.qnx.com/sf/go/post83509
|
|
|
Colin Burgess(deleted)
|
Re: Can't use more than 3.5GB of RAM
|
Colin Burgess(deleted)
02/25/2011 10:23 AM
post83512
|
Re: Can't use more than 3.5GB of RAM
I think the usb stack can't handle 64-bit physical addresses
On 11-02-25 10:21 AM, Jerry Sui wrote:
>
> >>Do you know of any instance where that is not true?
>
> I am not sure of that. I did not search all the drivers to find out if
> they are all capable of 64bit physaddrs.
>
> Jerry Sui | Software Developer | jsui@qnx.com
> O:613-591-0836 x 2904 C:613-240-9239 | www.qnx.com | QNX Software
> Systems
> Cisco Support Team email: cisco_support@qnx.com
>
>
> -----Original Message-----
> From: Lewis Donzis [mailto:community-noreply@qnx.com]
> Sent: February 25, 2011 10:14 AM
> To: ostech-core_os
> Subject: Re: RE: Can't use more than 3.5GB of RAM
>
> > >> I was under the impression that beginning with 6.4, all of the
> > QNX-supplied drivers were supposed to be 64-bit physaddr compliant?
> >
> > Not necessary. The driver just use mmap with default parameters, it is
> > the kernel who decides which physical memory to map with the default
> > flags. However, you can control which part of physical memory you like
> > to get with posix_typed_mem_open().
>
> I guess what I meant was that all of the QNX-supplied drivers are
> designed to be aware of 64-bit physical addresses, wherever possible.
>
> For example, in io-net, the way it was compiled in 6.3.2, an npkt could
> only handle a 32-bit physaddr, no matter whether the driver could do
> better or not. In io-pkt, everything is capable of 64-bit physaddrs,
> and the drivers, at least the ones we use, are properly setting the
> entire 64-bit address in the hardware for DMA.
>
> So what I was trying to say was that I was under the impression that in
> 6.4/6.5, all of the QNX-supplied drivers, wherever the hardware supports
> it, properly handle memory above 4GB.
>
> Do you know of any instance where that is not true?
>
> Thanks,
> lew
>
>
>
> _______________________________________________
>
> OSTech
> http://community.qnx.com/sf/go/post83509
>
>
>
>
> _______________________________________________
>
> OSTech
> http://community.qnx.com/sf/go/post83511
>
--
cburgess@qnx.com
|
|
|
Lewis Donzis
|
Re: Can't use more than 3.5GB of RAM
|
Lewis Donzis
02/25/2011 10:45 AM
post83513
|
Re: Can't use more than 3.5GB of RAM
> I think the usb stack can't handle 64-bit physical addresses
That's good to know. I presume this is a limitation of the stack architecture, not the hardware, per se.
Sorry if it sounds like a silly question, but it would be nice, if the system is going to provide support for >4GB RAM,
that it's not so "experimental" and that the limitations are known.
If there are drivers that can't handle high physical memory, perhaps there could be a flag to mmap() or something like
that (we already have that for the rather antique 16M problem).
lew
|
|
|
Jerry Sui
|
RE: Can't use more than 3.5GB of RAM
|
Jerry Sui
02/25/2011 10:53 AM
post83514
|
RE: Can't use more than 3.5GB of RAM
Please refer to the following thread about allocating physical memory
below 4G.
http://community.qnx.com/sf/go/post80250
Jerry Sui | Software Developer | jsui@qnx.com
O:613-591-0836 x 2904 C:613-240-9239 | www.qnx.com | QNX Software
Systems
Cisco Support Team email: cisco_support@qnx.com
-----Original Message-----
From: Lewis Donzis [mailto:community-noreply@qnx.com]
Sent: February 25, 2011 10:46 AM
To: ostech-core_os
Subject: Re: Can't use more than 3.5GB of RAM
> I think the usb stack can't handle 64-bit physical addresses
That's good to know. I presume this is a limitation of the stack
architecture, not the hardware, per se.
Sorry if it sounds like a silly question, but it would be nice, if the
system is going to provide support for >4GB RAM, that it's not so
"experimental" and that the limitations are known.
If there are drivers that can't handle high physical memory, perhaps
there could be a flag to mmap() or something like that (we already have
that for the rather antique 16M problem).
lew
_______________________________________________
OSTech
http://community.qnx.com/sf/go/post83513
|
|
|
Lewis Donzis
|
Re: RE: Can't use more than 3.5GB of RAM
|
Lewis Donzis
02/25/2011 12:14 PM
post83522
|
Re: RE: Can't use more than 3.5GB of RAM
Thanks, Jerry.
Is it possible to reverse the option, in other words, to start the kernel with -mP and then launch programs that we know
are not 64-bit paddr capable with "on"?
The reason I ask is because 99% of programs don't care anything about physical memory and due to message passing
architecture, there is no direct DMA access to their memory anyway.
It's only drivers that use DMA that have to worry about this problem, so it would be nice if we could specify particular
programs that need physical memory below 4G and let everything else use whatever is available.
lew
|
|
|
Jerry Sui
|
RE: RE: Can't use more than 3.5GB of RAM
|
Jerry Sui
02/25/2011 12:24 PM
post83523
|
RE: RE: Can't use more than 3.5GB of RAM
To my knowledge, it is not possible. The kernel checks flag
SPAWN_PADDR64_SAFE to decide where to allocate physical memory by
default when -mP is not enabled. There is no SPAWN_PADDR64_UNSAFE flag
when -mP is enabled.
Jerry Sui | Software Developer | jsui@qnx.com
O:613-591-0836 x 2904 C:613-240-9239 | www.qnx.com | QNX Software
Systems
Cisco Support Team email: cisco_support@qnx.com
-----Original Message-----
From: Lewis Donzis [mailto:community-noreply@qnx.com]
Sent: February 25, 2011 12:14 PM
To: ostech-core_os
Subject: Re: RE: Can't use more than 3.5GB of RAM
Thanks, Jerry.
Is it possible to reverse the option, in other words, to start the
kernel with -mP and then launch programs that we know are not 64-bit
paddr capable with "on"?
The reason I ask is because 99% of programs don't care anything about
physical memory and due to message passing architecture, there is no
direct DMA access to their memory anyway.
It's only drivers that use DMA that have to worry about this problem, so
it would be nice if we could specify particular programs that need
physical memory below 4G and let everything else use whatever is
available.
lew
_______________________________________________
OSTech
http://community.qnx.com/sf/go/post83522
|
|
|
Lewis Donzis
|
Re: RE: RE: Can't use more than 3.5GB of RAM
|
Lewis Donzis
02/26/2011 8:26 AM
post83550
|
Re: RE: RE: Can't use more than 3.5GB of RAM
> To my knowledge, it is not possible. The kernel checks flag
> SPAWN_PADDR64_SAFE to decide where to allocate physical memory by
> default when -mP is not enabled. There is no SPAWN_PADDR64_UNSAFE flag
> when -mP is enabled.
I've been doing some more testing with the idea of using SPAWN_PADDR64_SAFE, and wanted to point out that this is highly
problematic.
The trouble is that from a system perspective, many programs spawn other programs, and it appears that
SPAWN_PADDR64_SAFE does not inherit from the parent. Even fixing system() to include the flag on spawn() doesn't help
because it ends up spawning a shell which when spawns the program, and of course the shell spawns without
SPAWN_PADDR64_SAFE.
Because of the nature of the QNX architecture, it seems that every program is guaranteed to be physaddr-agnostic except
for the few drivers that do DMA, e.g., the disk, network, USB drivers, etc., basically devx-xxx. From examining the
network drivers, most of the devnp-xxx ones are already 64-bit physaddr capable, but it appears that devb-eide may not
be.
In retrospect, I think it would have been cleaner to make a SPAWN_PADDR64_UNSAFE flag (or mark the ELF in some way that
the loader can tell) and load drivers that are not known to be 64-bit physaddr safe with that flag. "Everything else,"
i.e., normal programs, obviously cannot care about their physaddrs and due to the message passing architecture, there is
never any direct access to their physical memory, even from drivers.
lew
|
|
|
Jerry Sui
|
RE: RE: RE: Can't use more than 3.5GB of RAM
|
Jerry Sui
02/28/2011 10:18 AM
post83573
|
RE: RE: RE: Can't use more than 3.5GB of RAM
The correct fix would be fixing those drivers to allocate memory for DMA
from the "below4G" pool. It can be done like this:
Dma_fd = posix_typed_mem_open("sysram&below4G", oflag, tflag);
dma_mem = mmap(addr, len, prot, flags, dma_fd, off);
Doing it this way results only the DMA buffer is allocated from the
below4G pool, the rest allocation in the driver can still allocate
memory without restrictions.
Jerry Sui | Software Developer | jsui@qnx.com
O:613-591-0836 x 2904 C:613-240-9239 | www.qnx.com | QNX Software
Systems
Cisco Support Team email: cisco_support@qnx.com
-----Original Message-----
From: Lewis Donzis [mailto:community-noreply@qnx.com]
Sent: February 26, 2011 8:27 AM
To: ostech-core_os
Subject: Re: RE: RE: Can't use more than 3.5GB of RAM
> To my knowledge, it is not possible. The kernel checks flag
> SPAWN_PADDR64_SAFE to decide where to allocate physical memory by
> default when -mP is not enabled. There is no SPAWN_PADDR64_UNSAFE flag
> when -mP is enabled.
I've been doing some more testing with the idea of using
SPAWN_PADDR64_SAFE, and wanted to point out that this is highly
problematic.
The trouble is that from a system perspective, many programs spawn other
programs, and it appears that SPAWN_PADDR64_SAFE does not inherit from
the parent. Even fixing system() to include the flag on spawn() doesn't
help because it ends up spawning a shell which when spawns the program,
and of course the shell spawns without SPAWN_PADDR64_SAFE.
Because of the nature of the QNX architecture, it seems that every
program is guaranteed to be physaddr-agnostic except for the few drivers
that do DMA, e.g., the disk, network, USB drivers, etc., basically
devx-xxx. From examining the network drivers, most of the devnp-xxx
ones are already 64-bit physaddr capable, but it appears that devb-eide
may not be.
In retrospect, I think it would have been cleaner to make a
SPAWN_PADDR64_UNSAFE flag (or mark the ELF in some way that the loader
can tell) and load drivers that are not known to be 64-bit physaddr safe
with that flag. "Everything else," i.e., normal programs, obviously
cannot care about their physaddrs and due to the message passing
architecture, there is never any direct access to their physical memory,
even from drivers.
lew
_______________________________________________
OSTech
http://community.qnx.com/sf/go/post83550
|
|
|
Lewis Donzis
|
Re: RE: RE: RE: Can't use more than 3.5GB of RAM
|
Lewis Donzis
03/03/2011 1:01 AM
post83706
|
Re: RE: RE: RE: Can't use more than 3.5GB of RAM
> The correct fix would be fixing those drivers to allocate memory for DMA
> from the "below4G" pool. It can be done like this:
>
> Dma_fd = posix_typed_mem_open("sysram&below4G", oflag, tflag);
> dma_mem = mmap(addr, len, prot, flags, dma_fd, off);
>
> Doing it this way results only the DMA buffer is allocated from the
> below4G pool, the rest allocation in the driver can still allocate
> memory without restrictions.
>
> Jerry Sui | Software Developer | jsui@qnx.com
> O:613-591-0836 x 2904 C:613-240-9239 | www.qnx.com | QNX Software
> Systems
> Cisco Support Team email: cisco_support@qnx.com
Thanks, Jerry.
I agree, it would be great to fix the appropriate drivers such as devb-eide and possibly devb-ahci, either to use 64-bit
physical addresses if possible, or to use your example above and allocate their DMA buffers below 4G. Perhaps you can
put in a good word with the devb-xxx driver developers...
It appears that most of the devnp-xxx drivers already handle 64-bit physical addresses properly, so there may not be
much problem with io-pkt.
If everything worked properly, then specifying -mP to the kernel would be much easier, and if it was known to work,
maybe that could even become the default in the future.
Meanwhile, we've changed to spawning everything except the devb drivers with SPAWN_PADDR64_SAFE and modified all of our
programs that spawn other programs to do the same, and we're mostly able to take advantage of the large memory. So this
is a reasonable interim solution.
Thanks for your help,
lew
|
|
|
Lewis Donzis
|
Re: RE: RE: RE: Can't use more than 3.5GB of RAM
|
Lewis Donzis
03/08/2011 11:03 PM
post83842
|
Re: RE: RE: RE: Can't use more than 3.5GB of RAM
> > Dma_fd = posix_typed_mem_open("sysram&below4G", oflag, tflag);
> > dma_mem = mmap(addr, len, prot, flags, dma_fd, off);
Is there a typed memory name for "all memory, including that above 4G" when the process was spawned without PADDR64_SAFE
?
For example, let's say I want to load io-pkt without PADDR64_SAFE because I'm not sure that all drivers are 64-bit safe.
But then I want to make a driver that is going to allocate some large structures that will not be used for DMA, so
it's ok for them to be in memory above 4G. Is there a way to allocate it?
Thanks,
lew
|
|
|
Jerry Sui
|
RE: RE: RE: RE: Can't use more than 3.5GB of RAM
|
Jerry Sui
03/09/2011 9:40 AM
post83865
|
RE: RE: RE: RE: Can't use more than 3.5GB of RAM
You can create a typed memory named "above4G" in your startup code, then
doing the same thing as allocating typed memory below 4G. We don't
create "above4G" by default in our startup library. Type "pidin
syspage=asinfo", you can find out the memory layout and where below4G is
created. You can create an entry named "above4G" or whatever name you
like.
Jerry Sui | Software Developer | jsui@qnx.com
O:613-591-0836 x 2904 C:613-240-9239 | www.qnx.com | QNX Software
Systems
Cisco Support Team email: cisco_support@qnx.com
-----Original Message-----
From: Lewis Donzis [mailto:community-noreply@qnx.com]
Sent: March 8, 2011 11:04 PM
To: ostech-core_os
Subject: Re: RE: RE: RE: Can't use more than 3.5GB of RAM
> > Dma_fd = posix_typed_mem_open("sysram&below4G", oflag, tflag);
> > dma_mem = mmap(addr, len, prot, flags, dma_fd, off);
Is there a typed memory name for "all memory, including that above 4G"
when the process was spawned without PADDR64_SAFE?
For example, let's say I want to load io-pkt without PADDR64_SAFE
because I'm not sure that all drivers are 64-bit safe. But then I want
to make a driver that is going to allocate some large structures that
will not be used for DMA, so it's ok for them to be in memory above 4G.
Is there a way to allocate it?
Thanks,
lew
_______________________________________________
OSTech
http://community.qnx.com/sf/go/post83842
|
|
|
Jerry Sui
|
RE: RE: RE: RE: Can't use more than 3.5GB of RAM
|
Jerry Sui
03/09/2011 9:44 AM
post83867
|
RE: RE: RE: RE: Can't use more than 3.5GB of RAM
But you must be careful to restrict the allocation above 4G, your
software will fail if it is running on a system has less than 4G
physical memory. Of course you can fallback to below 4G allocation, but
cares must be taken.
Jerry Sui | Software Developer | jsui@qnx.com
O:613-591-0836 x 2904 C:613-240-9239 | www.qnx.com | QNX Software
Systems
Cisco Support Team email: cisco_support@qnx.com
-----Original Message-----
From: Jerry Sui
Sent: March 9, 2011 9:40 AM
To: ostech-core_os
Subject: RE: RE: RE: RE: Can't use more than 3.5GB of RAM
You can create a typed memory named "above4G" in your startup code, then
doing the same thing as allocating typed memory below 4G. We don't
create "above4G" by default in our startup library. Type "pidin
syspage=asinfo", you can find out the memory layout and where below4G is
created. You can create an entry named "above4G" or whatever name you
like.
Jerry Sui | Software Developer | jsui@qnx.com
O:613-591-0836 x 2904 C:613-240-9239 | www.qnx.com | QNX Software
Systems
Cisco Support Team email: cisco_support@qnx.com
-----Original Message-----
From: Lewis Donzis [mailto:community-noreply@qnx.com]
Sent: March 8, 2011 11:04 PM
To: ostech-core_os
Subject: Re: RE: RE: RE: Can't use more than 3.5GB of RAM
> > Dma_fd = posix_typed_mem_open("sysram&below4G", oflag, tflag);
> > dma_mem = mmap(addr, len, prot, flags, dma_fd, off);
Is there a typed memory name for "all memory, including that above 4G"
when the process was spawned without PADDR64_SAFE?
For example, let's say I want to load io-pkt without PADDR64_SAFE
because I'm not sure that all drivers are 64-bit safe. But then I want
to make a driver that is going to allocate some large structures that
will not be used for DMA, so it's ok for them to be in memory above 4G.
Is there a way to allocate it?
Thanks,
lew
_______________________________________________
OSTech
http://community.qnx.com/sf/go/post83842
|
|
|
Mario Charest
|
RE: RE: Can't use more than 3.5GB of RAM
|
Mario Charest
02/25/2011 2:37 PM
post83528
|
RE: RE: Can't use more than 3.5GB of RAM
> -----Message d'origine-----
> De : Lewis Donzis [mailto:community-noreply@qnx.com]
> Envoyé : 25 février 2011 12:14
> À : ostech-core_os
> Objet : Re: RE: Can't use more than 3.5GB of RAM
>
> Thanks, Jerry.
>
> Is it possible to reverse the option, in other words, to start the kernel with -
> mP and then launch programs that we know are not 64-bit paddr capable
> with "on"?
>
> The reason I ask is because 99% of programs don't care anything about
> physical memory and due to message passing architecture, there is no direct
> DMA access to their memory anyway.
>
> It's only drivers that use DMA that have to worry about this problem, so it
> would be nice if we could specify particular programs that need physical
> memory below 4G and let everything else use whatever is available.^
A x64 os would make thing (from a user`s perspective) so much nicer ;-) But I know you, the QNX staffer, know ;!
If only one car make would be bold enough to label their car 64Bit inside out ;-)
>
> lew
>
>
>
> _______________________________________________
>
> OSTech
> http://community.qnx.com/sf/go/post83522
>
|
|
|
Lewis Donzis
|
Re: RE: RE: Can't use more than 3.5GB of RAM
|
Lewis Donzis
02/25/2011 2:55 PM
post83530
|
Re: RE: RE: Can't use more than 3.5GB of RAM
> A x64 os would make thing (from a user`s perspective) so much nicer ;-) But I
> know you, the QNX staffer, know ;!
>
> If only one car make would be bold enough to label their car 64Bit inside out
> ;-)
Yeah, add our vote to having full 64-bit address capability! That would make things so much easier for what we're doing
.
lew
|
|
|
Mario Charest
|
RE: RE: RE: Can't use more than 3.5GB of RAM
|
Mario Charest
02/25/2011 4:25 PM
post83537
|
RE: RE: RE: Can't use more than 3.5GB of RAM
> -----Message d'origine-----
> De : Lewis Donzis [mailto:community-noreply@qnx.com]
> Envoyé : 25 février 2011 14:56
> À : ostech-core_os
> Objet : Re: RE: RE: Can't use more than 3.5GB of RAM
>
> > A x64 os would make thing (from a user`s perspective) so much nicer
> > ;-) But I know you, the QNX staffer, know ;!
> >
> > If only one car make would be bold enough to label their car 64Bit
> > inside out
> > ;-)
>
> Yeah, add our vote to having full 64-bit address capability! That would make
> things so much easier for what we're doing.
Don`t know if that is still true but last I heard, x86 was, for QSSL, at the bottom of the food chain, I`m not expecting
x64 anytime soon, but would *LOVE* to be showed wrong. I keep a copy of the old QUICS system, just for fun. In it
there are strong words coming for senior staff saying that it would be extremely unlikely QNX would ever be available
on anything but x86. Going through the QUICS stuff is quite entertaining !
I know without the -x option of startup-bios we would have probably switch OS by know (some of our system use 24Gig of
ram...). Multi-core is nice but it increases the need for memory. We can still deal with the 3.5G per process limit,
for a while, but virtual adresses is becoming an issue.
>
> lew
>
>
>
> _______________________________________________
>
> OSTech
> http://community.qnx.com/sf/go/post83530
>
|
|
|
Lewis Donzis
|
Re: RE: RE: Can't use more than 3.5GB of RAM
|
Lewis Donzis
02/25/2011 3:51 PM
post83534
|
Re: RE: RE: Can't use more than 3.5GB of RAM
> I am not sure of that. I did not search all the drivers to find out if
> they are all capable of 64bit physaddrs.
Jerry, one last question...
It looks like /dev/shmem is also limited to low memory, is that correct? In other words, even if I have 5GB free, I
can't create a 5GB file on /dev/shmem, it says it's full as soon as the 3.5G memory is allocated?
lew
|
|
|
Jerry Sui
|
RE: RE: RE: Can't use more than 3.5GB of RAM
|
Jerry Sui
02/25/2011 4:14 PM
post83536
|
RE: RE: RE: Can't use more than 3.5GB of RAM
Not sure about /dev/shmem. It might be limited by virtual memory space.
Remember the virtual space is limited to 32 bits, because the kernel
takes 1G, the user can only have about 3G. You will exhaust your virtual
space to have a shared memory larger than 3G.
Jerry Sui | Software Developer | jsui@qnx.com
O:613-591-0836 x 2904 C:613-240-9239 | www.qnx.com | QNX Software
Systems
Cisco Support Team email: cisco_support@qnx.com
-----Original Message-----
From: Lewis Donzis [mailto:community-noreply@qnx.com]
Sent: February 25, 2011 3:52 PM
To: ostech-core_os
Subject: Re: RE: RE: Can't use more than 3.5GB of RAM
> I am not sure of that. I did not search all the drivers to find out if
> they are all capable of 64bit physaddrs.
Jerry, one last question...
It looks like /dev/shmem is also limited to low memory, is that correct?
In other words, even if I have 5GB free, I can't create a 5GB file on
/dev/shmem, it says it's full as soon as the 3.5G memory is allocated?
lew
_______________________________________________
OSTech
http://community.qnx.com/sf/go/post83534
|
|
|
Jerry Sui
|
RE: Can't use more than 3.5GB of RAM
|
Jerry Sui
02/25/2011 9:40 AM
post83503
|
RE: Can't use more than 3.5GB of RAM
You must be careful to use procnto -mP option, when this option is
given, all anonymous allocation will default to physical memory beyond
4G, some of your hardware might not be able to handle 64 bit DMA
physical address, cares must be given to those hardware. One alternate
solution is to start procnto without -mP option, but start your process
that can handle 64bit physical address using on -P option.
http://www.qnx.com/developers/docs/6.4.1/neutrino/utilities/o/on.html
Jerry Sui | Software Developer | jsui@qnx.com
O:613-591-0836 x 2904 C:613-240-9239 | www.qnx.com | QNX Software
Systems
Cisco Support Team email: cisco_support@qnx.com
-----Original Message-----
From: Jerry Sui
Sent: February 25, 2011 9:25 AM
To: ostech-core_os
Subject: RE: Can't use more than 3.5GB of RAM
Start procnto with -mP option.
http://www.qnx.com/developers/docs/6.4.1/neutrino/utilities/p/procnto.ht
ml
Jerry
-----Original Message-----
From: Lewis Donzis [mailto:community-noreply@qnx.com]
Sent: February 25, 2011 8:57 AM
To: ostech-core_os
Subject: Can't use more than 3.5GB of RAM
We have systems (x86) with 8GB total RAM, using "startup-apic -x" to
enable PAE.
"pidin info" shows 8GB total.
However, as soon as 3.5GB is allocated in total, no more is available.
This is not per-process, this is the total across all processes. For
example, if one process allocates 2.5GB, then another process can
allocate 1GB, and then no more is available, even though there should be
4.5GB free.
I've tried allocating with malloc(), and with mmap(), and there's no
significant difference.
Does this make any sense?
Thanks,
lew
_______________________________________________
OSTech
http://community.qnx.com/sf/go/post83499
|
|
|
Vitaly
|
Re: Can't use more than 3.5GB of RAM
|
Vitaly
06/22/2012 4:57 AM
post93824
|
Re: Can't use more than 3.5GB of RAM
Maybe QSS can build special version of QNX SDK Platform (self-hosted) for x86 with out-of-box support more 4GB of RAM.
I'm think this will be usable feature.
Thanks.
|
|
|
|