|
Mario Charest
01/30/2009 2:54 PM
post21131
|
The doc on _amblksiz says memory request larger then 32KB are services by mmap.
Does that mean if the heap has 1Meg free and the code does a malloc of 64K will not get the memory from the heap?
I assume if multiple processes running on multiple cores do mmap() simultaneously to obtain memory that there is some
sort of locking happening inside proc.
|
|
|
|
|
|
Oleh Derevenko(deleted)
01/30/2009 3:15 PM
post21136
|
Why do you think there is no locking in heap allocation routine? ;-)
|
|
|
|
|
|
Shiv Nagarajan(deleted)
01/30/2009 3:18 PM
post21137
|
yes, proc has the necessary locking gear to permit multiple mmap
requests at the same time.
re: the note in amblksz requests > 32k are sent directly to mmap without
consulting the free list, because pieces larger than 32k are not put on
the free list. We cache some of the arena blocks in the library, so if
you look at the "heapsize" it includes cached blocks, so some of these
cache blocks could be used by a malloc for a block > 32k
malloc itself has a large lock, so multiple threads can call malloc, and
the underlying mmap call, is made while that malloc lock is held
shiv
Fri Jan 30 15:17:51 EST 2009
--> According to Mario Charest <--
The doc on _amblksiz says memory request larger then 32KB are services by mmap.
Does that mean if the heap has 1Meg free and the code does a malloc of 64K will not get the memory from the heap?
I assume if multiple processes running on multiple cores do mmap() simultaneously to obtain memory that there is some
sort of locking happening inside proc.
_______________________________________________
OSTech
http://community.qnx.com/sf/go/post21131
--
****
Shiv Nagarajan,
Kernel Developer, QNX Software Systems,
Ottawa, Canada
****
|
|
|
|
|
|
Mario Charest
01/30/2009 3:35 PM
post21142
|
Thanks, that confirms what I'm seeing. We have a pool of processes each getting some work do. They all start the work
pretty much at the same time, and that means doing a bunch of allocation simultaneously. But because the memory comes
from the OS instead of the heap it causes a bottleneck and for a short while we can't get all cores to get busy!
I'm sure there must have been hours of meetings about the issue ;) I would have prefer a QNX4 like behaviour, where
everything comes from the heap and the heap only grows and never shrinks. That's more real-time friendly in my opinion.
> -----Original Message-----
> From: Shiv Nagarajan [mailto:community-noreply@qnx.com]
> Sent: January-30-09 3:18 PM
> To: ostech-core_os
> Subject: Re: malloc and mmap
>
> yes, proc has the necessary locking gear to permit multiple mmap
> requests at the same time.
>
> re: the note in amblksz requests > 32k are sent directly to mmap
> without
> consulting the free list, because pieces larger than 32k are not put on
> the free list. We cache some of the arena blocks in the library, so if
> you look at the "heapsize" it includes cached blocks, so some of these
> cache blocks could be used by a malloc for a block > 32k
>
> malloc itself has a large lock, so multiple threads can call malloc,
> and
> the underlying mmap call, is made while that malloc lock is held
>
> shiv
> Fri Jan 30 15:17:51 EST 2009
> --> According to Mario Charest <--
> The doc on _amblksiz says memory request larger then 32KB are
> services by mmap.
>
> Does that mean if the heap has 1Meg free and the code does a
> malloc of 64K will not get the memory from the heap?
>
> I assume if multiple processes running on multiple cores do
> mmap() simultaneously to obtain memory that there is some sort of
> locking happening inside proc.
>
>
>
> _______________________________________________
> OSTech
> http://community.qnx.com/sf/go/post21131
>
> --
> ****
> Shiv Nagarajan,
> Kernel Developer, QNX Software Systems,
> Ottawa, Canada
> ****
>
> _______________________________________________
> OSTech
> http://community.qnx.com/sf/go/post21137
>
|
|
|
|
|
|
Colin Burgess(deleted)
01/30/2009 3:40 PM
post21145
|
Mario you can
a) request a large preallocation upfront
b) turn off release back to the OS
via environment varilables/mallopt options. :-)
Colin
Mario Charest wrote:
> Thanks, that confirms what I'm seeing. We have a pool of processes each getting some work do. They all start the work
pretty much at the same time, and that means doing a bunch of allocation simultaneously. But because the memory comes
from the OS instead of the heap it causes a bottleneck and for a short while we can't get all cores to get busy!
>
> I'm sure there must have been hours of meetings about the issue ;) I would have prefer a QNX4 like behaviour, where
everything comes from the heap and the heap only grows and never shrinks. That's more real-time friendly in my opinion.
>
>
>> -----Original Message-----
>> From: Shiv Nagarajan [mailto:community-noreply@qnx.com]
>> Sent: January-30-09 3:18 PM
>> To: ostech-core_os
>> Subject: Re: malloc and mmap
>>
>> yes, proc has the necessary locking gear to permit multiple mmap
>> requests at the same time.
>>
>> re: the note in amblksz requests > 32k are sent directly to mmap
>> without
>> consulting the free list, because pieces larger than 32k are not put on
>> the free list. We cache some of the arena blocks in the library, so if
>> you look at the "heapsize" it includes cached blocks, so some of these
>> cache blocks could be used by a malloc for a block > 32k
>>
>> malloc itself has a large lock, so multiple threads can call malloc,
>> and
>> the underlying mmap call, is made while that malloc lock is held
>>
>> shiv
>> Fri Jan 30 15:17:51 EST 2009
>> --> According to Mario Charest <--
>> The doc on _amblksiz says memory request larger then 32KB are
>> services by mmap.
>>
>> Does that mean if the heap has 1Meg free and the code does a
>> malloc of 64K will not get the memory from the heap?
>>
>> I assume if multiple processes running on multiple cores do
>> mmap() simultaneously to obtain memory that there is some sort of
>> locking happening inside proc.
>>
>>
>>
>> _______________________________________________
>> OSTech
>> http://community.qnx.com/sf/go/post21131
>>
>> --
>> ****
>> Shiv Nagarajan,
>> Kernel Developer, QNX Software Systems,
>> Ottawa, Canada
>> ****
>>
>> _______________________________________________
>> OSTech
>> http://community.qnx.com/sf/go/post21137
>>
>
>
> _______________________________________________
> OSTech
> http://community.qnx.com/sf/go/post21142
>
--
cburgess@qnx.com
|
|
|
|
|
|
Shiv Nagarajan(deleted)
01/30/2009 3:48 PM
post21148
|
thanks colin.
Thats right.
MALLOC_MEMORY_PREALLOCATE=size
would pre-allocate "size" core memory up front, on the first entry into the
allocator.
and MALLOC_MEMORY_HOLD=1
will never release memory back to the system from the heap.
shiv
Fri Jan 30 15:47:53 EST 2009
--> According to Colin Burgess <--
Mario you can
a) request a large preallocation upfront
b) turn off release back to the OS
via environment varilables/mallopt options. :-)
Colin
Mario Charest wrote:
> Thanks, that confirms what I'm seeing. We have a pool of processes
each getting some work do. They all start the work pretty much at the
same time, and that means doing a bunch of allocation simultaneously.
But because the memory comes from the OS instead of the heap it causes a
bottleneck and for a short while we can't get all cores to get busy!
>
> I'm sure there must have been hours of meetings about the issue ;) I
would have prefer a QNX4 like behaviour, where everything comes from the
heap and the heap only grows and never shrinks. That's more real-time
friendly in my opinion.
>
>
>> -----Original Message-----
>> From: Shiv Nagarajan [mailto:community-noreply@qnx.com]
>> Sent: January-30-09 3:18 PM
>> To: ostech-core_os
>> Subject: Re: malloc and mmap
>>
>> yes, proc has the necessary locking gear to permit multiple mmap
>> requests at the same time.
>>
>> re: the note in amblksz requests > 32k are sent directly to mmap
>> without
>> consulting the free list, because pieces larger than 32k are not put
on
>> the free list. We cache some of the arena blocks in the library, so
if
>> you look at the "heapsize" it includes cached blocks, so some of
these
>> cache blocks could be used by a malloc for a block > 32k
>>
>> malloc itself has a large lock, so multiple threads can call malloc,
>> and
>> the underlying mmap call, is made while that malloc lock is held
>>
>> shiv
>> Fri Jan 30 15:17:51 EST 2009
>> --> According to Mario Charest <--
>> The doc on _amblksiz says memory request larger then 32KB are
>> services by mmap.
>>
>> Does that mean if the heap has 1Meg free and the code does a
>> malloc of 64K will not get the memory from the heap?
>>
>> I assume if multiple processes running on multiple cores do
>> mmap() simultaneously to obtain memory that there is some sort of
>> locking happening inside proc.
>>
>>
>>
>> _______________________________________________
>> OSTech
>> http://community.qnx.com/sf/go/post21131
>>
>> --
>> ****
>> Shiv Nagarajan,
>> Kernel Developer, QNX Software Systems,
>> Ottawa, Canada
>> ****
>>
>> _______________________________________________
>> OSTech
>> http://community.qnx.com/sf/go/post21137
>>
>
>
> _______________________________________________
> OSTech
> http://community.qnx.com/sf/go/post21142
>
--
cburgess@qnx.com
_______________________________________________
OSTech
http://community.qnx.com/sf/go/post21145
--
****
Shiv Nagarajan,
Kernel Developer, QNX Software Systems,
Ottawa, Canada
****
|
|
|
|
|
|
Mario Charest
01/30/2009 4:09 PM
post21152
|
> -----Original Message-----
> From: Shiv Nagarajan [mailto:community-noreply@qnx.com]
> Sent: January-30-09 3:48 PM
> To: ostech-core_os
> Subject: Re: malloc and mmap
>
> thanks colin.
> Thats right.
>
> MALLOC_MEMORY_PREALLOCATE=size
>
That's not in the mallopt's doc, does that mean it's only available via environment variable?
|
|
|
|
|
|
Shiv Nagarajan(deleted)
01/30/2009 4:20 PM
post21155
|
Yeah, because this option is evaluated at the first pass thru the allocator and that happens before main is invoked. And
that means it cannot be set by a mallopt option
Shiv
Shiv Nagarajan
Kernel Developer, QNX Software Systems
Canada
----- Original Message -----
From: Mario Charest <community-noreply@qnx.com>
To: ostech-core_os <post21152@community.qnx.com>
Sent: Fri Jan 30 16:09:19 2009
Subject: RE: malloc and mmap
> -----Original Message-----
> From: Shiv Nagarajan [mailto:community-noreply@qnx.com]
> Sent: January-30-09 3:48 PM
> To: ostech-core_os
> Subject: Re: malloc and mmap
>
> thanks colin.
> Thats right.
>
> MALLOC_MEMORY_PREALLOCATE=size
>
That's not in the mallopt's doc, does that mean it's only available via environment variable?
_______________________________________________
OSTech
http://community.qnx.com/sf/go/post21152
|
|
|
|
|
|
Mario Charest
01/30/2009 4:01 PM
post21151
|
> -----Original Message-----
> From: Colin Burgess [mailto:community-noreply@qnx.com]
> Sent: January-30-09 3:41 PM
> To: ostech-core_os
> Subject: Re: malloc and mmap
>
> Mario you can
>
> a) request a large preallocation upfront
> b) turn off release back to the OS
>
> via environment varilables/mallopt options. :-)
I guess since my IDE 4.5.0 is set to use 6.3.2 it showed me the 6.3.2 doc ;-) Figure it out by looking at the help on a
6.40 self hosted machine thanks.
I'm confused, _amblksiz says the max size is 256K but that anything over 32K will go through mmap? What happen if
_amblksiz is set to 128K. Also MALLOC_MEMORY_HOLD talks about not releasing memory from the arena cache, but does big
block allocated via mmap are in the arena cache?
>
> Colin
>
> Mario Charest wrote:
> > Thanks, that confirms what I'm seeing. We have a pool of processes
> each getting some work do. They all start the work pretty much at the
> same time, and that means doing a bunch of allocation simultaneously.
> But because the memory comes from the OS instead of the heap it causes
> a bottleneck and for a short while we can't get all cores to get busy!
> >
> > I'm sure there must have been hours of meetings about the issue ;) I
> would have prefer a QNX4 like behaviour, where everything comes from
> the heap and the heap only grows and never shrinks. That's more real-
> time friendly in my opinion.
> >
> >
> >> -----Original Message-----
> >> From: Shiv Nagarajan [mailto:community-noreply@qnx.com]
> >> Sent: January-30-09 3:18 PM
> >> To: ostech-core_os
> >> Subject: Re: malloc and mmap
> >>
> >> yes, proc has the necessary locking gear to permit multiple mmap
> >> requests at the same time.
> >>
> >> re: the note in amblksz requests > 32k are sent directly to mmap
> >> without
> >> consulting the free list, because pieces larger than 32k are not put
> on
> >> the free list. We cache some of the arena blocks in the library, so
> if
> >> you look at the "heapsize" it includes cached blocks, so some of
> these
> >> cache blocks could be used by a malloc for a block > 32k
> >>
> >> malloc itself has a large lock, so multiple threads can call malloc,
> >> and
> >> the underlying mmap call, is made while that malloc lock is held
> >>
> >> shiv
> >> Fri Jan 30 15:17:51 EST 2009
> >> --> According to Mario Charest <--
> >> The doc on _amblksiz says memory request larger then 32KB are
> >> services by mmap.
> >>
> >> Does that mean if the heap has 1Meg free and the code does a
> >> malloc of 64K will not get the memory from the heap?
> >>
> >> I assume if multiple processes running on multiple cores do
> >> mmap() simultaneously to obtain memory that there is some sort of
> >> locking happening inside proc.
> >>
> >>
> >>
> >> _______________________________________________
> >> OSTech
> >> http://community.qnx.com/sf/go/post21131
> >>
> >> --
> >> ****
> >> Shiv Nagarajan,
> >> Kernel Developer, QNX Software Systems,
> >> Ottawa, Canada
> >> ****
> >>
> >> _______________________________________________
> >> OSTech
> >> http://community.qnx.com/sf/go/post21137
> >>
> >
> >
> > _______________________________________________
> > OSTech
> > http://community.qnx.com/sf/go/post21142
> >
>
> --
> cburgess@qnx.com
>
>...
|
|
|
|
|
|
