Jump to ID:
QNX Operating System

Project Home

Documents

Discussions

Wiki

Project Info
Forum Topic - malloc and mmap: (9 Items)
   
 
 
malloc and mmap  
The doc on _amblksiz says memory request larger then 32KB are services by mmap.  

Does that mean if the heap has 1Meg free and the code does a malloc of 64K will not get the memory from the heap?

I assume if multiple processes running on multiple cores do mmap() simultaneously to obtain memory that there is some 
sort of locking happening inside proc.

Re: malloc and mmap  
Why do you think there is no locking in heap allocation routine? ;-)
Re: malloc and mmap  
yes, proc has the necessary locking gear to permit multiple mmap
requests at the same time. 

re: the note in amblksz requests > 32k are sent directly to mmap without
consulting the free list, because pieces larger than 32k are not put on
the free list. We cache some of the arena blocks in the library, so if
you look at the "heapsize" it includes cached blocks, so some of these
cache blocks could be used by a malloc for a block > 32k

malloc itself has a large lock, so multiple threads can call malloc, and
the underlying mmap call, is made while that malloc lock is held

shiv
Fri Jan 30 15:17:51 EST 2009
 --> According to Mario Charest <--
	The doc on _amblksiz says memory request larger then 32KB are services by mmap.  
	
	Does that mean if the heap has 1Meg free and the code does a malloc of 64K will not get the memory from the heap?
	
	I assume if multiple processes running on multiple cores do mmap() simultaneously to obtain memory that there is some 
sort of locking happening inside proc.
	
	
	
	_______________________________________________
	OSTech
	http://community.qnx.com/sf/go/post21131

-- 
****
Shiv Nagarajan,
Kernel Developer, QNX Software Systems,
Ottawa, Canada
****
RE: malloc and mmap  
Thanks,  that confirms what I'm seeing. We have a pool of processes each getting some work do. They all start the work 
pretty much at the same time, and that means doing a bunch of allocation simultaneously.  But because the memory comes 
from the OS instead of the heap it causes a bottleneck and for a short while we can't get all cores to get busy!  

I'm sure there must have been hours of meetings about the issue ;) I would have prefer a QNX4 like behaviour, where 
everything comes from the heap and the heap only grows and never shrinks. That's more real-time friendly in my opinion. 



> -----Original Message-----
> From: Shiv Nagarajan [mailto:community-noreply@qnx.com]
> Sent: January-30-09 3:18 PM
> To: ostech-core_os
> Subject: Re: malloc and mmap
> 
> yes, proc has the necessary locking gear to permit multiple mmap
> requests at the same time.
> 
> re: the note in amblksz requests > 32k are sent directly to mmap
> without
> consulting the free list, because pieces larger than 32k are not put on
> the free list. We cache some of the arena blocks in the library, so if
> you look at the "heapsize" it includes cached blocks, so some of these
> cache blocks could be used by a malloc for a block > 32k
> 
> malloc itself has a large lock, so multiple threads can call malloc,
> and
> the underlying mmap call, is made while that malloc lock is held
> 
> shiv
> Fri Jan 30 15:17:51 EST 2009
>  --> According to Mario Charest <--
> 	The doc on _amblksiz says memory request larger then 32KB are
> services by mmap.
> 
> 	Does that mean if the heap has 1Meg free and the code does a
> malloc of 64K will not get the memory from the heap?
> 
> 	I assume if multiple processes running on multiple cores do
> mmap() simultaneously to obtain memory that there is some sort of
> locking happening inside proc.
> 
> 
> 
> 	_______________________________________________
> 	OSTech
> 	http://community.qnx.com/sf/go/post21131
> 
> --
> ****
> Shiv Nagarajan,
> Kernel Developer, QNX Software Systems,
> Ottawa, Canada
> ****
> 
> _______________________________________________
> OSTech
> http://community.qnx.com/sf/go/post21137
> 
Re: malloc and mmap  
Mario you can

a) request a large preallocation upfront
b) turn off release back to the OS

via environment varilables/mallopt options. :-)

Colin

Mario Charest wrote:
> Thanks,  that confirms what I'm seeing. We have a pool of processes each getting some work do. They all start the work
 pretty much at the same time, and that means doing a bunch of allocation simultaneously.  But because the memory comes 
from the OS instead of the heap it causes a bottleneck and for a short while we can't get all cores to get busy!  
> 
> I'm sure there must have been hours of meetings about the issue ;) I would have prefer a QNX4 like behaviour, where 
everything comes from the heap and the heap only grows and never shrinks. That's more real-time friendly in my opinion. 

> 
> 
>> -----Original Message-----
>> From: Shiv Nagarajan [mailto:community-noreply@qnx.com]
>> Sent: January-30-09 3:18 PM
>> To: ostech-core_os
>> Subject: Re: malloc and mmap
>>
>> yes, proc has the necessary locking gear to permit multiple mmap
>> requests at the same time.
>>
>> re: the note in amblksz requests > 32k are sent directly to mmap
>> without
>> consulting the free list, because pieces larger than 32k are not put on
>> the free list. We cache some of the arena blocks in the library, so if
>> you look at the "heapsize" it includes cached blocks, so some of these
>> cache blocks could be used by a malloc for a block > 32k
>>
>> malloc itself has a large lock, so multiple threads can call malloc,
>> and
>> the underlying mmap call, is made while that malloc lock is held
>>
>> shiv
>> Fri Jan 30 15:17:51 EST 2009
>>  --> According to Mario Charest <--
>> 	The doc on _amblksiz says memory request larger then 32KB are
>> services by mmap.
>>
>> 	Does that mean if the heap has 1Meg free and the code does a
>> malloc of 64K will not get the memory from the heap?
>>
>> 	I assume if multiple processes running on multiple cores do
>> mmap() simultaneously to obtain memory that there is some sort of
>> locking happening inside proc.
>>
>>
>>
>> 	_______________________________________________
>> 	OSTech
>> 	http://community.qnx.com/sf/go/post21131
>>
>> --
>> ****
>> Shiv Nagarajan,
>> Kernel Developer, QNX Software Systems,
>> Ottawa, Canada
>> ****
>>
>> _______________________________________________
>> OSTech
>> http://community.qnx.com/sf/go/post21137
>>
> 
> 
> _______________________________________________
> OSTech
> http://community.qnx.com/sf/go/post21142
> 

-- 
cburgess@qnx.com
Re: malloc and mmap  
thanks colin.
Thats right. 

MALLOC_MEMORY_PREALLOCATE=size

would pre-allocate "size" core memory up front, on the first entry into the 
allocator.

and MALLOC_MEMORY_HOLD=1

will never release memory back to the system from the heap.

shiv
Fri Jan 30 15:47:53 EST 2009
 --> According to Colin Burgess <--
	Mario you can
	
	a) request a large preallocation upfront
	b) turn off release back to the OS
	
	via environment varilables/mallopt options. :-)
	
	Colin
	
	Mario Charest wrote:
	> Thanks,  that confirms what I'm seeing. We have a pool of processes
	each getting some work do. They all start the work pretty much at the
	same time, and that means doing a bunch of allocation simultaneously.
	But because the memory comes from the OS instead of the heap it causes a
	bottleneck and for a short while we can't get all cores to get busy!  
	> 
	> I'm sure there must have been hours of meetings about the issue ;) I
	would have prefer a QNX4 like behaviour, where everything comes from the
	heap and the heap only grows and never shrinks. That's more real-time
	friendly in my opinion. 
	> 
	> 
	>> -----Original Message-----
	>> From: Shiv Nagarajan [mailto:community-noreply@qnx.com]
	>> Sent: January-30-09 3:18 PM
	>> To: ostech-core_os
	>> Subject: Re: malloc and mmap
	>>
	>> yes, proc has the necessary locking gear to permit multiple mmap
	>> requests at the same time.
	>>
	>> re: the note in amblksz requests > 32k are sent directly to mmap
	>> without
	>> consulting the free list, because pieces larger than 32k are not put
	on
	>> the free list. We cache some of the arena blocks in the library, so
	if
	>> you look at the "heapsize" it includes cached blocks, so some of
	these
	>> cache blocks could be used by a malloc for a block > 32k
	>>
	>> malloc itself has a large lock, so multiple threads can call malloc,
	>> and
	>> the underlying mmap call, is made while that malloc lock is held
	>>
	>> shiv
	>> Fri Jan 30 15:17:51 EST 2009
	>>  --> According to Mario Charest <--
	>> 	The doc on _amblksiz says memory request larger then 32KB are
	>> services by mmap.
	>>
	>> 	Does that mean if the heap has 1Meg free and the code does a
	>> malloc of 64K will not get the memory from the heap?
	>>
	>> 	I assume if multiple processes running on multiple cores do
	>> mmap() simultaneously to obtain memory that there is some sort of
	>> locking happening inside proc.
	>>
	>>
	>>
	>> 	_______________________________________________
	>> 	OSTech
	>> 	http://community.qnx.com/sf/go/post21131
	>>
	>> --
	>> ****
	>> Shiv Nagarajan,
	>> Kernel Developer, QNX Software Systems,
	>> Ottawa, Canada
	>> ****
	>>
	>> _______________________________________________
	>> OSTech
	>> http://community.qnx.com/sf/go/post21137
	>>
	> 
	> 
	> _______________________________________________
	> OSTech
	> http://community.qnx.com/sf/go/post21142
	> 
	
	-- 
	cburgess@qnx.com
	
	_______________________________________________
	OSTech
	http://community.qnx.com/sf/go/post21145
	

-- 
****
Shiv Nagarajan,
Kernel Developer, QNX Software Systems,
Ottawa, Canada
****
RE: malloc and mmap  

> -----Original Message-----
> From: Shiv Nagarajan [mailto:community-noreply@qnx.com]
> Sent: January-30-09 3:48 PM
> To: ostech-core_os
> Subject: Re: malloc and mmap
> 
> thanks colin.
> Thats right.
> 
> MALLOC_MEMORY_PREALLOCATE=size
> 

That's not in the mallopt's doc, does that mean it's only available via environment variable?
Re: malloc and mmap  
Yeah, because this option is evaluated at the first pass thru the allocator and that happens before main is invoked. And
 that means it cannot be set by a mallopt option

Shiv
Shiv Nagarajan
Kernel Developer, QNX Software Systems
Canada

----- Original Message -----
From: Mario Charest <community-noreply@qnx.com>
To: ostech-core_os <post21152@community.qnx.com>
Sent: Fri Jan 30 16:09:19 2009
Subject: RE: malloc and mmap



> -----Original Message-----
> From: Shiv Nagarajan [mailto:community-noreply@qnx.com]
> Sent: January-30-09 3:48 PM
> To: ostech-core_os
> Subject: Re: malloc and mmap
> 
> thanks colin.
> Thats right.
> 
> MALLOC_MEMORY_PREALLOCATE=size
> 

That's not in the mallopt's doc, does that mean it's only available via environment variable?

_______________________________________________
OSTech
http://community.qnx.com/sf/go/post21152

RE: malloc and mmap  

> -----Original Message-----
> From: Colin Burgess [mailto:community-noreply@qnx.com]
> Sent: January-30-09 3:41 PM
> To: ostech-core_os
> Subject: Re: malloc and mmap
> 
> Mario you can
> 
> a) request a large preallocation upfront
> b) turn off release back to the OS
> 
> via environment varilables/mallopt options. :-)

I guess since my IDE 4.5.0 is set to use 6.3.2 it showed me the 6.3.2 doc ;-)  Figure it out by looking at the help on a
 6.40 self hosted machine thanks.

I'm confused, _amblksiz says the max size is 256K but that anything over 32K will go through mmap?  What happen if 
_amblksiz is set to 128K.  Also MALLOC_MEMORY_HOLD talks about not releasing memory from the arena cache, but does big 
block allocated via mmap are in the arena cache?

> 
> Colin
> 
> Mario Charest wrote:
> > Thanks,  that confirms what I'm seeing. We have a pool of processes
> each getting some work do. They all start the work pretty much at the
> same time, and that means doing a bunch of allocation simultaneously.
> But because the memory comes from the OS instead of the heap it causes
> a bottleneck and for a short while we can't get all cores to get busy!
> >
> > I'm sure there must have been hours of meetings about the issue ;) I
> would have prefer a QNX4 like behaviour, where everything comes from
> the heap and the heap only grows and never shrinks. That's more real-
> time friendly in my opinion.
> >
> >
> >> -----Original Message-----
> >> From: Shiv Nagarajan [mailto:community-noreply@qnx.com]
> >> Sent: January-30-09 3:18 PM
> >> To: ostech-core_os
> >> Subject: Re: malloc and mmap
> >>
> >> yes, proc has the necessary locking gear to permit multiple mmap
> >> requests at the same time.
> >>
> >> re: the note in amblksz requests > 32k are sent directly to mmap
> >> without
> >> consulting the free list, because pieces larger than 32k are not put
> on
> >> the free list. We cache some of the arena blocks in the library, so
> if
> >> you look at the "heapsize" it includes cached blocks, so some of
> these
> >> cache blocks could be used by a malloc for a block > 32k
> >>
> >> malloc itself has a large lock, so multiple threads can call malloc,
> >> and
> >> the underlying mmap call, is made while that malloc lock is held
> >>
> >> shiv
> >> Fri Jan 30 15:17:51 EST 2009
> >>  --> According to Mario Charest <--
> >> 	The doc on _amblksiz says memory request larger then 32KB are
> >> services by mmap.
> >>
> >> 	Does that mean if the heap has 1Meg free and the code does a
> >> malloc of 64K will not get the memory from the heap?
> >>
> >> 	I assume if multiple processes running on multiple cores do
> >> mmap() simultaneously to obtain memory that there is some sort of
> >> locking happening inside proc.
> >>
> >>
> >>
> >> 	_______________________________________________
> >> 	OSTech
> >> 	http://community.qnx.com/sf/go/post21131
> >>
> >> --
> >> ****
> >> Shiv Nagarajan,
> >> Kernel Developer, QNX Software Systems,
> >> Ottawa, Canada
> >> ****
> >>
> >> _______________________________________________
> >> OSTech
> >> http://community.qnx.com/sf/go/post21137
> >>
> >
> >
> > _______________________________________________
> > OSTech
> > http://community.qnx.com/sf/go/post21142
> >
> 
> --
> cburgess@qnx.com
> 
>...