Project Home
Project Home		 Trackers
Trackers		 Documents
Documents		 Wiki
Wiki		 Discussion Forums
Discussions		 Project Information
Project Info
Community/Discussion/General/Slow mmap memory access/List of Posts
Forum Topic - Slow mmap memory access: (8 Items)
   
Update
Javier I. Acosta M.(deleted)
09/26/2012 11:14 AM
post95844
Slow mmap memory access   
Hi,
I have a driver that is streaming data onto DMA memory. The problem I am having is that when I try to write this data to
 the HDD since the DMA memory is so slow it is creating a bootle-neck for data processing.

Why is the mmap memory access so slow? It is in the order of 15 to 20 times slower than access non mmap memory why? Is 
there anything I can do to solve this problem?

Any help is appreciated,
JIAM
Mario Charest
09/26/2012 11:16 AM
post95845
RE: Slow mmap memory access   

> -----Message d'origine-----
> De : Javier I. Acosta M. [mailto:community-noreply@qnx.com]
> Envoyé : 26 septembre 2012 11:14
> À : general-community
> Objet : Slow mmap memory access
> 
> Hi,
> I have a driver that is streaming data onto DMA memory. The problem I am
> having is that when I try to write this data to the HDD since the DMA memory is
> so slow it is creating a bootle-neck for data processing.
> 
> Why is the mmap memory access so slow? It is in the order of 15 to 20 times
> slower than access non mmap memory why? Is there anything I can do to solve
> this problem?
> 

What type of processor?   Are you sure it`s slower, how did you come to that conclusion?

> Any help is appreciated,
> JIAM
> 
> 
> 
> _______________________________________________
> 
> General
> http://community.qnx.com/sf/go/post95844
> To cancel your subscription to this discussion, please e-mail general-
> community-unsubscribe@community.qnx.com
Javier I. Acosta M.(deleted)
09/26/2012 11:28 AM
post95846
Re: RE: Slow mmap memory access   
Processor: x86
Test: 
I timed the write to HDD operation (of a single buffer) and I did the same for a non-mapped memory and the speed is 
almost 20 times faster. The only difference I notices is that if I remove PROT_NOCACHE flag it does speed up.

Javier
Javier I. Acosta M.(deleted)
09/26/2012 11:32 AM
post95847
Re: RE: Slow mmap memory access   
Here is the mapping setting:

= mmap(0, sbuff_len, PROT_READ|PROT_WRITE | PROT_NOCACHE, MAP_PHYS|MAP_ANON|MAP_NOX64K, NOFD, 0);
Mario Charest
09/26/2012 11:51 AM
post95848
RE: RE: Slow mmap memory access   

> -----Message d'origine-----
> De : Javier I. Acosta M. [mailto:community-noreply@qnx.com]
> Envoyé : 26 septembre 2012 11:28
> À : general-community
> Objet : Re: RE: Slow mmap memory access
> 
> Processor: x86
> Test:
> I timed the write to HDD operation (of a single buffer) and I did the same for a
> non-mapped memory and the speed is almost 20 times faster. The only
> difference I notices is that if I remove PROT_NOCACHE flag it does speed up.
> 

Well here is your answer, NOCACHE it will make access much slower. Note that on most x86 hardware you don`t need this 
because the DMA/chipset will deal with the cache flushing/updating automatically.


> Javier
> 
> 
> 
> 
> _______________________________________________
> 
> General
> http://community.qnx.com/sf/go/post95846
> To cancel your subscription to this discussion, please e-mail general-
> community-unsubscribe@community.qnx.com
Javier I. Acosta M.(deleted)
09/26/2012 11:57 AM
post95849
Re: RE: RE: Slow mmap memory access   
Thanks Mario, I am testing with cache disabled but I wanted to know if there was any other tweaks I could apply. The 
fact that disabling the cache slows it that much it does not seem very convincing to me, there must be something going 
on under the hood that I am not aware.

Thanks again,
Javier
Mario Charest
09/26/2012 12:45 PM
post95850
RE: RE: RE: Slow mmap memory access   

> -----Message d'origine-----
> De : Javier I. Acosta M. [mailto:community-noreply@qnx.com]
> Envoyé : 26 septembre 2012 11:58
> À : general-community
> Objet : Re: RE: RE: Slow mmap memory access
> 
> Thanks Mario, I am testing with cache disabled but I wanted to know if there
> was any other tweaks I could apply. The fact that disabling the cache slows it
> that much it does not seem very convincing to me, there must be something
> going on under the hood that I am not aware.

Possible but modern processor suffer a MAJOR performance hit when going to system ram instead of cache. 

An i7-3960 (it has 4 channels) as a memory read speed of 20G/sec, while the L1 cache has 155G/Sec !!!!  L2 has 87G and 
L3 32G.  And the corresponding latency is RAM: 47ns, L1: 0.8ns, L2: 2.5ns, L3: 5.3ns. This is on an overlock system 
though, so the gap between memory and cache would be reduce if not overclocked.  But that gives you an idea...



> 
> Thanks again,
> Javier
> 
> 
> 
> _______________________________________________
> 
> General
> http://community.qnx.com/sf/go/post95849
> To cancel your subscription to this discussion, please e-mail general-
> community-unsubscribe@community.qnx.com
Joel Pilon(deleted)
09/26/2012 1:09 PM
post95851
Re: Slow mmap memory access   
On ARM there would be more options, but for x86 you can get some form of 
write-through cache with shm_ctl and SHMCTL_LAZYWRITE. You then need to 
map it using the FD, I can't remember if you need to map with or without 
PROT_NOCACHE.

-Joel

On 12-09-26 12:44 PM, Mario Charest wrote:
>
>
>> -----Message d'origine-----
>> De : Javier I. Acosta M. [mailto:community-noreply@qnx.com]
>> Envoyé : 26 septembre 2012 11:58
>> À : general-community
>> Objet : Re: RE: RE: Slow mmap memory access
>>
>> Thanks Mario, I am testing with cache disabled but I wanted to know if there
>> was any other tweaks I could apply. The fact that disabling the cache slows it
>> that much it does not seem very convincing to me, there must be something
>> going on under the hood that I am not aware.
>
> Possible but modern processor suffer a MAJOR performance hit when going to system ram instead of cache.
>
> An i7-3960 (it has 4 channels) as a memory read speed of 20G/sec, while the L1 cache has 155G/Sec !!!!  L2 has 87G and
 L3 32G.  And the corresponding latency is RAM: 47ns, L1: 0.8ns, L2: 2.5ns, L3: 5.3ns. This is on an overlock system 
though, so the gap between memory and cache would be reduce if not overclocked.  But that gives you an idea...
>
>
>
>>
>> Thanks again,
>> Javier
>>
>>
>>
>> _______________________________________________
>>
>> General
>> http://community.qnx.com/sf/go/post95849
>> To cancel your subscription to this discussion, please e-mail general-
>> community-unsubscribe@community.qnx.com
>
>
>
>
>
> _______________________________________________
>
> General
> http://community.qnx.com/sf/go/post95850
> To cancel your subscription to this discussion, please e-mail general-community-unsubscribe@community.qnx.com
>

Return


  A subsidiary of BlackBerry             All content ©2004-2013, QNX Software Systems