Yes Robert, I'm surprised too, and I was expecting the same conclusions you mention. Mario, I was in the idea that a 
memmove is a memmove (almost an assembly level function and DMA's task)

Thanks very much for your answers!!

Regards,
Juan Manuel

Memcopy is NOT a dma task, however there are lots of method to do it. Intel has a few pages on the subject memmove, for 
example some can use SSEx instructions/registers!!

> -----Original Message-----
> From: Juan Manuel Placco [mailto:community-noreply@qnx.com]
> Sent: December-03-08 12:55 PM
> To: technology-networking
> Subject: Re: QNX 6.4 - QNX 6.3 Comparison
> 
> Yes Robert, I'm surprised too, and I was expecting the same conclusions
> you mention. Mario, I was in the idea that a memmove is a memmove
> (almost an assembly level function and DMA's task)
> 
> Thanks very much for your answers!!
> 
> Regards,
> Juan Manuel
> 
> 
> _______________________________________________
> Technology
> http://community.qnx.com/sf/go/post17863
> 

Hi Juan:

	I finally got around to looking through your test code and there
are problems in there with the way that you're doing your timing
calculation in the client.

The resolution of the internal clock is usually set to 1 ms, so if your
single msg send takes less than this amount of time to run, then you'll
end up with invalid timing results.  Instead of doing your timing check
around the single message send and then summing the results together in
your main function (totmsg += mseg), you need to do the timing
comparison around your "for (i = 0; i < cant_retries; i++)" loop to get
the total time.  And you need to make absolutely sure that the number of
retries that you pass in gives you a good amount of time in order for
the timing results to be valid.

Also, the server side doesn't properly handle the network message case.
From the docs for MsgReceive:

"In a local message pass, the kernel would ordinarily limit the size of
the transfer to the minimum of both sizes. But in the networked case,
the message is received by the client's lsm-qnet.so into its own private
buffers and then sent via transport to the remote lsm-qnet.so. Since the
size of the server's receive data area can't be known in advance by the
client's lsm-qnet.so when the message is sent, only a fixed maximum size
(currently 8 KB) message is transferred between the client and the
server. "

(See the sample code for how this should be handled).


	Robert.


 

-----Original Message-----
From: Juan Manuel Placco [mailto:community-noreply@qnx.com] 
Sent: Wednesday, December 03, 2008 12:55 PM
To: technology-networking
Subject: Re: QNX 6.4 - QNX 6.3 Comparison

Yes Robert, I'm surprised too, and I was expecting the same conclusions
you mention. Mario, I was in the idea that a memmove is a memmove
(almost an assembly level function and DMA's task)

Thanks very much for your answers!!

Regards,
Juan Manuel


_______________________________________________
Technology
http://community.qnx.com/sf/go/post17863

> Hi Juan:
> 
> 	I finally got around to looking through your test code and there
> are problems in there with the way that you're doing your timing
> calculation in the client.
> 
> The resolution of the internal clock is usually set to 1 ms, so if your
> single msg send takes less than this amount of time to run, then you'll
> end up with invalid timing results.  Instead of doing your timing check
> around the single message send and then summing the results together in
> your main function (totmsg += mseg), you need to do the timing
> comparison around your "for (i = 0; i < cant_retries; i++)" loop to get
> the total time.  And you need to make absolutely sure that the number of
> retries that you pass in gives you a good amount of time in order for
> the timing results to be valid.
>
> 	Robert.

Thanks Robert for take your time to answer me... but... humm... wait, please...

>>  Instead of doing your timing check around the single message send

I'm not measuring a single message time... I think... Look,

In 'send_msgs' funtion I send lot of messages!! (cant_msgs = 5000 to 100000)

mseg = send_msgs ( cant_msgs, mensaje );
totmseg += mseg;

I'm totalizing the amount of time in the loop inside 'send_msgs'. Look, this is 'send_msgs' function...

	// Arranco a contar tiempo	
    ftime ( &time_ini );
    stime_ini = ctime ( &time_ini.time );
 
	for ( i=0; i< cant_msg; i++ )
	{
		ret = MsgSend ( _connect_id, mensaje, strlen(mensaje)+1, reply_msg, sizeof(reply_msg) );
		
		if ( ret < 0 )
		{ 
		    printf("Error in MsgSend errno=[%d] - '%s' (ret=%d)\n", errno, strerror ( errno ), ret );
		}
	}

	// Termino de contar tiempo
    ftime ( &time_fin );
    stime_fin = ctime ( &time_fin.time );

	return ( difftimeb (time_fin, time_ini) );

And 'cant_msg' is big enought. 'cant_msg' is, much more bigger than 'cant_retries'. In fact, I could avoid 
'cant_retries' loop at all. This just was a simple (and innocuous) idea to have more 'samples' of the same test to 
calculate an average. The 'REAL' loop is the one I show you before inside the function 'send_msgs' ... for ( i=0; i< 
cant_msg; i++ ) ... 'cant_msg' values varies from 5000 to 10000!!.. while 'cant_retries' was fixed to 10. Even I though 
to let cant_retries equal to just 1 or not make THAT loop at all.

I could be confused, sorry, but I'm calculating the time to send at least 5000 messages!! and as the test goes on, this 
loop reaches values of 100000 messages. And I start to count time before the 1st msg and stop after de 5000 or 100000th.
.. 
After that I have the other (and maybe obsolete) loop, to average 10 times the same 5000 or 100000 msg test...

Please, let me know if I confused, but I think 5000 messages is enought to totalize, isn't? Do you think, I have to send
 more than 5000 messages to have a good result?

> Also, the server side doesn't properly handle the network message case.
> From the docs for MsgReceive:
> 
> "In a local message pass, the kernel would ordinarily limit the size of
> the transfer to the minimum of both sizes. But in the networked case,
> the message is received by the client's lsm-qnet.so into its own private
> buffers and then sent via transport to the remote lsm-qnet.so. Since the
> size of the server's receive data area can't be known in advance by the
> client's lsm-qnet.so when the message is sent, only a fixed maximum size
> (currently 8 KB) message is transferred between the client and the
> server. "
> 
> (See the sample code for how this should be handled).

My sended messages were of 1K exactly (< 8K), so I supposed that I didn't have to worry about it. Is this wrong?

Thank you very much again, and please, let me know your opinions.

Regards, 
Juan Manuel

Hi Juan:
	Missing out on the command line would certainly have led to part
of the confusion :->>.  From running the code on my machine, 1000
iterations is not enough, 5000 iterations and up are probably OK
(although you should also make sure that you're running the test at a
very high priority to make sure that other system processes aren't
pre-empting the test).

OK...  So what I was REALLY looking for was the code that you used to do
the message size comparisons locally.  My preliminary tests showed very
little difference between 6.3.2 and 6.4.0.

	R.


-----Original Message-----
From: Juan Manuel Placco [mailto:community-noreply@qnx.com] 
Sent: Thursday, December 04, 2008 4:03 PM
To: technology-networking
Subject: Re: RE: QNX 6.4 - QNX 6.3 Comparison

> Hi Juan:
> 
> 	I finally got around to looking through your test code and there
are 
> problems in there with the way that you're doing your timing 
> calculation in the client.
> 
> The resolution of the internal clock is usually set to 1 ms, so if 
> your single msg send takes less than this amount of time to run, then 
> you'll end up with invalid timing results.  Instead of doing your 
> timing check around the single message send and then summing the 
> results together in your main function (totmsg += mseg), you need to 
> do the timing comparison around your "for (i = 0; i < cant_retries; 
> i++)" loop to get the total time.  And you need to make absolutely 
> sure that the number of retries that you pass in gives you a good 
> amount of time in order for the timing results to be valid.
>
> 	Robert.

Thanks Robert for take your time to answer me... but... humm... wait,
please...

>>  Instead of doing your timing check around the single message send

I'm not measuring a single message time... I think... Look,

In 'send_msgs' funtion I send lot of messages!! (cant_msgs = 5000 to
100000)

mseg = send_msgs ( cant_msgs, mensaje ); totmseg += mseg;

I'm totalizing the amount of time in the loop inside 'send_msgs'. Look,
this is 'send_msgs' function...

	// Arranco a contar tiempo	
    ftime ( &time_ini );
    stime_ini = ctime ( &time_ini.time );
 
	for ( i=0; i< cant_msg; i++ )
	{
		ret = MsgSend ( _connect_id, mensaje, strlen(mensaje)+1,
reply_msg, sizeof(reply_msg) );
		
		if ( ret < 0 )
		{ 
		    printf("Error in MsgSend errno=[%d] - '%s'
(ret=%d)\n", errno, strerror ( errno ), ret );
		}
	}

	// Termino de contar tiempo
    ftime ( &time_fin );
    stime_fin = ctime ( &time_fin.time );

	return ( difftimeb (time_fin, time_ini) );

And 'cant_msg' is big enought. 'cant_msg' is, much more bigger than
'cant_retries'. In fact, I could avoid 'cant_retries' loop at all. This
just was a simple (and innocuous) idea to have more 'samples' of the
same test to calculate an average. The 'REAL' loop is the one I show you
before inside the function 'send_msgs' ... for ( i=0; i< cant_msg; i++ )
... 'cant_msg' values varies from 5000 to 10000!!.. while 'cant_retries'
was fixed to 10. Even I though to let cant_retries equal to just 1 or
not make THAT loop at all.

I could be confused, sorry, but I'm calculating the time to send at
least 5000 messages!! and as the test goes on, this loop reaches values
of 100000 messages. And I start to count time before the 1st msg and
stop after de 5000 or 100000th... 
After that I have the other (and maybe obsolete) loop, to average 10
times the same 5000 or 100000 msg test...

Please, let me know if I confused, but I think 5000 messages is enought
to totalize, isn't? Do you think, I have to send more than 5000 messages
to have a good result?

> Also, the server side doesn't properly handle the network message
case.
> From the docs for MsgReceive:
> 
> "In a local message pass, the kernel would ordinarily limit the size 
> of the transfer to the minimum of both...

>> My preliminary tests showed very little 
>> difference between 6.3.2 and 6.4.0.

SO... This was longer as I thought :) sorry. But there're still some little things rounding my head. Let'see if I can 
get just one final idea. Please.

For 1024 bytes local messages I have aprox 17% "more efficient" in 6.4. 

But again, for longer messages the difference is HUGE (better in 6.4!!)

But why SO HUGE? (and we must agree that is a VERY good thing!)... But just, I want to know why? 

This are the numbers (always with more than 5000 msg, look at the excel file): (We're talking again about message 
passing-same node).

Bytes	"QNX 6.4 Better performance than 6.3 [%]"
10	3.90
1200	26.28
4000	63.67
8000	83.78
16000	89.92
64000	92.64   ( WOW !!!!)

COULD THIS BE CORRECT?

Please... Take a little time to look at this numbers again, here I re-post the excel file.

AND REALLY THANK YOU !!!!!!

Regards,
Juan Manuel

Hi Juan:
	I need to see the code that you used for testing longer messages
in order to be able to comment.

	Robert. 

-----Original Message-----
From: Juan Manuel Placco [mailto:community-noreply@qnx.com] 
Sent: Friday, December 05, 2008 1:20 PM
To: technology-networking
Subject: Re: RE: RE: QNX 6.4 - QNX 6.3 Comparison

>> My preliminary tests showed very little difference between 6.3.2 and 
>> 6.4.0.

SO... This was longer as I thought :) sorry. But there're still some
little things rounding my head. Let'see if I can get just one final
idea. Please.

For 1024 bytes local messages I have aprox 17% "more efficient" in 6.4. 

But again, for longer messages the difference is HUGE (better in 6.4!!)

But why SO HUGE? (and we must agree that is a VERY good thing!)... But
just, I want to know why? 

This are the numbers (always with more than 5000 msg, look at the excel
file): (We're talking again about message passing-same node).

Bytes	"QNX 6.4 Better performance than 6.3 [%]"
10	3.90
1200	26.28
4000	63.67
8000	83.78
16000	89.92
64000	92.64   ( WOW !!!!)

COULD THIS BE CORRECT?

Please... Take a little time to look at this numbers again, here I
re-post the excel file.

AND REALLY THANK YOU !!!!!!

Regards,
Juan Manuel


_______________________________________________
Technology
http://community.qnx.com/sf/go/post18040

Ok, Robert, here it is.

The differences with the previous are:

1) Another loop to change the message size (10, 1200, and so on...)
2) Using variable type: char mensaje [64000];
3) In order to use strlen() when sending, I truncate with '\0' the message at the desired lenght. Then I pass 
'strlen(mensaje)+1' to MsgSend function's 3rd arg.
4) Apart from that, is the same code.

5) Ah. A big one!... I corrected the -m usage to 'number of messages to send' as hp suggested. :-)

Thank you!

Regards,
Juan Manuel

Hi:
	I've managed to take a quick look at this and I seem to be
getting similar results to you.  I THINK that at least part of this
large difference is compiler related.

If you compile the binaries under 6.3.2 and run them on 6.4, they
produce the same results as running them on 6.3.2.  Use a 6.4 compiled
binary for either the server or the client and you'll see that the
results improve.

I'm not quite sure what this means.  The funny thing is that when I
specified optimization levels with 6.3.2 (e.g. -O3 and -O2), the results
got worse rather than better.

Can you confirm?


   Robert.


 

-----Original Message-----
From: Juan Manuel Placco [mailto:community-noreply@qnx.com] 
Sent: Friday, December 05, 2008 5:20 PM
To: technology-networking
Subject: Re: RE: RE: RE: QNX 6.4 - QNX 6.3 Comparison

Ok, Robert, here it is.

The differences with the previous are:

1) Another loop to change the message size (10, 1200, and so on...)
2) Using variable type: char mensaje [64000];
3) In order to use strlen() when sending, I truncate with '\0' the
message at the desired lenght. Then I pass 'strlen(mensaje)+1' to
MsgSend function's 3rd arg.
4) Apart from that, is the same code.

5) Ah. A big one!... I corrected the -m usage to 'number of messages to
send' as hp suggested. :-)

Thank you!

Regards,
Juan Manuel


_______________________________________________
Technology
http://community.qnx.com/sf/go/post18061

Humm... I see.... very interesting...
I was waiting this answer anxiously

BTW... I'm flying to Argentina right now, but you can be sure I'll check this again when I return (to Mexico), on early 
january, God willing.

Thank you very much Robert, and...

"Merry Christmas for you and Foundry27 community!!"

And thank everyone for the support. You are awesome !!!

I learned A LOT from all of you. Thank you !! 

Regards,
Juan Manuel

> Hi:
> 	I've managed to take a quick look at this and I seem to be
> getting similar results to you.  I THINK that at least part of this
> large difference is compiler related.
> 
> If you compile the binaries under 6.3.2 and run them on 6.4, they
> produce the same results as running them on 6.3.2.  Use a 6.4 compiled
> binary for either the server or the client and you'll see that the
> results improve.
> 
> I'm not quite sure what this means.  The funny thing is that when I
> specified optimization levels with 6.3.2 (e.g. -O3 and -O2), the results
> got worse rather than better.
> 
> Can you confirm?
> 

Hi again. At last I could make me the time to continue with this test! 

The pending question was: to compile the binaries in 6.3 and run them on 6.4 and see what happens. That is what I did.

The results are in the increasingly complex attached Excel file :) (I'm sure you can go directly to the BLUE bar graph 
at the end)

Ok, as I see "for messages > 1200 bytes":

1) The same code (6.3 compiled binary) run faster in 6.4 than in 6.3, so it can be that 6.4 kernel is faster, because 
the code is the same. Right?

2) The 6.3 compiled binary running on 6.4 is slower than 6.4 binary compiled and also running on 6.4. So the code was 
optimized too. Right?

So, we can say both ("for msgs >1200 bytes"): 

* 6.4 kernel is faster than 6.3
* 6.4 compiled code is faster (an much smaller) than 6.3

Do you agree with these first conclutions?

BUT !!
---------

Why the results "for messages < 1200" are not the same? This confuses me with the previous conclusions, because in small
 msgs, the context-switch overhead is heavier... So the context-switch (kernel) is slower in this case? Who can it be?

Is not so simple, but I need to understand what is going on...

Please, I will greatly appreciate your own conclusions about this..

Thank you in advance!!!

Regards,
Juan Manuel

OK...  My head is spinning with all of those numbers :>

Let me see if I can summarise your question...

Running a 6.4 binary on 6.4 is always faster than the 6.3 version at all
times.

When you compare running the 6.3 binary on 6.4 with running the 6.3
binary on 6.3, for all cases EXCEPT the 10 byte case,  a performance
increase is seen.  For the 10 byte case, a performance decrease is seen.
Why?

I'll wait and see if that's the question before I try to answer :>.

	Robert.



-----Original Message-----
From: Juan Manuel Placco [mailto:community-noreply@qnx.com] 
Sent: Tuesday, January 27, 2009 6:07 PM
To: technology-networking
Subject: Re: RE: RE: RE: RE: QNX 6.4 - QNX 6.3 Comparison

> Hi:
> 	I've managed to take a quick look at this and I seem to be
getting 
> similar results to you.  I THINK that at least part of this large 
> difference is compiler related.
> 
> If you compile the binaries under 6.3.2 and run them on 6.4, they 
> produce the same results as running them on 6.3.2.  Use a 6.4 compiled

> binary for either the server or the client and you'll see that the 
> results improve.
> 
> I'm not quite sure what this means.  The funny thing is that when I 
> specified optimization levels with 6.3.2 (e.g. -O3 and -O2), the 
> results got worse rather than better.
> 
> Can you confirm?
> 

Hi again. At last I could make me the time to continue with this test! 

The pending question was: to compile the binaries in 6.3 and run them on
6.4 and see what happens. That is what I did.

The results are in the increasingly complex attached Excel file :) (I'm
sure you can go directly to the BLUE bar graph at the end)

Ok, as I see "for messages > 1200 bytes":

1) The same code (6.3 compiled binary) run faster in 6.4 than in 6.3, so
it can be that 6.4 kernel is faster, because the code is the same.
Right?

2) The 6.3 compiled binary running on 6.4 is slower than 6.4 binary
compiled and also running on 6.4. So the code was optimized too. Right?

So, we can say both ("for msgs >1200 bytes"): 

* 6.4 kernel is faster than 6.3
* 6.4 compiled code is faster (an much smaller) than 6.3

Do you agree with these first conclutions?

BUT !!
---------

Why the results "for messages < 1200" are not the same? This confuses me
with the previous conclusions, because in small msgs, the context-switch
overhead is heavier... So the context-switch (kernel) is slower in this
case? Who can it be?

Is not so simple, but I need to understand what is going on...

Please, I will greatly appreciate your own conclusions about this..

Thank you in advance!!!

Regards,
Juan Manuel




_______________________________________________
Technology
http://community.qnx.com/sf/go/post20891

>>Let me see if I can summarise your question...
>>
>>Running a 6.4 binary on 6.4 is always faster than the 6.3 version at all
>>times.
>>
>> When you compare running the 6.3 binary on 6.4 with running the
>> 6.3 binary on 6.3, for all cases EXCEPT the 10 byte case,  a
>> performance increase is seen.  For the 10 byte case, a performance
>> decrease is seen. Why?

>> I'll wait and see if that's the question before I try to answer :>.

Hi Robert!

I delay my answer to do ALL this tests again in order to be totally sure

That is EXACTLY what is happening. The question is why? To what conclusions can we arrive?

Thank you in advance!

Regards,
Juan Manuel

Hi... is there any news about this topic?

Thank you!!

Juan Manuel

Ummm... I thought in your last posting you mentioned:

"I delay my answer to do ALL this tests again in order to be totally
sure "

So I was waiting for you to confirm your results (?)

I have to admit that I'm floundering on this one.  Without some
extensive profiling (of system and application)
I'd only be making speculative guesses.  Maybe this is one that would be
best asked in the OS forums?

	R.

-----Original Message-----
From: Juan Manuel Placco [mailto:community-noreply@qnx.com] 
Sent: Tuesday, February 03, 2009 6:15 PM
To: technology-networking
Subject: Re: RE: RE: RE: RE: RE: QNX 6.4 - QNX 6.3 Comparison

Hi... is there any news about this topic?

Thank you!!

Juan Manuel


_______________________________________________
Technology
http://community.qnx.com/sf/go/post21370

> Ummm... I thought in your last posting you mentioned:
> 
> "I delay my answer to do ALL this tests again in order to be totally
> sure "
> 
> So I was waiting for you to confirm your results (?)
> 
> I have to admit that I'm floundering on this one.  Without some
> extensive profiling (of system and application)
> I'd only be making speculative guesses.  Maybe this is one that would be
> best asked in the OS forums?
> 
> 	R.
> 
> -----Original Message-----
> From: Juan Manuel Placco [mailto:community-noreply@qnx.com] 
> Sent: Tuesday, February 03, 2009 6:15 PM
> To: technology-networking
> Subject: Re: RE: RE: RE: RE: RE: QNX 6.4 - QNX 6.3 Comparison
> 
> Hi... is there any news about this topic?
> 
> Thank you!!
> 
> Juan Manuel
> 
> 
> _______________________________________________
> Technology
> http://community.qnx.com/sf/go/post21370


Hi Robert, thank you very much for the anwer.

Yes... You are right but I've already tried once in the OS forum, but with less lucky than here :), a much worst 
response time :)

Anyway, if you have a feeling, I'am sure it will be better than mines, so I would be extremely grateful to have your 
opinion. 

The question is exactly what you summarised in last post.

>> Let me see if I can summarise your question...

>> Running a 6.4 binary on 6.4 is always faster than the 6.3 version at
>> all times.

>> When you compare running the 6.3 binary on 6.4 with running 
>> the 6.3 binary on 6.3, for all cases EXCEPT the 10 byte case,  
>> a performance increase is seen.  For the 10 byte case, 
>> a performance decrease is seen. Why?

>> I'll wait and see if that's the question before I try to answer :>.

>>	Robert.

The answer is YES. 

BUT, if you think the only way is to re-post to the OS Forum, just let me know...

Thank you in advance, Robert. 

Regards,
Juan Manuel