Hi Aboyd, 

Thanks a lot for replying me.

> Can you post the nicinfo output?  Also, if you
> can start the mpc85xx driver with the verbose
> option, and post the sloginfo output, that will
> help diagnose why the packets are being dropped.
==> We have turned on the verbose mode, Drop packet does not produce any more messages. 

>  
> Frankly, I'm surprised - the tsec/etsec hardware
> is dma-ring based, and I've seen gige wire rate
> with this hardware and driver.
>  
> I will be interested to learn if your hardware is
> etsec or (older, slower) tsec - I remember on
> the 8313 (with the tsec) it was a bit of a trick 
> to get it's speed up.
==> We are using io-net based tsec, not the io-pkt native 85xx tsec driver.
   Not sure this makes a difference. Our 8572E TSEC is the latest freescale chip, We use the driver QNX provided us.

> We need to figure out if you have a bus bandwidth
> problem - nic is unable to DMA fast enough - OR 
> if you are simply running out of receive descriptors 
> because of excessive scheduling latency.
==> When packets are dropped, increase io-pkt rx thread's priority reduce it at a ratio 1000 to 3. The problem is 
scheduling latency.  Also we calculate the packets received by TSEC HW is what it supposed to receive and nicinfo 
records packet drop. This leads to the conclusion of excessive scheduling latency.

> If it's door #2, you can try specifying "receive=2048"
> or some other large number to mpc85xx to give it
> more receive buffers, to tolerate scheduling latency.
==> I shall find out if receive=2048 will improve it, since I do not know the default setting.

> Are there any really high priority threads running
> which would starve the driver?
==> Yes, we do have our own hardware demanding CPU at interrupt priority.

Btw,

We found after MDI_ResetPhy and MDI_SyncPhy, MDI_AutoNegotiate, The AutoNegotiate would fail a few time. This causes the
 Rx reports CRC error. I am not sure Drop packet is related.

Regards,
Sherry

> Hi Sherry,
> 
> > - FTP large file (>40M) transfer slows down significantly after a couple 
> iterations.
> > The more transfer, the worse the transfer.
> Are you saying that each FTP file transfer is slower than the previous 
> one or that each file transfer slows down during the transfer after 
> initially having a higher speed?
==> Yes, Exactly. Compare to previous transfer of the same file, it takes longer. Here, I am not sure this is caused by 
packet drop or CRC error (see my previous message).
Part of this info was given to me.

As for my own test, 1) The packet drop case, Initially, it drops much less for the same file, as iterate goes on, drop 
count increases then levels out at about the same level. 2) CRC error case, it is caused by bad auto-negotiation, It 
seems taking even longer for the same file to transfer, may be the effective baud rate was bad, even ifconfig says it 
transfer at the same 100M FD. My college told me, it starts fast, then gradually   slow down, eventually failed. 

> 
> In general, packet drop is treated as congestion by TCP. I suggest you 
> start look at why you have packet drop (incl. the suggestions Andrew 
> posted) 
==> We very possibly have a scheduling latency problem, since increase io-pkt rx priority relieves the symptom 
dramatically. I do not know if bad auto-negotiation would cause TSEC HW to report packet drop. Could you clarify this? I
 do not have experience with this situation.

and if that doesn't make anything obvious you should look at 
> what TCP congestion algorithms you employ (SACK, New Reno, socket 
> options, etc, - see e.g sysctl -a |grep tcp) and analyze a packet trace 
> of the TCP session.
> 
> Hope this helps!
> /P

The following is my sysctl -a| grep tcp. Analyze the packet trace will take me a while,
 I am not a TCP person yet.

test_eth(130):set_verbose=0, rc=0
# sysctl -a | grep tcp
net.inet.tcp.rfc1323 = 1
net.inet.tcp.sendspace = 22528
net.inet.tcp.recvspace = 32768
net.inet.tcp.mssdflt = 536
net.inet.tcp.syn_cache_limit = 10255
net.inet.tcp.syn_bucket_limit = 105
net.inet.tcp.init_win = 0
net.inet.tcp.mss_ifmtu = 0
net.inet.tcp.sack.enable = 1
net.inet.tcp.sack.maxholes = 32
net.inet.tcp.sack.globalmaxholes = 1024
net.inet.tcp.sack.globalholes = 0
net.inet.tcp.win_scale = 1
net.inet.tcp.timestamps = 1
net.inet.tcp.compat_42 = 0
net.inet.tcp.cwm = 0
net.inet.tcp.cwm_burstsize = 4
net.inet.tcp.ack_on_push = 0
net.inet.tcp.keepidle = 14400
net.inet.tcp.keepintvl = 150
net.inet.tcp.keepcnt = 8
net.inet.tcp.slowhz = 2
net.inet.tcp.log_refused = 0
net.inet.tcp.init_win_local = 4
net.inet.tcp.rstppslimit = 100
net.inet.tcp.delack_ticks = 23
net.inet.tcp.hiwat_adjust = 1
net.inet.tcp.do_loopback_cksum = 0
net.inet.tcp.congctl.available = reno newreno
net.inet.tcp.congctl.selected = newreno
net.inet.tcp.ecn.enable = 0
net.inet.tcp.ecn.maxretries = 1
net.inet.tcp.iss_hash = 0
net.inet.tcp.abc.enable = 1
net.inet.tcp.abc.aggressive = 1
qnx.net.inet.tcp.hiwat_adjust = 1