During the RAW-socket based packet send testing, I found very irritating symptoms. With a default RAW socket setting (especially for SO_SNDBUF size), the raw socket send 100,000 packets without problem but it took about 8 seconds to send all the packets, and the packets are correctly received by the receiver process. It means about 10,000 pps (packets per second) is achieved by the default setting. (I think it's too small figure contrary to my expectation.)
Anyway, to increase the pps value, I increased the packet send buffer size by adjusting the /proc/sys/net/core/{wmem_max, wmem_default}. After increasing the two system parameters, I have identified the irritating symptom. The 100,000 packets are sent promptly, but only the 3,000 packets are received by the receiver process (located at a remote node).
At the sending Linux box (Centos 5.2), I did netstat -a -s and ifconfig. Netstat showed that 100,000 requests sent out, but the ifconfig shows that only 3,000 packets are TXed.
I want to know the reason why this happens, and I also want to know how can I solve this problem (of course I don't know whether it is really a problem).
Could anybody give me some advice, examples, or references to this problem?
Best regards, bjlee
You didn't say what size your packets were or any characteristics of your network, NIC, hardware, or anything about the remote machine receiving the data.
I suspect that instead of playing with /proc/sys stuff, you should be using ethtool to adjust the number of ring buffers, but not necessarily the size of those buffers.
Also, this page is a good resource.