I have a gentoo diskless client that mounts using nfs and it seems like the throughput of nfs is slow. iperf show the network able to push 770 Mb/s and hdparm shows the sata disk on the server at 90 Mb/s, but doing a dd across the network only shows up as 10 Mb/s (is about 40 Mb/s when the same test is done on the server to a third machine).
Odd thing is that the pxeboot mounts nfsver2. Not sure if this is a problem or not, but other mounts after boot mount as nfsver3. Really seems like it is a config problem with nfs somewhere, but I have not found anything that makes a difference. Doing rsize=8k,wsize=8k, noatime, async. Anyone have any tips to look for or is this 10 Mb/s really the throughput I should expect for a diskless client?
Interesting what the fix was. When you mount a pxelinux nfs root, you have to set the nfs parameters in the pxelinux.cfg file (not in fstab). In setting the nfs root partition to nfsver3, tcp I got good performance gains. However, what really made the difference was removing the rsize and wsize parameters in both the pxelinux config file and fstab and letting nfs figure out what the block size should be. This let the nfs read/write perform to what I expected it to be. Here is my config and some metrics if it helps anyone else.
one ~ # nfsstat -m
/ from /dev/root
Flags: rw,vers=3,rsize=4096,wsize=4096,namlen=255,hard,nointr,nolock,proto=tcp,
timeo=600,retrans=2,sec=sys,addr=192.168.0.5
/store/tv from 192.168.99.99:/store/tv
Flags: rw,noatime,vers=3,rsize=524288,wsize=524288,namlen=255,hard,nointr,noloc
k,proto=tcp,timeo=600,retrans=2,sec=sys,mountaddr=192.168.99.99,mountvers=3,mountp
roto=tcp,addr=192.168.99.99
one ~ # time dd if=/dev/null of=/store/tv/test bs=16k count=10000
10000+0 records in
10000+0 records out
163840000 bytes (164 MB) copied, 2.79913 s, 58.5 MB/s
real 0m3.048s
user 0m0.033s
sys 0m1.983s
one pxelinux.cfg # time dd if=/store/tv/test of=/dev/null bs=16k
154614+1 records in
154614+1 records out
2533208064 bytes (2.5 GB) copied, 56.4591 s, 44.9 MB/s
real 0m56.509s
user 0m0.163s
sys 0m26.802s