I'm using pfsend tool to replay a huge pcap file borrowed from the Swedish Defence Research Agency to assess and stress Bro cluster.
I'm not able to get the result I'm expecting about the rate at which the packets inside the pcap file are sent i.e when I ask pfsend to send the pcap file on an interface let say eth2 with a rate 1.0 Gbps I'm having random rates for each set of packets. Here a snapshot of the output of pfsend.
TX rate: [current 42'437.52 pps/0.04 Gbps][average 42'437.52 pps/0.04 Gbps][total 41'569.00 pkts]
TX rate: [current 30'720.88 pps/0.03 Gbps][average 36'444.27 pps/0.03 Gbps][total 73'080.00 pkts]
TX rate: [current 86'795.22 pps/0.07 Gbps][average 53'217.22 pps/0.04 Gbps][total 160'020.00 pkts]
TX rate: [current 85'260.77 pps/0.07 Gbps][average 61'214.71 pps/0.05 Gbps][total 245'287.00 pkts]
TX rate: [current 108'434.54 pps/0.09 Gbps][average 70'687.72 pps/0.06 Gbps][total 354'329.00 pkts]
TX rate: [current 260'567.83 pps/0.22 Gbps][average 104'649.36 pps/0.09 Gbps][total 638'824.00 pkts]
TX rate: [current 13'499.74 pps/0.01 Gbps][average 90'396.03 pps/0.08 Gbps][total 654'099.00 pkts]
TX rate: [current 383'864.46 pps/0.32 Gbps][average 126'031.97 pps/0.11 Gbps][total 1'038'003.00 pkts]
TX rate: [current 248'141.39 pps/0.21 Gbps][average 139'426.46 pps/0.12 Gbps][total 1'289'802.00 pkts]
TX rate: [current 416'671.83 pps/0.35 Gbps][average 166'474.76 pps/0.14 Gbps][total 1'706'508.00 pkts]
TX rate: [current 282'022.66 pps/0.24 Gbps][average 178'789.64 pps/0.15 Gbps][total 2'051'378.00 pkts]
TX rate: [current 294'856.96 pps/0.25 Gbps][average 189'686.22 pps/0.16 Gbps][total 2'401'896.00 pkts]
TX rate: [current 188'430.59 pps/0.16 Gbps][average 189'560.85 pps/0.16 Gbps][total 2'666'556.00 pkts]
TX rate: [current 283'843.09 pps/0.24 Gbps][average 196'536.24 pps/0.16 Gbps][total 2'985'563.00 pkts]
TX rate: [current 242'010.87 pps/0.21 Gbps][average 199'394.27 pps/0.17 Gbps][total 3'232'114.00 pkts]
TX rate: [current 375'702.87 pps/0.31 Gbps][average 211'366.10 pps/0.18 Gbps][total 3'675'768.00 pkts]
TX rate: [current 245'153.10 pps/0.20 Gbps][average 213'258.35 pps/0.18 Gbps][total 3'928'703.00 pkts]
TX rate: [current 238'448.39 pps/0.20 Gbps][average 214'901.65 pps/0.18 Gbps][total 4'235'268.00 pkts]
TX rate: [current 457'495.27 pps/0.38 Gbps][average 226'618.04 pps/0.19 Gbps][total 4'692'820.00 pkts]
TX rate: [current 244'843.82 pps/0.20 Gbps][average 227'460.43 pps/0.19 Gbps][total 4'938'522.00 pkts]
TX rate: [current 175'544.43 pps/0.15 Gbps][average 224'765.27 pps/0.19 Gbps][total 5'147'218.00 pkts]
TX rate: [current 352'358.47 pps/0.33 Gbps][average 230'227.67 pps/0.19 Gbps][total 5'508'118.00 pkts]
TX rate: [current 193'981.37 pps/0.18 Gbps][average 227'879.82 pps/0.19 Gbps][total 5'829'556.00 pkts]
TX rate: [current 397'879.00 pps/0.37 Gbps][average 234'276.34 pps/0.20 Gbps][total 6'227'511.00 pkts]
TX rate: [current 262'636.29 pps/0.23 Gbps][average 235'524.01 pps/0.20 Gbps][total 6'548'784.00 pkts]
TX rate: [current 324'789.05 pps/0.27 Gbps][average 238'623.07 pps/0.20 Gbps][total 6'873'588.00 pkts]
TX rate: [current 234'831.35 pps/0.20 Gbps][average 238'479.16 pps/0.20 Gbps][total 7'140'448.00 pkts]
TX rate: [current 465'954.10 pps/0.39 Gbps][average 245'831.24 pps/0.21 Gbps][total 7'606'424.00 pkts]
TX rate: [current 236'658.29 pps/0.20 Gbps][average 245'544.05 pps/0.21 Gbps][total 7'843'092.00 pkts]
TX rate: [current 224'552.54 pps/0.44 Gbps][average 244'906.73 pps/0.22 Gbps][total 8'067'678.00 pkts]
TX rate: [current 316'888.38 pps/0.27 Gbps][average 247'027.72 pps/0.22 Gbps][total 8'384'606.00 pkts]
TX rate: [current 194'861.11 pps/0.17 Gbps][average 245'534.68 pps/0.22 Gbps][total 8'579'479.00 pkts]
TX rate: [current 247'182.76 pps/0.25 Gbps][average 245'586.19 pps/0.22 Gbps][total 8'858'096.00 pkts]
TX rate: [current 300'956.59 pps/0.28 Gbps][average 247'921.20 pps/0.22 Gbps][total 9'336'025.00 pkts]
TX rate: [current 335'746.08 pps/0.34 Gbps][average 250'723.85 pps/0.22 Gbps][total 9'752'794.00 pkts]
TX rate: [current 335'041.19 pps/0.28 Gbps][average 252'921.26 pps/0.23 Gbps][total 10'101'529.00 pkts]
TX rate: [current 261'693.83 pps/0.22 Gbps][average 253'208.11 pps/0.23 Gbps][total 10'454'834.00 pkts]
TX rate: [current 191'362.36 pps/0.16 Gbps][average 251'077.25 pps/0.22 Gbps][total 10'736'781.00 pkts]
TX rate: [current 350'642.23 pps/0.30 Gbps][average 253'483.58 pps/0.22 Gbps][total 11'108'149.00 pkts]
TX rate: [current 208'154.30 pps/0.17 Gbps][average 252'472.20 pps/0.22 Gbps][total 11'316'316.00 pkts]
TX rate: [current 211'121.31 pps/0.18 Gbps][average 251'568.45 pps/0.22 Gbps][total 11'527'754.00 pkts]
TX rate: [current 31'135.38 pps/0.03 Gbps][average 245'780.23 pps/0.22 Gbps][total 11'566'228.00 pkts]
TX rate: [current 137'098.79 pps/0.16 Gbps][average 243'442.00 pps/0.22 Gbps][total 11'708'087.00 pkts]
TX rate: [current 102'561.54 pps/0.09 Gbps][average 240'565.19 pps/0.21 Gbps][total 11'810'911.00 pkts]
TX rate: [current 114'894.56 pps/0.10 Gbps][average 238'056.49 pps/0.21 Gbps][total 11'925'812.00 pkts]
TX rate: [current 532'935.78 pps/0.45 Gbps][average 243'843.43 pps/0.22 Gbps][total 12'460'248.00 pkts]
TX rate: [current 89'972.12 pps/0.08 Gbps][average 239'563.56 pps/0.21 Gbps][total 12'591'785.00 pkts]
TX rate: [current 84'611.41 pps/0.09 Gbps][average 236'670.39 pps/0.21 Gbps][total 12'676'402.00 pkts]
As you can see the TX rate is not the same at each step and pfsend does not send the packets at the desired speed. I tried to read the source code of pfsend and understand the behavior of pfsend but I did not get it.
I can understand that the size of packets are not the same and may be the reason but I'm not sure that is the only reason. Any one can guide me and give me insights ? I will appreciate. Thanks.
Most of your trace (I downloaded just a bit) consist of TCP traffic.
tcpreplay may best suite your needs.
But be warned that this will be "fake TCP". There's no way you can dictate the egress rate of real TCP connections.
Apart form the hardware on both side, the network condition between the two, and their respective implementation of the TCP protocol (at the operating system level). Remember that TCP is a connected protocol, hence:
This is done by carefully managed receive/send window and timers in the TCP implementations at both side (again, that's done by the operating systems).
For example, if the sender send too fast and the receiver's application can't keep up, the receiver window (roughly said, the size of the buffer available for new data) will start to shrink and ultimately will go to 0. Then you'll have a "zero TCP window" condition, and the sender will stop sending anything until there's a TCP window update.
The TCP protocol is designed for this. Each side will adjust to the other + the state of the network between the two.
Indeed, if you look at the source of pfsend, you'll see some stuff for:
...IP (needed for UDP):
http://github.com/ntop/PF_RING/blob/dev/userland/examples/pfsend.c#L53
...UDP:
http://github.com/ntop/PF_RING/blob/dev/userland/examples/pfsend.c#L75
http://github.com/ntop/PF_RING/blob/dev/userland/examples/pfsend.c#L282
...but there's nothing for TCP, simply because it cannot be really done. The minimal 'help' from pfsend should clearly states that.