hping3 rtt >> sockperf latency

[Vikrantsingh]

I tried to run tcp hping3
sudo hping3 -S -p 22 10.1.0.8 -c 5
HPING 10.1.0.8 (eth0 10.1.0.8): S set, 40 headers + 0 data bytes
len=44 ip=10.1.0.8 ttl=64 DF id=0 sport=22 flags=SA seq=0 win=29200 rtt=1.6 ms
len=44 ip=10.1.0.8 ttl=64 DF id=0 sport=22 flags=SA seq=1 win=29200 rtt=3.9 ms
len=44 ip=10.1.0.8 ttl=64 DF id=0 sport=22 flags=SA seq=2 win=29200 rtt=6.0 ms
len=44 ip=10.1.0.8 ttl=64 DF id=0 sport=22 flags=SA seq=3 win=29200 rtt=4.4 ms
len=44 ip=10.1.0.8 ttl=64 DF id=0 sport=22 flags=SA seq=4 win=29200 rtt=11.9 ms

--- 10.1.0.8 hping statistic ---
5 packets transmitted, 5 packets received, 0% packet loss
round-trip min/avg/max = 1.6/5.5/11.9 ms

If I measure latency using sockperf tool avg latency is getting ~ 0.5ms
(sockperf.bin ping-pong -i 10.1.0.8 -p 8302 -t 15 --pps=max )

Does internal way of measuring tcp rtt from hping3 and tcp latency from sockperf same?
Am I doing anything wrong here?
Could you please help with this issue?

[FerasHamad]

Hi Vikrantsingh,

Two main differences between hping3 and Sockperf:

1. Hping3 measures round trip by sending number of sync messages to listening port. So it gets the measurements from N Sync messages.
   Sockperf TCP ping-pong establishes the connections then starts measuring the latency by sending/receiving messages of specific size and rate (default msg size is 14).
   • We recommend the usage of Sockperf since it also make sure all path is warm and also because it's the reference tool been used by VMA to measure latency

Below captures for both hping3 and Sockperf:

Hping3 capture for 5 iterations: “sudo hping3 -S -p 22 11.4.3.33 -c 5”:
21:21:12.822396 IP 11.4.3.32.enl > 11.4.3.33.ssh: Flags [S], seq 600893519, win 512, length 0
21:21:12.822416 IP 11.4.3.33.ssh > 11.4.3.32.enl: Flags [S.], seq 2650573128, ack 600893520, win 14600, options [mss 1460], length 0
21:21:12.822512 IP 11.4.3.32.enl > 11.4.3.33.ssh: Flags [R], seq 600893520, win 0, length 0

21:21:13.822505 IP 11.4.3.32.enl-name > 11.4.3.33.ssh: Flags [S], seq 1159970548, win 512, length 0
21:21:13.822520 IP 11.4.3.33.ssh > 11.4.3.32.enl-name: Flags [S.], seq 20173942, ack 1159970549, win 14600, options [mss 1460], length 0
21:21:13.822587 IP 11.4.3.32.enl-name > 11.4.3.33.ssh: Flags [R], seq 1159970549, win 0, length 0

21:21:14.822622 IP 11.4.3.32.musiconline > 11.4.3.33.ssh: Flags [S], seq 457729767, win 512, length 0
21:21:14.822638 IP 11.4.3.33.ssh > 11.4.3.32.musiconline: Flags [S.], seq 2794341660, ack 457729768, win 14600, options [mss 1460], length 0
21:21:14.822726 IP 11.4.3.32.musiconline > 11.4.3.33.ssh: Flags [R], seq 457729768, win 0, length 0

21:21:15.822730 IP 11.4.3.32.fhsp > 11.4.3.33.ssh: Flags [S], seq 1374506466, win 512, length 0
21:21:15.822745 IP 11.4.3.33.ssh > 11.4.3.32.fhsp: Flags [S.], seq 3278048276, ack 1374506467, win 14600, options [mss 1460], length 0
21:21:15.822838 IP 11.4.3.32.fhsp > 11.4.3.33.ssh: Flags [R], seq 1374506467, win 0, length 0

21:21:16.822836 IP 11.4.3.32.oracle-vp2 > 11.4.3.33.ssh: Flags [S], seq 595293511, win 512, length 0
21:21:16.822851 IP 11.4.3.33.ssh > 11.4.3.32.oracle-vp2: Flags [S.], seq 3302379416, ack 595293512, win 14600, options [mss 1460], length 0
21:21:16.822944 IP 11.4.3.32.oracle-vp2 > 11.4.3.33.ssh: Flags [R], seq 595293512, win 0, length 0

Part of Sockperf capture for : “sockperf ping-pong -i 11.4.3.32 -p 8302 --pps=max --tcp”:
21:29:46.049208 IP 11.4.3.33.42766 > 11.4.3.32.8302: Flags [S], seq 2164861365, win 14600, options [mss 1460,nop,wscale 9], length 0
21:29:46.049229 IP 11.4.3.32.8302 > 11.4.3.33.42766: Flags [S.], seq 549934589, ack 2164861366, win 14600, options [mss 1460,nop,wscale 9], length 0
21:29:46.049321 IP 11.4.3.33.42766 > 11.4.3.32.8302: Flags [.], ack 1, win 29, length 0
21:29:48.049595 IP 11.4.3.33.42766 > 11.4.3.32.8302: Flags [P.], seq 1:15, ack 1, win 29, length 14
21:29:48.049611 IP 11.4.3.32.8302 > 11.4.3.33.42766: Flags [.], ack 15, win 29, length 0
21:29:48.049681 IP 11.4.3.32.8302 > 11.4.3.33.42766: Flags [P.], seq 1:15, ack 15, win 29, length 14
21:29:48.049767 IP 11.4.3.33.42766 > 11.4.3.32.8302: Flags [.], ack 15, win 29, length 0
21:29:48.049776 IP 11.4.3.33.42766 > 11.4.3.32.8302: Flags [P.], seq 15:29, ack 15, win 29, length 14
21:29:48.049797 IP 11.4.3.32.8302 > 11.4.3.33.42766: Flags [P.], seq 15:29, ack 29, win 29, length 14
21:29:48.049970 IP 11.4.3.33.42766 > 11.4.3.32.8302: Flags [P.], seq 29:43, ack 29, win 29, length 14
21:29:48.050021 IP 11.4.3.32.8302 > 11.4.3.33.42766: Flags [P.], seq 29:43, ack 43, win 29, length 14
21:29:48.050197 IP 11.4.3.33.42766 > 11.4.3.32.8302: Flags [P.], seq 43:57, ack 43, win 29, length 14
21:29:48.050247 IP 11.4.3.32.8302 > 11.4.3.33.42766: Flags [P.], seq 43:57, ack 57, win 29, length 14
21:29:48.050429 IP 11.4.3.33.42766 > 11.4.3.32.8302: Flags [P.], seq 57:71, ack 57, win 29, length 14
21:29:48.050499 IP 11.4.3.32.8302 > 11.4.3.33.42766: Flags [P.], seq 57:71, ack 71, win 29, length 14
21:29:48.050710 IP 11.4.3.33.42766 > 11.4.3.32.8302: Flags [P.], seq 71:85, ack 71, win 29, length 14
21:29:48.050772 IP 11.4.3.32.8302 > 11.4.3.33.42766: Flags [P.], seq 71:85, ack 85, win 29, length 14
21:29:48.050952 IP 11.4.3.33.42766 > 11.4.3.32.8302: Flags [P.], seq 85:99, ack 85, win 29, length 14
21:29:48.051014 IP 11.4.3.32.8302 > 11.4.3.33.42766: Flags [P.], seq 85:99, ack 99, win 29, length 14
21:29:48.051210 IP 11.4.3.33.42766 > 11.4.3.32.8302: Flags [P.], seq 99:113, ack 99, win 29, length 14
21:29:48.051272 IP 11.4.3.32.8302 > 11.4.3.33.42766: Flags [P.], seq 99:113, ack 113, win 29, length 14
21:29:48.051475 IP 11.4.3.33.42766 > 11.4.3.32.8302: Flags [P.], seq 113:127, ack 113, win 29, length 14
21:29:48.051538 IP 11.4.3.32.8302 > 11.4.3.33.42766: Flags [P.], seq 113:127, ack 127, win 29, length 14
21:29:48.051680 IP 11.4.3.33.42766 > 11.4.3.32.8302: Flags [P.], seq 127:141, ack 127, win 29, length 14
21:29:48.051803 IP 11.4.3.32.8302 > 11.4.3.33.42766: Flags [P.], seq 127:141, ack 141, win 29, length 14
21:29:48.051941 IP 11.4.3.33.42766 > 11.4.3.32.8302: Flags [P.], seq 141:155, ack 141, win 29, length 14
21:29:48.051974 IP 11.4.3.32.8302 > 11.4.3.33.42766: Flags [P.], seq 141:155, ack 155, win 29, length 14
21:29:48.052032 IP 11.4.3.33.42766 > 11.4.3.32.8302: Flags [P.], seq 155:169, ack 155, win 29, length 14
21:29:48.052042 IP 11.4.3.32.8302 > 11.4.3.33.42766: Flags [P.], seq 155:169, ack 169, win 29, length 14
21:29:48.052094 IP 11.4.3.33.42766 > 11.4.3.32.8302: Flags [P.], seq 169:183, ack 169, win 29, length 14
21:29:48.052118 IP 11.4.3.32.8302 > 11.4.3.33.42766: Flags [P.], seq 169:183, ack 183, win 29, length 14
21:29:48.052210 IP 11.4.3.33.42766 > 11.4.3.32.8302: Flags [P.], seq 183:197, ack 183, win 29, length 14
21:29:48.052233 IP 11.4.3.32.8302 > 11.4.3.33.42766: Flags [P.], seq 183:197, ack 197, win 29, length 14
21:29:48.052305 IP 11.4.3.33.42766 > 11.4.3.32.8302: Flags [P.], seq 197:211, ack 197, win 29, length 14
21:29:48.052314 IP 11.4.3.32.8302 > 11.4.3.33.42766: Flags [P.], seq 197:211, ack 211, win 29, length 14
21:29:48.052356 IP 11.4.3.33.42766 > 11.4.3.32.8302: Flags [P.], seq 211:225, ack 211, win 29, length 14
.
.
.
2. With the used hping3 command “sudo hping3 -S -p 22 11.4.3.33 -c 5” it sends message per second and round trip is 0.2 ms (latency is 0.1 ms)
Sockperf doesn’t work with ping-pong of mps 1 (message per second) but with mps 2 “sockperf ping-pong -i 11.4.3.32 -p 8302 --mps=2 –tcp” shows latency 121.880 usec (~0.1 ms)

• We recommend to use 50K Messages Per Second in order not to impact latency by cache warmup

hping3 and Sockperf output is shown below:

Hping3 output:
$ sudo hping3 -S -p 22 11.4.3.33 -c 5
len=46 ip=11.4.3.33 ttl=64 DF id=0 sport=22 flags=SA seq=495 win=14600 rtt=0.2 ms
len=46 ip=11.4.3.33 ttl=64 DF id=0 sport=22 flags=SA seq=496 win=14600 rtt=0.2 ms
len=46 ip=11.4.3.33 ttl=64 DF id=0 sport=22 flags=SA seq=497 win=14600 rtt=0.2 ms
len=46 ip=11.4.3.33 ttl=64 DF id=0 sport=22 flags=SA seq=498 win=14600 rtt=0.2 ms
len=46 ip=11.4.3.33 ttl=64 DF id=0 sport=22 flags=SA seq=499 win=14600 rtt=0.2 ms

--- 11.4.3.33 hping statistic ---
500 packets transmitted, 500 packets received, 0% packet loss
round-trip min/avg/max = 0.1/0.2/0.2 ms

Sockperf output:
$ sockperf ping-pong -i 11.4.3.32 -p 8302 --mps=2 --tcp
sockperf: == version #3.1-14.gita9f6056282ef ==
sockperf[CLIENT] send on:sockperf: using recvfrom() to block on socket(s)

[ 0] IP = 11.4.3.32 PORT = 8302 # TCP
sockperf: Warmup stage (sending a few dummy messages)...
sockperf: Starting test...
sockperf: Test end (interrupted by timer)
sockperf: Test ended
sockperf: [Total Run] RunTime=1.450 sec; SentMessages=3; ReceivedMessages=3
sockperf: ========= Printing statistics for Server No: 0
sockperf: [Valid Duration] RunTime=0.000 sec; SentMessages=1; ReceivedMessages=1
sockperf: ====> avg-lat=121.880 (std-dev=0.000)
sockperf: # dropped messages = 0; # duplicated messages = 0; # out-of-order messages = 0
sockperf: Summary: Latency is 121.880 usec
sockperf: Total 1 observations; each percentile contains 0.01 observations
sockperf: ---> observation = 121.880
sockperf: ---> percentile 99.999 = 121.880
sockperf: ---> percentile 99.990 = 121.880
sockperf: ---> percentile 99.900 = 121.880
sockperf: ---> percentile 99.000 = 121.880
sockperf: ---> percentile 90.000 = 121.880
sockperf: ---> percentile 75.000 = 121.880
sockperf: ---> percentile 50.000 = 121.880
sockperf: ---> observation = 121.880

Please note that "sockperf.bin ping-pong -i 10.1.0.8 -p 8302 -t 15 --pps=max" is for UDP while for TCP you need to pass "--tcp".

Thanks

[Vikrantsingh]

Thank you for insight.