Getting started with iperf3 - Network Troubleshooting

CaffeineFueled

2023/03/10

iperf3 is available for all kinds of operating systems. The download page is on their official homepage. I’ll use Linux as a reference for the server and client.

Basic usage

iperf3 is a tool to measure the throughput between hosts in a network and can test TCP, UDP, and SCPT, whereby TCP is the default. iperf3 must be installed and active on two hosts in which one host acts as a server and the other one as a client. By default, you measure the upload from the client to the server, but you can test the download from the client with the -R flag.

Side note: You can use the network stack of your host (localhost) or public iperf3 servers for testing.

Server-side:
iperf3 -s 10.10.20.51 -p 5555
-s # starts iperf3 server
10.10.20.51 # sets IP on which the server is listening
-p 5555 # sets listening port to 5555. The default port is 5201
TCP connection will be tested by default
measure interval is 1 sec by default

Stop the server by pressing CTRL + c.

Client-side:
iperf3 -c 10.10.20.51 -p 5555
-c # starts iperf3 as a client
10.10.20.51 # sets the server destination
-p 5555 # sets the port on which the server is listening

Output on the client

kuser@pleasejustwork:~$ iperf3 -c 10.10.20.51 -p 5555
Connecting to host 10.10.20.51, port 5555
[  5] local 10.10.20.50 port 53512 connected to 10.10.20.51 port 5555
[ ID] Interval           Transfer     Bitrate         Retr  Cwnd
[  5]   0.00-1.00   sec  5.35 MBytes  44.9 Mbits/sec    9    197 KBytes       
[  5]   1.00-2.00   sec  4.72 MBytes  39.6 Mbits/sec    0    227 KBytes       
[  5]   2.00-3.00   sec  5.09 MBytes  42.7 Mbits/sec    1    172 KBytes       
[  5]   3.00-4.00   sec  4.65 MBytes  39.0 Mbits/sec    0    192 KBytes       
[  5]   4.00-5.00   sec  4.96 MBytes  41.6 Mbits/sec    0    206 KBytes       
[  5]   5.00-6.00   sec  4.96 MBytes  41.6 Mbits/sec    0    223 KBytes       
[  5]   6.00-7.00   sec  4.65 MBytes  39.0 Mbits/sec    0    237 KBytes       
[  5]   7.00-8.00   sec  4.84 MBytes  40.6 Mbits/sec    2    198 KBytes       
[  5]   8.00-9.00   sec  4.96 MBytes  41.6 Mbits/sec    0    220 KBytes       
[  5]   9.00-10.00  sec  4.96 MBytes  41.6 Mbits/sec    0    233 KBytes       
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.00  sec  49.2 MBytes  41.2 Mbits/sec   12             sender
[  5]   0.00-10.08  sec  48.7 MBytes  40.5 Mbits/sec                  receiver

iperf Done.
Side notes on the results of a TCP connection:
ID column shows the internal ID per stream (bi-directional, parallel, etc.)
Transfer column - how much data was transferred per interval
Bitrate column - throughput per interval
Retr column - retries needed in case of packet loss
The throughput can be found in the summary

Results are shown on both sides. If you want to retrieve the server-side output, you can use the --get-server-output flag.

iperf3 initials a so-called ‘contol connection’ to exchange parameters and exchange test results and for the test data a separate TCP connection, the flow of UDP packets or SCTP connection. If not otherwise stated, random data is used for the test.

Side note: To check the connection for Jitter, choose UDP for your test. The summary will have an additional column for jitter.

It is important to know that every server instance can only handle 1 test at a time. The server will refuse all new attempts if it is currently in use.

iperf3: error - unable to send control message: Connection reset by peer

General options

The following options can be used on the server and client sides.

Use a specific interface

Use a specific interface for iperf3:
-B IPADDRESS / -bind IPADDRESS

Measurement interval

Set a specific delay between intervals:
-i NUMBER / --interval NUMBER in seconds
the default is 1 second, and the delay between the intervals can be disabled with 0

Results / Output

Get verbose output with -V / --verbose to get a more detailed output on your test.

You can use -- logfile FILENAME to save the output to a logfile. Noted that iperf3 won’t display any output to the console and by using the same file you append to the already existing logfile.

The results are shown on both sides. If you want to retrieve the output of the server-side, you can use the --get-server-output flag.

Determine the format of the throughput:
-k/-m/-g/-t - Kbits/Mbits/Gbits/Tbits
-K/-M/-G/-T - Kbyts/Mbyts/Gbyts/Tbyts

Output the results as JSON with the -J flag.

Reference data for the transfer

Use a specific file to simulate the transfer:
-F FILENAME / --file FILENAME
otherwise, random data will be used
the actual file won’t be transferred and will only be used as a reference

Server-specific options

As mentioned in the basic usage section, you can start an iperf3 server with the -s flag.

Start iperf3 server:
-s / --server
Run the server in the background as a daemon:
-D / --daemon

Client-specific options

Initiate iperf3 connection as a client:
-c / --client

As mentioned before, by default you test the upload from the client to the server. Use the -R / --reverse flag to test the download from the server to the client.

You can use the --bidir flag to test down- and upload at the same time. Just a sample of how the output looks:

[...]
[ ID][Role] Interval           Transfer     Bitrate         Retr  Cwnd                                                                                                    
[  5][TX-C]   0.00-1.00   sec  4.17 MBytes  35.0 Mbits/sec   14    157 KBytes                                                                                             
[  7][RX-C]   0.00-1.00   sec  19.9 MBytes   167 Mbits/sec                                                                                                                
[  5][TX-C]   1.00-2.00   sec  4.10 MBytes  34.4 Mbits/sec    0    185 KBytes                                                                                             
[  7][RX-C]   1.00-2.00   sec  20.1 MBytes   169 Mbits/sec                                                                                                                
[  5][TX-C]   2.00-3.00   sec  4.47 MBytes  37.5 Mbits/sec    0    202 KBytes                                                                                             
[  7][RX-C]   2.00-3.00   sec  17.6 MBytes   147 Mbits/sec                                   
[...]
[  5][TX-C]   0.00-10.00  sec  41.5 MBytes  34.8 Mbits/sec   17             sender                                                                                        
[  5][TX-C]   0.00-10.06  sec  41.2 MBytes  34.3 Mbits/sec                  receiver                                                                                      
[  7][RX-C]   0.00-10.00  sec   205 MBytes   172 Mbits/sec  285             sender                                                                                        
[  7][RX-C]   0.00-10.06  sec   201 MBytes   168 Mbits/sec                  receiver  
Specific transport protocol:
TCP is used by default
-u / --udp for a flow of UDP packets
--sctp for a SCTP connection
Choose between IPv4 and IPv6:
-4 / --version4 # IPv4
-6 / --version6 # IPv6
Determine the lengths of the test:
-t NUMBER / --time NUMBER # time in seconds; default is 10 seconds.
Sets number of parallel client streams:
-P NUMBER / --parallel NUMBER # default is 1

Protocol specific

The following options are used for specific problems or troubleshooting sessions, but they are worth mentioning for sure.

MSS / Maximum segment size:
-M NUMBER / --set-mss NUMBER # in bytes
MTU / maximum transmission unit minus 40 bytes = MSS
Window size:
-w NUMBER[kmgt] / --window NUMBER[kmgt]
k/m/g/t - Kbits/Mbits/Gbits/Tbits
No TCP delay:
-N / --no-delay
disables “Nagle’s Algorithm”
Omit the first n seconds of the test in the beginning:
-O NUMBER / --omit NUMBER in seconds
used to ignore the TCP ramp-up phase

Sample output

[...]
[ ID] Interval           Transfer     Bitrate         Retr  Cwnd
[  5]   0.00-1.00   sec  5.57 MBytes  46.7 Mbits/sec   13    196 KBytes       (omitted)
[  5]   1.00-2.00   sec  4.65 MBytes  39.0 Mbits/sec    0    227 KBytes       (omitted)
[  5]   0.00-1.00   sec  5.09 MBytes  42.7 Mbits/sec    1    174 KBytes       
[  5]   1.00-2.00   sec  4.65 MBytes  39.0 Mbits/sec    0    192 KBytes       
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-2.00   sec  9.74 MBytes  40.9 Mbits/sec    1             sender
[  5]   0.00-2.07   sec  10.1 MBytes  41.1 Mbits/sec                  receiver



Most recent Articles:
  • Dummy IP & MAC Addresses for Documentation & Sanitization
  • Deploying ISSO Commenting System for Static Content using Docker
  • Generate a Vanity v3 Hidden Service Onion Address with mkp224o
  • ssh-audit Primer - Audit your SSH Server
  • mtr - More Detailed Traceroute - Network Troubleshooting