To make it quick, I wish I had known about port forwarding and tunneling earlier. With this blog post, I try to understand it better myself and share some experiences and tips with you. **Topics**: use cases, configuration, SSH jumphosts, local/remote/dynamic port forwarding, and limitations ## Use cases # SSH tunneling and port forwarding can be used to forward TCP traffic over a secure SSH connection from the SSH client to the SSH server, or vice versa. TCP ports or UNIX sockets can be used, but in this post I'll focus on TCP ports only. I won't go into details, but the following post should show enough examples and options to find use in your day-to-day work. Security: : encrypt insecure connections (FTP, other legacy protocols) : access web admin panels via secure SSH tunnel (Pub Key Authentication) : having potentially less ports exposed (only 22, instead of additional 80/443) Troubleshooting: : bypassing firewalls/content filters : choosing different routes Connection: : reach server behind NAT : use jumphost to reach internal servers over the internet : exposing local ports to the internet There are many more use cases, but this overview should give you a sense of possibilities. # Port forwarding Before we start: the options of the following examples and be combined and configured to suit your setup. As a side note: if the `bind_address` isn't set, localhost will be the default ## Configuration / Preparation # * The **local and remote users must have the necessary permissions** on the local and remote machines respectivly to open ports. **Ports between 0-1024 require root privileges** - if not configured differently - and the rest of the ports can be configured by standard users. * **configure clients and network firewalls accordingly** SSH port forwarding must be enabled on the server: : `AllowTcpForwarding yes` : *It is enabled by default, if I recall it correctly* If you forward ports on interfaces other than 127.0.01, then you'll need to enable `GatewayPorts` on the SSH server: : `GatewayPorts yes` Remember to **restart the ssh server service**. ## SSH jumphost / SSH tunnel # Transparently connecting to a remote host through one or more hosts. `ssh -J user@REMOTE-MACHINE:22 -p 22 user@10.99.99.1`  **Side note**: The port addressing can be removed, if the default port 22 is used! On REMOTE-MACHINE as jumphost: ```markdown [user@REMOTE-MACHINE]$ ss | grep -i ssh tcp ESTAB 0 0 167.135.173.108:ssh 192.160.140.207:45960 tcp ESTAB 0 0 10.99.99.2:49770 10.99.99.1:ssh ``` Explanation: : `167.135.173.108` - public IP of REMOTE-MACHINE : `92.160.120.207` - public IP of LOCAL-MACHINE : `10.99.99.2` - internal IP of REMOTE-MACHINE : `10.99.99.1` - internal IP of REMOTE-WEBAPP #### Using multiple jumphosts Jumphosts must be separated by commas: : `ssh -J user@REMOTE-MACHINE:22,user@ANOTHER-REMOTE-MACHINE:22 -p 22 user@10.99.99.1` ## Local Port Forwarding # #### Example 1 `ssh -L 10.10.10.1:8001:localhost:8000 user@REMOTE-MACHINE`  Access logs of the webserver on REMOTE-MACHINE that only listens on 127.0.0.1: : `127.0.0.1 - - [30/Dec/2022 18:05:15] "GET / HTTP/1.1" 200` : the request originates from LOCAL-MACHINE #### Example 2 `ssh -L 8001:10.99.99.1:8000 user@REMOTE-MACHINE`  Access logs of the webserver on REMOTE-WEBAPP: : `10.99.99.2 - - [30/Dec/2022 21:28:42] "GET / HTTP/1.1" 200` : the request originates from the intern IP of LOCAL-MACHINE (10.99.99.2) ## Remote Port Forwarding # #### Example 1+2 `ssh -R 8000:localhost:8001 user@REMOTE-MACHINE`  `ssh -R 8000:10.10.10.2:8001 user@REMOTE-MACHINE`  #### Example 3 `ssh -R 10.99.99.2:8000:10.10.10.2:8001 user@REMOTE-MACHINE`  **Important**: `GatewayPorts yes` must be enabled on the SSH server to listen on another interface than the loopback interface. ## Dynamic port forwarding # To forward more than one port, SSH uses the [SOCKS](https://en.wikipedia.org/wiki/SOCKS) protocol. This is a transparent proxy protocol and SSH makes us of the most recent version SOCKS5. Default port for SOCKS5 server is 1080 as defined in [RFC 1928](https://datatracker.ietf.org/doc/html/rfc1928). The client must be configured correctly to use a SOCKS proxy. Either on the application or OS layer. #### Example `ssh -D 10.10.10.1:5555 user@REMOTE-MACHINE`  Use `curl` on a 'LOCAL' client to test the correct connection/path: : `curl -L -x socks5://10.10.10.1:5555 brrl.net/ip` : *If everything works out, you should get the public IP of the REMOTE-MACHINE back* ## SSH TUN/TAP tunneling I won't go into detail, but you can create a bi-directional TCP tunnel with the `-w` flag. The interfaces must be created beforehand, and I haven't tested it yet. `-w local_tun[:remote_tun]` ## How to run SSH in the background # The native way to run the tunnel in the background would be `-fN`: : `-f` - run in the background : `-N` - no shell `ssh -fN -L 8001:127.0.0.1:8000 user@REMOTE-MACHINE` Others than that: use screen or some other tools. #### Stop the SSH running in the background ```markdown user@pleasejustwork:~$ ps -ef | grep ssh [...] user 19255 1 0 11:40 ? 00:00:00 ssh -fN -L 8001:127.0.0.1:8000 user@REMOTE-MACHINE [...] ``` Kill the process with the PID: : `kill 19255` ## Keep SSH connection alive I won't go into detail, but there are different ways to keep the SSH connection alive. #### Handle timeouts with heartbeats Both options can be set on the client or server, or both. `ClientAliveInterval` will send a request every `n` seconds to keep the connection alive: : `ClientAliveInterval 15` `ClientAliveCountMax` is the number of heartbeat requests sent after not receiving an respond from the other side of the connection before terminating the connection: : `ClientAliveCountMax 3` : `3` is the default, and setting it to `0` will disable connection termination. In this example, the connection would drop after around 45 seconds without any responds. #### Reconnecting after termination There are mutliple ways to do it; autossh, scripts, cronjobs, and so on. This is beyond this post and I might write about in the future. ## Limitations # #### UDP SSH depends on a reliable delivery to be able to decrypt everything correctly. UDP does not offer any reliability and is therefore not supported and recommended to use over the SSH tunnel. That said, there are ways to do it as described in [this post](http://zarb.org/~gc/html/udp-in-ssh-tunneling.html). I still need to test it. #### TCP-over-TCP It lowers the throughput due to more overhead and increases the latency. On connections with packet loss or high latencies (e.x. satellite) it can cause a [TCP meltdown](https://openvpn.net/faq/what-is-tcp-meltdown/). [This post](http://sites.inka.de/sites/bigred/devel/tcp-tcp.html) is a great write-up. Nevertheless, I'd been using OpenVPN-over-TCP for a while, and it worked flawlessly. Less throughput than UDP, but reliable. So, it highly depends on your setup. #### Not a VPN replacement Overall, it is not a VPN replacement. SSH tunneling can be used as such, but a VPN is better suited for better performance. #### Potential security risk If you do not need those features, it is recommended to turn them of. Threat actors could use said features to avoid firewalls and other security measures. --- General links: : [SSH manual](https://www.man7.org/linux/man-pages/man1/ssh.1.html) : [sshd_config manual](https://www.man7.org/linux/man-pages/man5/sshd_config.5.html) The inspiration of this blog post are the following [unix.stackexchange answer](https://unix.stackexchange.com/a/115906) and [blog post of Dirk Loss](http://dirk-loss.de/ssh-port-forwarding.htm). Thanks to Frank and ruffy for valuable feedback! --- {{% contact %}} --- **More reading:** {{% post-list tags=ssh stop=5 %}}{{% /post-list %}} {{% post-list stop=5 %}}{{% /post-list %}}