How do you guys use Tailscale (or other VPN) with containers
I wanted to run my VPN/Tailscale setup past you, see if anybody has suggestions on how I could do things better.
Setup: home LAN (10.0.0.0/24), router+DNS on 10.0.0.1, server running docker containers on 10.0.0.2.
LAN DNS points *.local.dom.tld to the server, public DNS points *.dom.tld to my dynamic public IP.
Containers run in bridge mode with host, expose ports on host IPs via "ports:" mapping.
NPM with LE certs also in container, exposes 10.0.0.2:443, forwards to various other services.
Goals for Tailscale:
Accessing HTTP services via NPM from my phone when away from home.
Exposing select UDP and TCP non-HTTP services such as syncthing (:22000) or deluge RCP admin (:58846) to other tailnet devices or to phone on the go.
Goals in general:
Some containers need to expose ports on the LAN.
Some containers need to expose ports via Tailscale.
Some containers need to broadcast on the LAN (DLNA stuff) – but I don't want them broadcasting to Tailscale.
Generally speaking I'd like to explicitly control what's exposed from each container on either LAN or Tailscale.
I'd like to avoid hacking images with Dockerfile. I can make my own images to do stuff, just don't want to keep up with hacking other images.
How I progresed with Tailscale:
First tried running it directly on the host. Good: tailnet IP (let's call it 100.64.0.2) available on the host's default network stack. Containers can use "ports:" to map to 100.64.0.2 (tailscale) and/or 10.0.0.2 (LAN). Bad: tailscale would mess with /etc/resolv.conf on host. Also bad: tailscale0 on host picked up stuff that binds to 0.0.0.0.
Moved tailscale to a container running on the host network stack (network_mode: host). Made it leave /etc/resolv.conf alone. tailscale0 on host stack still picks up everything on 0.0.0.0.
This is kinda where I'm stuck. I can make the tailscale container bridged which would put the tailscale0 interface inside the container. It wouldn't pick up 0.0.0.0 from host but how would I publish ports to it?
The tailscale recommended way of doing it is by putting other containers in the tailscale's container network stack (network_mode: container:tailscale). This would prevent said containers from using "ports:" to map to host anymore. Also, everything they publish locally would end up on tailscale0 whether I like it or not.
Tailscale has an env var TS_DEST_IP that can mirror another IP. I could allocate an IP on host eth0 like 10.1.1.1, mirror that from the tailscale container, and target it from other containers explicitly with "ports:" when I want to publish a port to tailscale. Downside: 10.1.1.1 would be in the host's network stack so still picks up 0.0.0.0.
I could bridge the tailscale container with other containers on a private subnet, say 192.168.1.0/24 and use tailscale serve to forward specific ports to other containers over that subnet. Unfortunately serve is fairly limited; it can't do UDP and technically it refuses to forward TCP either to non-localhost (but you can dump the serve config to JSON, and hack that config, and use it with TS_SERVE_CONFIG= 🤮).
I could bridge tailscale with other containers and create a special container with a fixed IP on that subnet, mirror the IP from tailscale, and use iptables on that container to forward specific ports to other containers. This would actually solve everything I want except...
If I ever want to use another VPN which doesn't have the mirror feature. I don't know how I'd deal with that.
I just posted how I handled this after spending almost a day or two trying to work it out. I'm on phone right now so I will post it when I get on my pc or you can check it from my history