For trying out new things, sometimes I want to make them accessible from the internet. I’ve got a small reverse proxy VPS on one of the cloud hosts. This is how I currently set them up on a VM in my basement and expose them via that proxy. Writing out all the steps from install to accessibility makes this seem like a lot of work, but it isn’t hard and goes quickly.

My VM server runs an ESXi 6.5 hypervisor, primarily because my VM zoo is compatible with that and it’s nice to be able to run them from either Workstation Pro or ESXi. If I were building from scratch today, I might make a different choice. The specific hypervisor is not particularly important to me lately, though. What I really need is an easy way to host containers. I’m using alpine for that right now.

VMWare’s photon project seems like a very promising container host for an ESXi environment, but it’s not readily compatible with wireguard. Fedora CoreOS looks like it will be an awesome container host overall in the future, but using it with wireguard and docker-compose files currently involves shaving a couple of yaks. And wireguard in particular is really important to my reverse proxy setup. Alpine is very nearly as fast as photon, so the only downside is that I have the minor mental overhead of two different package management toolchains on my systems.

Container Host Setup

I like to deploy one container host per project. When I start a new one, I grab the most recent iso from the alpine download site and install using the ESXi web console and mostly sticking with the alpine defaults.

I use the web console long enough to set a root password and temporary modify the sshd configuration so that I can use it for ssh. Then I set up my keys, set up wireguard, set up docker/git, and I’m off.

Initial OS Setup

  • From the console, install the ncurses and python2 packages so that terminfo can be configured for kitty (my preferred terminal)
apk add ncurses python2
  • SSH in as root. Since I use kitty and have too many ssh keys on my machine, I need to force terminfo update and password use:
kitty +kitten ssh -o PreferredAuthentications=password -o PubkeyAuthentication=no root@$VM_IP
  • Add my ssh keys and restore the ssh configuration to the default, then bounce sshd again.
rc-service sshd restart
  • Install sudo and create a non-root user for myself
apk add sudo
# visudo and enable ALL for wheel
adduser geoff
adduser geoff wheel
cp -rv ~/.ssh ~geoff/
chown -R geoff:geoff ~geoff/.ssh
  • Log in as my non-root user and work that way from now on.

Install docker, git and docker-compose

  • First enable the community repository and run apk update
  • Then:
sudo apk add docker py-pip python3-dev libffi-dev openssl-dev gcc libc-dev make git docker-compose
  • Add myself to the docker group and enable the daemon at boot time
sudo addgroup geoff docker
sudo rc-update add docker boot
sudo service docker start

Log out and back in for the group addition to take effect.

Test run

git clone https://git.sr.ht/~tuxpup/stock-toolkit
cd stock-toolkit

Create .env with site-appropriate values, e.g.:

POSTGRES_ADDRESS=pgsql
POSTGRES_PORT=5432
POSTGRES_DB=stock_toolkit
POSTGRES_USER=fastapi
POSTGRES_PASSWORD=CHANGEME
APP_HOST=0.0.0.0
APP_PORT=8000

docker-compose build
docker-compose up

At this point, a browser connection to port 8000 should work. I generally go and add a static entry for the backend server in my DHCP server and DNS server for easy access at this point, then kill the services.

Connect to VPN/Reverse Proxy

From here, all that’s left is to connect the project VM to my reverse proxy VPS using wireguard, then tell the reverse proxy about the new app/name.

On the project VM:

apk add wireguard-lts wireguard-tools-wg-quick wireguard-tools-wg bash
sudo -s
mkdir -p /etc/wireguard
umask 077
cd /etc/wireguard
wg genkey | tee /etc/wireguard/privatekey | wg pubkey > /etc/wireguard/publickey

Then create /etc/wireguard/wg0.conf with the server details:

[Interface]
Address    = 10.20.30.3/24
PrivateKey = CONTENTS_OF_CLIENT/etc/wireguard/privatekey

[Peer]
PublicKey    = PUBLIC_KEY_FROM_SERVER
PresharedKey = CONTENTS_OF_SERVER/etc/wireguard/psk
Endpoint = server.example.com:51820
AllowedIPs   = 10.20.30.1/32
PersistentKeepAlive = 25

On the VPS, I need to add a block to /etc/wireguard/wg0.conf for the new client:

[Peer]
# stock-toolkit
PublicKey    = cM+JW5wZHkqHApmPQNkVPpmKUbD9PiEM+muj4Sol9jc=
PresharedKey = CONTENTS_OF_SERVER/etc/wireguard/psk
AllowedIPs   = 10.20.30.3/32

then add a block to the Caddyfile for the new subdomain I’m proxying:

stock-toolkit.tuxpup.com {
	reverse_proxy 10.20.30.3:8000
log {
	output file /var/log/caddy/stock-toolkit.tuxpup.com.caddy_access.log
}
}

and restart caddy so it can go get a TLS cert from LetsEncrypt for stock-toolkit.tuxpup.com:

systemctl restart caddy

(Getting the cert takes a few minutes, especially the first time.)

Then on the client, once I bring the wireguard connection up (as root), the VPN should be in place:

wg-quick up wg0

And I can start the app’s services as a normal user, within the app checkout:

docker-compose up -d

Writing out all the steps in this level of detail makes this sound like a great deal of trouble. But it’s quick to do, and is the nicest balance I’ve found for a local development server I can work on effectively and share easily online without punching holes in my firewall.