intermittent.network blog

The initial tech stack

Goals

I wanted to try self-hosting something, and I wanted to see how far I could get combining an old PC with a modern CDN.

I also really don’t want to get DoSed - or worse - have someone pop a shell on my webserver and then pivot to my entire home.

The architecture

Building from the inside out, the architecture looks like this.

The old PC

I have a Lenovo H30-05 mini-PC sitting in my office. (Several years back I walked into an OfficeWorks where it was a display model they were trying to get rid of - I think it set me back all of $60!) This is not a fast computer: it has a 2 core, 1.5GHz AMD processor, 4 GB of RAM, and spinning rust storage. It’s so low-power that its PSU is an external laptop-style brick rather than your usual desktop PSU.

The computer has been running as my print server for some time. Out of a misplaced desire to broaden my experience from just Linux, I tried to set it up with FreeBSD. In hindsight, an AirPrint enabled print server was the wrong first FreeBSD project! But I really really did not want to fight with CUPS again, and I’ve wanted a good excuse to try jails for a long time, so I stuck with it. All of this would work just fine with Linux, just substitute containers in place of jails, and nftables instead of pf and so on.

Inside the PC: FreeBSD and nginx

The web server that hosts the authoritative copy of the page you’re reading is an nginx server that runs in the www jail. The www jail has a virtual ethernet device connected to a local bridge that only connects to another jail, the ingress jail. There’s no internet access within the jail at all.

The ingress jail is connected to the internal bridge, and has another virtual ethernet device which is bridged out to my home network. The external interface only has an IPv6 address - no IPv4. Given how huge the IPv6 address space is, I’m hoping that makes it basically unfindable by automated scanners.

So to fetch this page, a connection comes in via IPv6, nginx does TLS termination, proxies the connection to the www jail, and returns the result.

Outside the jail the pf firewall is set up to ensure that the ingress jail can’t make outbound IPv6 connections, only respond to inbound ones. So again, no internet access unless you can do a jail escape.

PC to outside world

The next stop in our journey is my router, which needs a small pinhole in the firewall.

IPv6, or rather, software support for it, makes this harder.

As far as I know, my ISP is one of the few in Australia that offers a static IPv6 prefix. At least they used to: after several rounds of being eaten by larger ISPs, they’re pretty quiet about static IPv6 now… Hopefully it still works, because my router is very consumer-grade and will only let me specify individual IPv6 addresses to receive inbound connections. No wildcards here! I’ve specified the current public IPv6 address, so if the prefix changes that will break.

This is definitely the most fragile part of this whole setup, which is a good opportunity to remind you that rather than bookmarking this website or hoping it shows up in a search engine in future, you should just clone the git repo!

Annoyingly this is a problem that dynamic DNS can’t solve: even if I could change a domain name, I don’t know how to automate rewriting the IPv6 firewall rules.

Outside world to you

Maybe this is unusual for self-hosting, but I decided to serve my site via a CDN. I picked Fastly because I work there and therefore I know the platform, but there’s no reason you couldn’t use another CDN.

This gives me a few benefits:

A have a theory that a lot of the risks and challenges of self-hosting become a lot more tractable if you design with a CDN. There are providers with free plans, or the ability to limit your spend.

The journey of a request

So then, say you load this blog post.

Does that mean there’s no dynamic content at all?

Well no, I do have one little status thingy using fastfetch.

Do we meet our goals?

Self-hosting

This machine in my house has half the cores of a Raspberry Pi 5, each running at less than two thirds of the speed. It’s not a quick machine. Fastly, on the other hand, has lots of very fast machines. I suppose time will tell, but I think this setup will work to keep the load on my machine manageable.

Hopefully I will write a followup post on the simple things I do to optimise the caching that the CDN provides. None of it is complex but it’s a bit fiddly to bend nginx to your will!

Security

DoS

I rely on my home internet working for such trivial niceties as “being able to work from home”, so I’m keen to avoid things that might allow someone to flood my fairly small pipe. Work here includes:

Ultimately it’s very difficult to prevent a DDoS - all of this is ultimately probabilistic - but hopefully it’s enough to avoid common attacks.

Popping a shell and pivoting

Obviously it’s not ideal if someone defaces the website, but the thing I’m really most concerned about is pivoting to my home network.

Updates

<< Previous Post

|

Next Post >>