Introduction

Introduction

  ·  3 min read

So I’ve decided to build what might be eventually considered a homelab. Every homelab tagged post after this one will chronicle what I’ve done so far.

Much credit goes to Ben Vincent who gave me the idea of what hardware and software to use and answered my endless stupid questions so far.

Where I am today #

Currently I have a disparate set of random computers that run all the time and provide the functionality I “require” at home.

There is no rhyme or reason to it. It is an evolved mess.

All of it is extremely manually configured. If i lost a machine I’d need to rebuild it from scratch. I do back up important data, but I don’t have any real semblance of config as code. I don’t even put my dotfiles in git like a normal person.

Current hardware:

Name OS CPU, RAM, Storage Description
dan Fedora 42 AMD Ryzen 5 7600X, 64GB, 512GB SSD, 3 x 4TB HDD Runs random stuff and stores all my data on zfs raidz1
bob FreeBSD AMD Turion II Neo N54L, 16GB, 256GB SSD, 3 x 4TB HDD HP Microserver N54L - a backup of dan
alice FreeBSD AMD Turion II Neo N54L, 16GB, 256GB SSD HP Microserver N54L - used to be a backup of bob, because bob did the job of dan. Now gathers dust
glootie Ubuntu 24.04.1 LTS Intel Celeron N3050, 4GB, 512GB SSD Oldish Intel NUC - runs nginx and reverse proxies and TLS terminates all my sites for some reason
will Fedora 40 Intel i3 8109U, 24GB, 2 x 256GB SSD Newish Intel NUC - I run IRC from it. Not sure why
bill Windows 11 Intel i9-13900K, 32GB, 2TB SSD My desktop and gaming machine. Has an ssh client too

Where I want to be #

Ultimately, I’d like to be able to magically bootstrap an entire cluster of machines with as little manual effort as possible from nothing.

All the hardware above is a mess, so I want to migrate everything to be automatically provisioned by config as code:

Name OS CPU, RAM, Storage Description
TBA AlmaLinux 9.6 i7-10700T / 64GB / TBA Dell Optiplex 7080 - One of n servers provisioned via config as code entirely.
TBA AlmaLinux 9.6 i7-10700T / 64GB / TBA [same as above]

I have two machines so far to get up and running and start experimenting.

Goals:

  • No manual configuration or installations or provisioning of containers. All config and code in source control.
  • As much as practical self-hosted. No reliance on external services within reason. Exceptions:
    • Fetching source from e.g. github
    • Fetching release binaries from e.g. github
    • OS updates and packages
  • Be able to bootstrap the entire lab from scratch with absolute minimal manual setup
  • No cyclic dependencies if possible
  • Minimise single points of failure towards zero
  • Liberal use of metrics and alerting
  • Logging pipeline
  • Must not be fundamentally different to how things are done at work (e.g. no nixos/guix) as hopefully some of this will be useful.

Phase 1 #

Manually install the OS and containers, automatically provision software within the containers.

Rough plan:

  1. Install AlmaLinux 9 manually on these machines.
  2. Provision incus semi-manually to provide a bunch of containers within the machines.
  3. Configure each container to run something via ansible. These will gradually take over the functions provided by the legacy systems.

Consider adding helper infrastructure on the way, including:

  • CI/CD pipelines
  • Local gitea instance for git and self-hosting of home-grown rpms.
  • consul
  • vault
  • packer

Phase 2 #

Manually install the OS, provision containers via terraform.

Phase 3 #

Automatically install the OS via PXE and bootstrap.

Phase 4 #

High availability – every service will stay up if one computer fails.

Play with Ceph Storage