Homelab

Don't be fooled, having a home server is really just hundreds of hours of badblocks.

Features

• Proxmox physical hosts and Arch Linux virtual guests
• Lots of self-hosted services in a portable setup
• Ansible node provisioning and container deployments (Ansible roles and compose files)
• Terraform DNS records setup
• SOPS secrets stored in Git
• Renovate bot dependency updates
• Cloudflared HTTP tunnel
• OIDC authentication with LDAP
• Automatic Cloudflare DNS updates
• ZFS
• JBOD mergerfs union NFS with SnapRAID backup for low-touch media files snapraid-runner cronjob
• Restic backups to remote and local buckets
• go-task shorthand for useful commands (Taskfile and taskfiles)

Historical revisions of this repository went from a single-node compose orchestration, then Podman rootless containers deployed with Ansible as systemd units, then a kubernetes cluster extended from this template. With other responsibilities, I've had to take on a much more minimal approach to my homelab and I strive for simplicity over high availability at this time.

Usage

Setup

Install go-task

task init

Then, provision your infrastructure.

task ansible:{list,setup,status}

Edit provision/terraform/cloudflare/secret.sops.yaml with your own values and encrypt with task sops:encrypt -- <filepath>.

Setup Cloudflare DNS.

task terraform:{init,cloudflare-plan,cloudflare-apply}

Bastion server

Edit provision/terraform/bastion/secret.sops.yaml with your own values. Generate WireGuard keys.

Deploy the remote bastion VPN server.

task terraform:{init,plan,apply}

Then, setup VPN services.

task ansible:bastion

Deployments

Most deployments in this repo use an app-template chart with these configuration options.

Update

The Renovate bot will help find updates for charts and images. Install Renovate Bot, add to your repository and view Renovate bot activity, or use the self-hosted option.

Hardware

Resources

• DataHoarder Wiki: Hardware
• 2020 DIY NAS
• 2020 Economy DIY NAS
• Home server case recommendations

Memory

• Why use ECC (discussion)
• If you love your data, use ECC RAM.
• Error rates increase rapidly with rising altitude.

Storage

Controller

I used a widely-known and inexpensive method to add additional SATA storage via a Host Bus Adapter (HBA). I purchased a Dell Perc H310 a long while back. Mine did come from overseas, but it turned out to be legit. This video shows how it can be flashed to an LSI 9211-8i IT (see also 1, 2).

Here are other recommended controllers.

2.5" drive stackers

These printable stackers are great for stacking SSDs in a homelab.

Home automation

IoT

• USB Zigbee/Z-Wave receiver and upgrade Zigbee firmware for compatibility with Home Assistant (notice this issue)
• Zigbee/Matter receiver

Software

Linux

• Arch Linux
• Raspberry Pi Debian

SSH

Here's a nice convenience for setting up authorized_keys stored on Github or Gitlab:

curl https://github.com/<username>.keys -o authorized_keys

You could pipe the output to sed to only grab a specific line sed '4!d'.

Check disks

Here's a handy script to automatically test disks with badblocks and SMART: Spearfoot/disk-burnin-and-testing.

Testing disks takes a long time for larger drives, but it's worth it to be thorough before determining whether to make a return. This is a destructive test, so it's probably best to use /dev/disk/by-id to be certain you're targeting the correct drive.

Use tune2fs -l <partition> to identify the block size.

sudo badblocks -wsv -b 4096 /dev/sda > sda_badblocks.txt

Here's some additional advice from /r/DataHoarders.

JBOD

MergerFS is a union filesystem for pooling drives together. It's a great pair with SnapRAID. An alternative is SnapRAID-BTRFS.

mkdir /mnt/disk{1,2,3,4}
mkdir /mnt/parity1 # adjust this command based on your parity setup
mkdir /mnt/storage # this will be the main mergerfs mount point (a collection of your drives)

Mount drives to these folders, then add /etc/fstab entries by ID.

ls /dev/disk/by-id

You must also include an entry for the MergerFS union, such as:

/mnt/disk* /mnt/storage fuse.mergerfs allow_other,use_ino,cache.files=partial,dropcacheonclose=true,category.create=mfs,fsname=mergerfs,minfreespace=10G 0 0

See also perfectmediaserver: MergerFS

Remember, for data that's irreplaceable RAID is not a backup.

ZFS

Install zfs-dkms and zfs-utils, and be sure to have linux-headers installed for dkms to work. Update the ZFS libraries together using a AUR helper.

OS Installation

Use Ventoy to bundle bootable ISO and IMG images on a single USB.

Setup Proxmox on the hosts with Arch Linux guests. Post setup for Proxmox.

Media

For a media server, it's a good idea to understand digital video.