But I want it so badly! All i need to figure out is:
reverse proxys (I stumbled through getting one caddy instance setup so far but gosh I struggle with that also, nginx proxy manager seems like my next step)
a rock solid backup/restore setup (but first I need to figure out where the vaultwarden alpine files live, then be able to get those off of the proxmox vm)
this is more of a vent, than a request for someone to spell it all out for me. But I wouldn’t be upset if anyone had the time to point me in the right direction for me.
Would it just be easier to run a keypass XC and syncthing setup?
Bitwarden is cheap enough, and I trust them as a company enough that I have no interest in self hosting vaultwarden.
However, all these hoops you have had to jump through are excellent learning experiences that are a benefit to apply to more of your self hosted setup.
Reverse proxies are the backbone of hosting and services these days.
Learning how to inspect docker containers, source code, config files and documentation to find where critical files are stored is extremely useful.
Learning how to set up more useful/granular backups beyond a basic VM snapshot in proxmox can be applied to any install anywhere.
The most annoying thing about a lot of these is that tutorials are “minimal viable setup” sorta things.
Like “now you have it setup, make sure you tune it for production” and it just ends.
And finding other tutorials that talk about the next step, to get things production ready, often reference out dated versions, or have different core setups so doesn’t quite apply.
I understand your frustrations.
The most annoying thing about a lot of these is that tutorials are “minimal viable setup” sorta things. Like “now you have it setup, make sure you tune it for production”
Dude I’m already in pain from trying to serve these models and you just have to go rub salt into my eyes. “Simplify your stack with <Tech>” they said. “Share your resources effectively and easily with <Tech>” they said. “Here’s your fuckin’ ‘Hello, World’ now GRTFM and buzz off” they said.
Working close to the metal do be like that.
At the homelab scale, proxmox is great.
Create a VM, install docker and use docker compose for various services.
Create additional VMs when you feel the need. You might never feel the need, and that’s fine. Or you might want a VM per service for isolation purposes.
Have proxmox take regular snapshots of the VMs.
Every now and then, copy those backups onto an external USB harddrive.
Take snapshots before, during and after tinkering so you have checkpoints to restore to. Copy the latest snapshot onto an external USB drive once you are happy with the tinkering.
Create a private git repository (on GitHub or whatever), and use it to store your docker-compose files, related config files, and little readmes describing how to get that compose file to work.
Proxmox solves a lot of headaches. Docker solves a lot of headaches. Both are widely used, so plenty of examples and documentation about them.
That’s all you really need to do.
At some point, you will run into an issue or limitation. Then you have to solve for that problem, update your VMs, compose files, config files, readmes and git repo.
Until you hit those limitations, what’s the point in over engineering it? It’s just going to over complicate things. I’m guilty of this.
Automating any of the above will become apparent when tinkering stops being fun.
The best thing to do to learn all these services is to comb the documentation, read GitHub issues, browse the source a bit.
Great points, as someone who is very happy with their current home automation and services, checking in the config files to a git repo was the critical step. Also backup volumes since many containers tend to store state in some binary or internal DB. At the very least try restoring the config to verify you have what’s needed. The containers should start even if they have no media on it.
In terms of tinkering not being fun anymore. That’s okay, sometimes you need a break.
A point that is sometimes not brought up enough in my opinion is to plan for loses. What can you afford to lose if you can’t backup everything (due to price, etc.)? config files and photos or personal data are relatively small (compared to something like a media library) and should be prioritized.
Vaultwarden itself is actually one of the easiest docker apps to deploy…if you already have the foundation of your home lab setup correctly.
The foundation has a steep learning curve.
Domain name, dynamic DNS update, port forwarding, reverse proxy. Not easy to get all this working perfectly but once it does you can use the same foundation to install any app. If you already had the foundation working, additional apps take only a few minutes.
Want ebooks? Calibre takes 10 mins. Want link archiving? Linkwarden takes 10 mins
And on and on
The foundation of your server makes a huge difference. Well worth getting it right at the start and then building on it.
I use this setup: https://youtu.be/liV3c9m_OX8
Local only websites that use https (Vaultwarden) and then external websites that also use https (jellyfin).
Honestly these things are really vital to learn if you want to be self hosting, however if you’re unfamiliar with them I would not start with your password vault. You’re almost certainly going to make mistakes and risk losing the vault. I would learn on something less vital then once you’re feeling more comfortable add vault warden.
That’s more of a general DevOps/server admin steep learning curve than Vaultwarden’s there, to be fair.
It looks a bit complicated at first as Docker isn’t a trivial abstraction, but it’s well worth it once it’s all set up and going. Each container is always the same, and always independent. Vaultwarden per-se isn’t too bad to run without a container, but the same Docker setup can be used for say, Jitsi which is an absolute mess of components to install and make work, some Java stuff, and all. But with Docker? Just docker compose up -d, wait a minute or two and it’s good to go, just need to point your reverse proxy to it.
Why do you need a reverse proxy? Because it’s a centralized location where everything comes in, and instead of having 10 different apps with their own certificates and ports, you have one proxy, one port, and a handful of certificates all managed together so you don’t have to figure out how to make all those apps play together nicely. Caddy is fine, you don’t need NGINX if you use Caddy. There’s also Traefik which lands in between Caddy and NGINX in ease of use. There’s also HAproxy. They all do the same fundamental thing: traffic comes in as HTTPS, it gets the Host header from the request and sends it to the right container as plain HTTP. Well it doesn’t have to work that way specifically but that’s the most common use case in self hosted.
As for your backups, if you used a Docker compose file, the volume data should be in the same directory. But it’s probably using some sort of database so you might want to look into how to do periodic data exports instead, as databases don’t like to be backed up live since the file is always being updated so you can’t really get a proper snapshot of it in one go.
But yeah, try to think of it as an infrastructure investment that makes deploying more apps in the future a breeze. Want to add a NextCloud? Add another docker compose file and start it, Caddy picks it up automagically and boom, it’s live and good to go!
Moving services to a new server is also pretty easy as well. Copy over your configs and composes, and volumes if applicable. Start them all, and they should all get back exactly in the same state as they were on the other box. No services to install and configure, no repos to add, no distro to maintain. All built into the container by someone else so you don’t have to worry about any of it. Each update of the app will bring with it the whole matching updated OS with the right packages in the right versions.
As a DevOps engineer we love the whole thing because I can have a Kubernetes cluster running on a whole rack and be like “here’s the apps I want you to run” and it just figures itself out, automatically balances the load, if a server goes down the containers respawn on another one and keeps going as if nothing happened. We don’t have to manually log into any of those servers to install services to run an app. More upfront work for minimal work afterwards.
Really? Have you set up services with docker before? I found it super easy compared to other systems. Curious what specifically threw you as I barely did anything except spin it up.
