gargravarr2112B
Power in the UK has gone through the roof. I’ve downsized my lab as much as I can and have at times wondered if I should shut it down completely.
Originally I was running an EdgeRouter 4, Zyxel 48-port managed switch and custom-built NAS with an i3-9100T, 32GB ECC and 6x 12TB SAS drives in a zpool. The NAS did everything - VMs, storage, backups etc. but it was pulling quite a lot of power.
A while back I ran a USFF PC as my server, which idled at 8W. Versus my 200W Xeon machine at the time, it paid for itself in 12 months. I dug that out and moved the VMs onto it. Storage went onto an ARM NAS. I was running too many VMs for a single USFF even maxed out, so I bought another 2 of them (identical). Now I run them in a Proxmox cluster. I use a passive cooled HP 1810 managed switch and an EdgeRouter Lite for the network, plus an Apple Airport with its transmitter dialled down to 25%. The ARM machine is much slower than my ZFS NAS, but it is much lighter on power - at that point, the HDDs are the significant draw, so I only run 2 spinners that are non-redundant and make sure they’re backed up to cold storage. I also power up my ZFS machine once a month or so and sync the data from it. Other than that, I keep the big x86 machines shut down until needed.
Several locations.
- bunch of metering smart plugs flashed with Tasmota, feeding into Home Assistant (power at the wall)
- UPS being polled by LibreNMS (difference in load Vs wall shows the UPS consumption)
- metering PDU polled by LibreNMS shows the server load
- PMBus PSUs in the servers report their own power consumption to the BMC
And after all this, I do… Nothing with this data.
Can you tell if the backplane supports SGPIO? If it and your RAID card do, and things are wired up correctly, you can tell your RAID card to Identify the drive and it’ll blink the LEDs. I just configured a Supermicro chassis to do this with an Adaptec controller.
Why not just use ThinOS? IIRC it supports the NX protocol that NoMachine uses. If not, then you have Wyse’s respin of Ubuntu called ThinLinux, which you should be able to install.
You can download the OS image and flash utility from Dell’s website.
Yes, this should work fine. SAS does not care what path the signal takes - it doesn’t differentiate between internal and external. You can run internal over external cables without issue. I’ve done similar by turning my old NAS chassis into a DAS, and connecting it to her internal ports of the HBA. And you can connect SAS or SATA drives to the DAS (system 1).
I have Adaptec controllers and I don’t recommend them, my ASR-71605 does not like heavy disk IO to multiple drives. It can’t handle full bandwidth to multiple drives at once and they start lagging or piling up the IO. And my ASR-78165 cannot deal with SAS devices that have multiple LUNs, like tape drives. The whole card locks up.
Definitely go for LSI-based cards. They are the industry standard.
DIY - No Regrets.
I built my NAS out of spare parts originally and then it evolved into needing dedicated purchases. I like having full control of the OS and everything on it - it helps me understand what daemons are doing what. It does a lot more than file sharing.
The likes of QNAP and Synology may make a more polished product with an easy UI, as well as offering support, but as far as I care, I am support, so I like to fix problems myself.
If you’re ping-ponging between the two options, from your post it reads like cost is the biggest problem you face. But as you say, storage is a critical part of the infrastructure and sometimes you do have to spend money on it if you want it to be reliable. I just upgraded my main NAS with a larger chassis and motherboard (from an ITX) so I can expand it further. It cost me a sizeable amount of money that might have bought me a low-end ready-made, but this is far more flexible.
- Domain auth (1 place to set passwords and SSH keys), no root SSH
- SSH by key only
- Passworded sudo (last line of defence)
- Only open firewall hole is OpenVPN with security dialled up high
- VLANs - laptops segregated from servers
- Strict firewall rules between VLANs
- TLS on everything
- Daily update check alerts (no automatic updates, but persists until I deal with them)
- Separate isolated syslog server for audit trails
- Cold backups
Motherboard, CPU and RAM - no problem at all (more accurately, problems are easy to spot with diagnostics and they shouldn’t wear out).
Chassis - a bit of a wild card. The backplane in one of my systems is faulty.
PSUs - ideally new.
HDDs - almost all of mine are secondhand. Enterprise- or NAS-grade drives should have many years of life left. Ideally buy new to benefit from warranty but my experience has been great.
SSDs - nope. Buy new. I bought some secondhand Samsung SSDs and they developed problems, both threw IO errors after a few weeks. SSDs are cheap enough not to bother with secondhand.
Everything else I bought used, including the rack. In fact, the only things I bought new in my entire homelab are my router and WiFi AP.
Heavy computation rack is in an unheated conservatory with a window cracked open. Keeps the HDD temperatures around 30 degrees. Temperature monitoring from my PDU shows a 3’C rise from the inlet to the exhaust side of the rack. This stuff is mostly powered off when not in use. In summer, it can get to 35’C in that room so I shut everything down at that point.
24/7 rack is in my lounge and vents the heat into the room (helps a little bit with heating costs). Top of the rack is about 37’C but I’ve seen it around 45’C with all my hypervisors doing stuff. Nothing complains. As long as the intake air is within the manufacturer’s stated range, it’s fine.
Might want to consider redirecting the heat into the house rather than venting it outside.