I tried to upload an image, but got the error IO error: Os { code: 28, kind: StorageFull, message: "No space left on device" }
I’m assuming it’s a different storage area than the main database, otherwise this will fail too…
Turns out there is a bug with pictrs which doesn’t clean up temp files. So we had ~88GB of temp data!
And since we moved to a dedicated pictrs server (so the main Lemmy procesees & DB could benefit) I didnt add that server to our monitoring 🫢
It’s fixed now and I’ll be adding disk monitoring once I get a moment away from $job