Welcome to today’s daily kōrero!
Anyone can make the thread, first in first served. If you are here on a day and there’s no daily thread, feel free to create it!
Anyway, it’s just a chance to talk about your day, what you have planned, what you have done, etc.
So, how’s it going?
I didn’t make the dashboards, but the data is available through an API call to each instance. I guess they have some polling set up and then feed it into the dashboard?
I’ve been testing some stuff with pictrs, and have been updating pictrs from 0.4 to 0.5 in my non-prod instance. There’s no progress meter and so far it has taken 2 days 😱.
It’s a lot less grunty than the production server but still seems excessive!
Interesting. I have some New Relic stuff setup with my cluster but most of that is just resource usage stuff. I ran out of RAM a while back so I’ve had to be a bit more restrictive about how many connections Lemmy can have to postgres db.
There’s no progress meter and so far it has taken 2 days 😱.
Uh oh. I considered updating to 0.5 as part of my 0.18.3-ish (I was running a custom fork I made with some image caching stuff that has since been merged in to real lemmy) -> 0.19.3 upgrade but I’m glad I didn’t.
Thanks for the heads up. Are you migrating to postgres for pictrs too, or sticking with sled?
I ran out of RAM a while back so I’ve had to be a bit more restrictive about how many connections Lemmy can have to postgres db.
I just have a cronjob to restart the backend lemmy container every night 😆
Thanks for the heads up. Are you migrating to postgres for pictrs too, or sticking with sled?
My plan is to go to postgres but this migration is just for sled. I was doing it for another reason, to test out a cache cleaning setup. Currently the pictrs image cache is 250-300gb because it’s never deleted anything (because lemmy doesn’t do that).
Lemmy.world said it took them 4 hours, and they have a grunty machine. Not sure what their cache looked like, though. I think they were also moving to postgres.
cronjob to restart the backend lemmy container
Fair enough, that’d work. I run my database in a different pod to lemmy (I run this all in kubernetes), and I cannot restart that pod without causing an outage for a bunch of other things like my personal website. I ended up just needing to tune my config to have a maximum RAM usage and then configuring k8s to request that much RAM for the DB pod, so it always has the resources it needs.
pictrs image cache is 250-300gb
oof :(
That’s what my custom lemmy patch was, it turned off pictrs caching. That’s now in lemmy as a config flag (currently a boolean but in 0.20 it will be on/off/proxy where the proxy option goes via your pictrs but does not cache). I then went back through mine and did a bunch of SQL to figure out which pictrs images I could safely delete and got my cache down to 3GB.
