Lemmy.world has been down between 02:00 UTC and 05:45 UTC. This was caused by the database spiking to 100% cpu (all 32 cores/64 threads!) due to inefficient queries been fired to the db very often.

I’ve collected the logs and we’ll be checking how to prevent this. (And what caused this)

96 points

lemmy.world is a test environment for Lemmy developers… 😅 Jokes aside, issue is an issue.

permalink
report
reply
39 points

Well, as pretty much the biggest instance, it provides the best data for load-testing. 🫣

permalink
report
parent
reply
37 points

All Lemmy instances are test environments right now. It’s just that lemmy.world is being tested the hardest.

permalink
report
parent
reply
21 points

It’s a good reminder that other large instances (lemm.ee, sh.itjust.works, reddthat.com, sopuli.xyz) usually offer higher uptime while showing the same content as LW

permalink
report
parent
reply
13 points

My issue with the smaller instances is that I enjoy browsing Local, and this is typically not as active or interesting on there. Is there a way to browse lemmy.world’s Local from another instance?

permalink
report
parent
reply
0 points

Local makes sense for a country or field focused instance (queer rights, programming, even porn or whatever), not so much for generalist instances, unfortunately.

The way you use local only makes sense now as most of the communities are on LW. Once a few of them start to branch out (such as lemdro.id or startrek.website), if you want to see that content, you will either subscribe or use all

permalink
report
parent
reply
1 point

Yeah. It doesn’t seem like a serious instance because of that

permalink
report
parent
reply
61 points

Every Lemmy update:

“We fixed some performance issues by optimising some queries.”

Also: “To balance it out, we added some new even more inefficient queries.”

permalink
report
reply
38 points
permalink
report
parent
reply
-27 points

Next time just god damn upload it directly. Thx

permalink
report
parent
reply
6 points

What do you mean?

permalink
report
parent
reply
9 points

How else are they supposed to patch things out? can’t leave it up to fate, gotta make your own destiny!

permalink
report
parent
reply
2 points

Huh, isn’t this instance only?

I just think tried from Connect for Lemmy in this comment and I couldn’t.

permalink
report
parent
reply
22 points

Thank you for the update!

permalink
report
reply
22 points

(And what caused this)

Prediction: bad database programming. ;)

permalink
report
reply
15 points

Are we extracting enough value out of our volunteer developers and DBAs?!

permalink
report
parent
reply
3 points

The beatings will continue until morale improves!

permalink
report
parent
reply
7 points

Or clever denial of service attacks

permalink
report
parent
reply
18 points
*

Thanks for keeping us updated. FYI i noticed an issue there was an error message saying to check (the matix) and (somewhere else, Lemmy community support?). Both of them pointed to the same URL, but im sure they were meant to point to different places.

Edit. Happed again and I took notes. Both point to lemmy.ml community support

permalink
report
reply

Lemmy.World Announcements

!lemmyworld@lemmy.world

Create post

This Community is intended for posts about the Lemmy.world server by the admins.

Follow us for server news 🐘

Outages 🔥

https://status.lemmy.world

For support with issues at Lemmy.world, go to the Lemmy.world Support community.

Support e-mail

Any support requests are best sent to info@lemmy.world e-mail.

Report contact

Donations 💗

If you would like to make a donation to support the cost of running this platform, please do so at the following donation URLs.

If you can, please use / switch to Ko-Fi, it has the lowest fees for us

Join the team

Community stats

  • 409

    Monthly active users

  • 805

    Posts

  • 37K

    Comments