Using a desktop web browser…
It doesn’t seem like a timeout, it is always fast to respond with the error and also fast on successful loading pages. It seems to me there is some kind of resource/parameter starvation in nginx or the nginx bridges to the NodeJS app…
I’m also seeing problems with static content loading, images and css files not loading and getting a mangled half-generated page or missing icons for upvote/downvote that I have to refresh the browser with. This happens at least 1 in 8 refreshes of the page.
In the most recent hour, it has gotten better. I’d say 500 errors from nginx are only coming up more like 1 in 25 web page refreshes.
I spent the past 20 minutes reading over on Beehaw.org instance, and the same problem is happening there. Pages always load or fail fast, but nginx 500 errors come in batches - where for 20 seconds I instantly get broken page/error page. And then the site resumes normal operation.
I’ve seen similar behavior (fail fast,) when there are outbound networking issues from any of the service containers. In one instance, lemmy backend was not able to make any outbound connections to any external services. It wild fail in milliseconds and show a timeout error in the logs.
One hypothesis here could be there are sporadic network issues on the lemmy.ml servers. 🤷♂️
Same for me.
Lemmy.ml is just now throwing me more nginx 500 errors.
Reports on Beehaw have been posted by end-users: https://beehaw.org/post/574558
Have all of the same issues from time to time. Also getting errors when using 3rd party apps as well which I presume are also HTTP 500 errors due to the fast response
Still going on today, 24 hours after creating this posting.
I’ve spent over 60 hours in the past 11 or 12 days on this problem, and it’s been brushed under the rug. There are MAJOR problems being logged on that server, and the server logs are not being shared.
ok, the past 5 minutes it was nearly constant. Mangled web pages half-loaded, images missing, nginx 500 errors. It is very fast to load each time, but failing to generate the content. Like only 2 out of 15 web page refreshes worked in that 5 minutes.
