Danbooru

Server throttling

Posted under General

I've added a throttling mechanism such that if the server load goes above 3.00 it'll try to fail gracefully instead of timing out. So you may end up seeing this page: http://danbooru.donmai.us/503. The URL won't change and this won't trigger on HTTP posts, so you should be able to just click refresh after a few minutes. This is still an experimental change and I will probably tweak the numbers.

Updated by jxh2154

So the server going over 3.0 must happen constantly, because i was on in the wee hours of the morning and got the server is overloaded message and to go to the pub etc. Got the same message over the last hour or so and its only a bit after noon here. Hopefully you do tweak the numbers a bit, as browseing the various posts and doing whatever for like 3-5 minutes at a time is kinda irritating. lol If i were bill gates i would just give you a monster server so we would not have to throttle anything...errr.. heh ;)

So I just checked Danbooru's stats at Alexa and there's a huge spike from last June 12th onwards. Perhaps this is the cause of the slowness lately but why such a huge spike all of a sudden? Don't tell me it's because of Black Rock Shooter?

Ive seen it several times but the server never really felt under load. A refresh would instantly bring the proper page back.

Also, I saw the page a moment ago and I find it hard to believe the server could have been under much load at the time, especially in comparison with at night.

Yeah I think the refreshing got worse today.

Really I don't think that under load pages are a very good idea unless they happen very infrequently. In the best case all you get is a bunch of people mashing the refresh hotkey.

Not sure if it is related, but I've posted this on IRC and I'll re-iterate it again here since no one on IRC knows what's up with it.

Earlier today, I was trying to upload an image, post #267716 to be precise, and as I finish the upload, it gave me some cryptic error message saying something went wrong and I should call my mom...

In a new tab, I went to /posts/index to check for it, and it wasn't there. I figured, no problem, I'll just go back and hit F5. A brief bit later, I got sent to the post, saying there is a duplicate post -- it was my previous post.

I have since gone back to check on /posts/index multiple times, and it is not listed. I don't know if it is listed in moderation queue, as well. It does seem to show up for my profile page, though.

Cam on IRC suggested that it may show up after maintenance... well, that came and went, and the index still doesn't list it. I'm suspecting some bug ate part of the post for whatever reason, and this seems to be the closest recent change that is related.

Can someone please look into this issue and provide feedback on this matter?

Thanks!

r6 said:
Really I don't think that under load pages are a very good idea unless they happen very infrequently. In the best case all you get is a bunch of people mashing the refresh hotkey.

Which isn't much different from them mashing the refresh button when the site is being slow. The 503 page should automatically refresh after two minutes in any case. I'd change it to one minute but I doubt that would stop the refresh-mashers.

Privileged members won't see the 503 page. Part of the rationale was to make the site more responsive for them.

Frankly, this 503 page compels me to mash refresh while the slow performance didn't. While the server is slowly chugging along, I know that my page/file is coming. If it suddenly stops with a 503 error, I know that nothing is coming. Now when this starts happening with almost EVERY page I go through, I'm probably the least patient.

I had to put up with this for the past month or so with Pooshlmer's Touhou wiki, until it moved over to Wikia, and I was just getting used to being rid of it, when it starts happening here. Can't win for losing, I guess. Sorry for hammering the server. I'll try not to contribute to the problem, but this is giving me troubles in my quest to tag all the music_video s in the flash tag.

I've been seeing it pretty often, but I'm probably being an ass and browsing too much. I don't mind, though; it gives me a chance to tear myself away from looking at pictures and do something a little more worthwhile for a minute or two. C:

1 2