Topic on Forum:Wiki Updates

Recent downtime

10
Southparkfan (talkcontribs)

Hello, as most of you might have experienced, All The Tropes (well Orain) has barely been accessible for over the past few days. Our loadbalancer (prod10) which spreads the traffic among the three MediaWiki servers we have, stopped serving users who do not use IPv6. The cause is unknown, but we suspect a DDoS (around the time prod10 started to have issues, an inbound DDoS (or something similiar) hit prod10). We suspect that DigitalOcean after that cut off prod10, which resulted into downtime. FastLizard4 (one of the sysadmins) and me discussed how to get Orain back up, and after a while, All The Tropes was brought back online at ~07:00 AM (UTC+0) today, and all Orain wikis were confirmed online at ~10:45 AM (UTC+0) today.

Some of the current traffic is still routed through prod10, I don't know why, but since no one told me they still are having issues with accessing Orain, I won't really look into that. I'll keep you updated as necessary. Sorry for the inconvenience, and thank you for the patience. A report is available on Meta.

On behalf of the Orain Staff,

Chemtrail Morgellons (talkcontribs)

It's still having a lot of problems. Orain is up about 4 hours a day on IPv6 and on IPv4 maybe 10 minutes a day. It started Tuesday and still going.

Arcane (talkcontribs)

We're well aware. Orain Staff is currently discussing more drastic measures to remedy the issues, since we recently learned they are more serious than the initial report suggested.

Southparkfan (talkcontribs)

Orain unexpectedly still had nullrouting (by DigitalOcean) issues, because we are still under fire of DDoS attacks. Since today the situation has more or less stabilized very much compared to the last three days, and with some very hard work we've fixed most of the issues affecting the public.

I don't expect any major interruptions again, but I can't guarantee that. We will keep you updated as necessary.

I am sorry for what happened these last few days, but we try to do what we can.

Tennessean (talkcontribs)

So, has anyone tried to see where the DDoS attacks are coming from? I hope that it's just something being done at random, as opposed to a specific attack on Orain or any of the wikis that it hosts.

Arcane (talkcontribs)

That matter is still under investigation, though it seems to have been an attack on Orain in general.

Chemtrail Morgellons (talkcontribs)

Most sites just get Cloudflare for it. People say it's cheap. There's different https://www.cloudflare.com/plans but most people sound like they are either getting the $20/month plan or even the free plan. Cloudflare says the free plan has "Basic DDoS protection". I personally suspect people just get the free plan. In fact, I've seen cheap, rarely visited websites with it and I'm certain they're on the free plan because hosting for those sites is really cheap with so little traffic. The only thing with Cloudflare is there's different control panel settings and sometimes people leave it on setting so all visitors from Tor and proxies have to enter a CAPTCHA before viewing each page.

Southparkfan (talkcontribs)

CloudFlare is an option, but moving over to CloudFlare comes also with a price (wildcard proxy), *many* infrastructure changes, and no one is sure if the hard work is worth the 'protection'.

Chemtrail Morgellons (talkcontribs)

Orain has always been down an average of 8 hours a day often on. Two weeks ago this went to around 20 hours a day. Cloudflare has a caching system that could keep Orain up all the time.

Arcane (talkcontribs)

We just moved to Cloudflare very recently to prevent this in the future, as we came to the same conclusion.