cloudflare outage!!

[AnIzzi0t]

so, while messing around on the web, downloading movies and music and such, i went to my favorite downloader directory: fmhy.net (free media heck yeah.net) and its down! no biggie, the server goes down for maintenance a ton, its fine. but when i go to some of the tabs i have bookmarked that have downloaders on them, they're down too! apparently cloudflare is down, so that takes about... 1/3rd of my fave sites off the web for a while.

outage reports are going down, but theres not yet a sign that it'll be back online soon is there anyone here who knows any more about this topic?

source: techradar.net

edit: here's what the error screen looks like

[Melooon]

Cloudflare is a web delivery service, they have half the web traffic going through their servers; this ones big enough to be in mainstream news, Twitter and ChatGPT are currently entirely offline.

I personally think cloudflare reliability is so low that I would not accept it for this tiny forum, I dunno how big companies allow it to to used 

It'll all come back in a few hours though 

[IndigoGolem]

Is this related at all to the Amazon Web Services thing a month ago? It seems weird that suddenly the Web is breaking in more places than usual.

It's hard to know sometimes if the world is actually suddenly falling apart faster or if i'm just more aware of it than i was as a more carefree kid.

[Melooon]

if the world is actually suddenly falling apart faster

Maybe not the world (though things are not super rosy there atm), but the web definitely is having more large scale outages    What's happened over the last 10 years is that more websites that used to be independently hosted with their own teams of webdevs and their own servers are now all using the same super-large services like AWS and Cloudflare to make their websites work. They do that because its cheaper and easier to outsource the work and the infrastructure, but it also means that there are now 3-4 HUGE fail points on a ridiculously vast amount of the web.

Its an extremely unstable system and I really don't think its sustainable; if I was a large company I'd be panic buying my own server hardware now, and hiring people to maintain it because in a few years everyone is gonna be doing that.

As an individual there's not much you need to worry about, but expect cloud services to become less reliable, and follow general indie-web practices of archiving and having local backups of all your stuff if you wanna avoid the annoyance of a downed service! 

[Dan Q]

Is this related at all to the Amazon Web Services thing a month ago? It seems weird that suddenly the Web is breaking in more places than usual.

Probably a coincidence. Possibly state cyber-terrorism... but probably not.

I think it's story-time...

Story Time: The Day I (Thought I) Broke The Entire Internet

Back in 2002 I was teaching myself BIND, which up until relatively recently was the way to manage DNS. I'd been teaching myself DNS and BIND with the help of a more-experienced friend, but this one particular day was my first time "flying solo".

There's one thing you need to know about BIND: it's really unforgiving. You miss a period at the end of a line that was supposed to have one? You've just trashed your DNS. Fail to correctly increment your zone version? You've just trashed the cache and maybe half the Internet can't reach your site any more. You make a mistake with something fundamental like a nameserver or a glue record and you can break your domain for hours or days. Nowadays DNS software is a lot better at detecting errors and warning you before you screw things up (especially with friendly tools like PowerDNS, not to mention the lovely Web-based tools people offer nowadays), but back in 2002 it... wasn't so user-friendly yet.

So I was really careful. I checked everything thrice. Nonetheless, I was sweating at least a little when I saved my changes and reloaded BIND. I changed my domain configuration, hit reload, and tried my new subdomain.

Nothing.

Not a good start. So I tried my primary domain, which I hadn't intended to change.

Nothing.

Oh damn, that's not good. I tried another domain hosted by the same DNS server.

Nothing.

Well this is getting worse. So I fired up my XMPP app to try to reach out to my mentor... but it won't connect.

Is my home Internet connection down? No: I've got an SSH session open and it's still working fine. But I can't make new connections.

I'm sure that at this moment all the blood drained from my face. Had I somehow... broken the Internet?

A few more tests confirmed that connectivity was fine if I knew or had cached the IP address I wanted, but I couldn't do DNS lookups for most domains. Not just mine, but 50%+ of the Internet.

Oh my god... I was warned that I could break things if I did BIND carelessly. I didn't know I could break everything. At this point I was having flashbacks to a time a decade earlier when I built my first beige box and, while playing about, knocked out all the telephones in my dad's neighbourhood. How much trouble would I be in for breaking the whole damn Internet?

I probably didn't start breathing normally again until an hour later when I learned that I'd coincidentally made my changes right at the same time as a major cyber-attack took down 9 of the 13 master root DNS servers for the entire Internet. For most users, local (computer, router, ISP, etc.) DNS caching hid the scope of the attack, but I was bypassing all the caches while I was testing my own configuration, and that meant that I was putting myself right in the middle of the damage. It wasn't me at all... it was just a coincidence.

But for a heartstopping hour in 2002, I thought I might have broken the entire Internet.

[Dan Q]

but the web definitely is having more large scale outages    What's happened over the last 10 years is that more websites that used to be independently hosted with their own teams of webdevs and their own servers are now all using the same super-large services like AWS and Cloudflare to make their websites work. They do that because its cheaper and easier to outsource the work and the infrastructure, but it also means that there are now 3-4 HUGE fail points on a ridiculously vast amount of the web.

100% this.

Don't be part of the problem! If you can choose, don't host on AWS, Azure, or GCloud; don't cache with CloudFlare.

The Internet was built to be resilient because it's a network. Companies who choose to centralise their stack on the same four systems as everybody else are actively working against the Internet's resilience.

[starbreaker]

Ah, good old Clownflare. Nothing wrong with it that can't be fixed with Federal antitrust persecution, or by website operators accepting that downtime -- like shit -- sometimes just happens. Sure, a CDN like Clownflare can protect you from a low-level DDOS, but if a DDOS can take Clownflare down, then you're still screwed. And this probably wasn't a DDOS, this was somebody on the inside screwing up royally. There's no protection against that, either.

[Dan Q]

The thing that gets me is the entirely anticipatable/mitigatable knock-on effects.

Like: all those websites that use FontAwesome icons via the CDN? Their icons are down, because the FontAwesome CDN is hosted by CloudFlare.

And there's absolutely no reason for it. Just download the icons onto your own server! Then any time your site is up, your dependencies are up too.

Long ago there used to be reasons to shard out your fonts and JS. Edge caching. Cacheability of common assets between websites. Multi-domain threaded performance. Every single one of those arguments except edge caching is dead in the water nowadays (and the edge caching one is pretty pointless if you don't also have edge caching for your site itself as well (your content is the killer bottleneck for most sites!).

And using a CDN introduces huge risks. Supply chain attacks. Firewall blocks. Privacy-destroying data-mining by big CDN providers. And, of course, the risk that your site can go "half-down" (which, if you lose a key JS library, can mean "all the way down") even when your server itself is fine.

Most of the time, for most people, using a CDN is just manually introducing an extra single-point-of-failure to your stack, and a whole lot of other problems too. I don't understand why, in the 2020s, so many companies still do it!

Okay, rage over.

[varve]

Is this related at all to the Amazon Web Services thing a month ago? It seems weird that suddenly the Web is breaking in more places than usual.

It may also be related to all the layoffs places (like Amazon) have been doing lately, in favour of so-called "AI". If you don't have problem solvers keeping an eye on things, problems snowball really quickly...

...and "AI" doesn't actually know anything, so it can't troubleshoot, which puts more load on fewer people.

[Julikins]

As someone that also witnessed the issues regarding the AWS and how the same company is cutting out human workers in favor for artificial intelligence, it just seems like a bad omen of what is to come. Like one has pointed out, the fact that they would favor AI over human workers would bring more issues than solving them. AI can only do so much before human intervention is needed. I could say the same for FB/Meta and how they use AI instead of actual people when it comes to monitoring posts for community standard violations. Things might of changed since the last time I've been flagged for such but it seems like the AI nitpicks petty things and not work efficiently as a human worker would.
With an actual person examining said content (or the service issues in this regard), they would be more inclined to troubleshoot instead of putting on what essentially feels like a bandaid on a problem that would more than likely come up again in the future if not cared for then and there. I have little to no server experience as I tend to use them as a customer than a tech geek (sorry to folks that enjoy learning how servers work, I couldn't find a proper term for the crowd that doesn't feel insulting). So there's a good probability that I'm wrong somewhere.

I thought that the way Cloudflare had handled it was better than how AWS did as I recall seeing an ad about their web services and chuckled at them claiming how reliable it is. I know its just how advertising works nowadays, but I feel like it's ironic considering the recent outage at that time.

[Limette]

I work on my neocities site maybe once a bluemoon, rarely feeling the drive to tackle coding. Of course, on the day I decided I wanted to finally set up an art gallery, in the middle of me uploading images, Cloudflare went down. Neocities itself was fine, but I host nonessential site assets like my gallery artworks on filegarden. I'm a free user on neocities, so I don't want to take up my storage there by uploading those artworks to neocities directly.

Filegarden, as it turns out, relies on Cloudflare.

And so, my plans for that day got derailed. In the end I decided to play video games while I wait out the outage.

I did learn something: I need backup options for when filegarden or Cloudflare have issues again. So, that's something to look into soon...

[PurpleHello98]

I'm very glad Neocities and Dreamwidth don't run on CloudFlare! I totally hate CloudFlare anyhow. On my older computers/computers with non-standard browsers it sticks me in an infinite captcha verification loop so I can never actually do anything on CloudFlare-hosted Web sites. How evil is that? So CloudFlare can go bung itself up for all I care. (I just wish Neocities didn't use AWS1)