Oddly this centralization allows a complete deferral of blame without you even doing anything: if you’re down, that’s bad. But if you’re down, Spotify is down, social media is down… then “the internet is broken” and you don’t look so bad.
It also reduces your incentive to change, if “the internet is down” people will put down their device and do something else. Even if your web site is up they’ll assume it isn’t.
I’m not saying this is a good thing but I’m simply being realistic about why we ended up where we are.
As a user I do care, because I waste so much time on Cloudflare's "prove you are human" blocking-page (why do I have to prove it over and over again?), and frequently run on websites blocking me entirely based on some bad IP-blacklist used along with Cloudflare.
If you have a site with valuable content the LLM crawlers hound you to no end. CF is basically a protection racket at this point for many sites. It doesnt even stop the more determined ones but it keeps some away.
Oh, they're still botnets. We just look the other way because they're useful.
And they're pretty tame as far as computer fraud goes - if my device gets compromised I'd much rather deal with it being used for fake YouTube views than ransomware or a banking trojan.
You can make a little bit of cash on the side letting companies use your bandwidth a bit for proxying. You won’t even notice. $50/month. Times are tough!
Of course the risk here being whatever nefarious or illegal shit is flowing through your pipes, which you consented to and even received consideration for.
Unfortunately the problem isn't just "the internet sucks" it's "the internet sucks, and everyone uses it" - meaning people are not doing stuff offline, and a lot of our lives require us to be online.
Absolutely. They have dramatically worsened the world, with little to no net positive impact. Nearly every (if not all) positive impacts have an associated negative that that dwarfs it.
LLMs aren't going anywhere, but the world would be a better place if they hadn't been developed. Even if they had more positive impacts, those would not outweigh the massive environmental degradation they are causing or the massive disincentive they created against researching other, more useful forms of AI.
IMO LLMs have been a net negative on society, including my life. But I'm merely pointing out the stark contrast on this website, and that fact that we can choose to live differently.
I am not anti-AI, nor unhappy about how any current LLM works. I'm unhappy about how AI is used and abused to collective detriment. LLM scraper spam leading to increased centralization and wider impacting failures is just one example.
Your position is similar to saying that medical drugs have been a net negative on society, because some drugs have been used and abused to collective detriment (and other negative effects, such as doctors prescribing pills instead of suggesting lifestyle changes). Does it mean that we would be better off without any medical drugs?
My position is that the negatives outweigh the positives, and I don't appreciate your straw man response. It's clear your question is not genuine and you're here to be contrarian.
A solid secondary option is making LLM scraping for training opt-in, and/or compensating sites that were/are scraped for training data. Hell, maybe then you could not knock websites over incentivizing them to use Cloudflare in the first place.
But that means LLM researchers have to respect other people's IP which hasn't been high on their todo lists as yet.
bUt ThAT dOeSn'T sCaLe - not my fuckin problem chief. If you as an LLM developer are finding your IP banned or you as a web user are sick of doing "prove you're human" challenges, it isn't the website's fault. They're trying to control costs being arbitrarily put onto them by a disinterested 3rd party who feels entitled to their content, which it costs them money to deliver. Blame the asshole scraping sites left and right.
Edit: and you wouldn't even need to go THAT far. I scrape a whole bunch of sites for some tools I built and a homemade news aggregator. My IP has never been flagged because I keep the number of requests down wherever possible, and rate-limit them so it's more in line with human like browsing. Like so much of this could be solved with basic fucking courtesy.
Not to speak for the other poster, but... That's not a good-faith question.
Most of the problems on the internet in 2025 aren't because of one particular technology. They're because the modern web was based on gentleman's agreements and handshakes, and since those things have now gotten in the way of exponential profit increases on behalf of a few Stanford dropouts, they're being ignored writ large.
CF being down wouldn't be nearly as big of a deal if their service wasn't one of the main ways to protect against LLM crawlers that blatantly ignore robots.txt and other long-established means to control automated extraction of web content. But, well, it is one of the main ways.
Would it be one of the main ways to protect against LLM web scraping if we investigated one of the LLM startups for what is arguably a violation of the Computer Fraud and Abuse Act, arrested their C-suite, and sent each member to a medium-security federal prison (I don't know, maybe Leavenworth?) for multiple years after a fair trial?
I'm Sure there will be an investigation... By the SEC when the bubble pops and takes the S&P with it. No prison though, probably jobs at the next ponzi scheme
I just realized, why don't they have some "definitely human" third party cookie that caches your humanness for 24h or so? I'm sure there's a reason, I've heard third party cookies were less respected now, but can someone chime in on why this doesn't work and save a ton of compute?
Yes, there are several, and the good one (linked below) lets you use the "humanness" token across different websites without them being able to use it as a tracking signal / supercookie. It's very clever.
That's a problem caused by bots and spammers and DDoSers, that Cloudflare is trying to alleviate.
And you generally don't have to prove it over and over again unless there's a high-risk signal associated with you, like you're using a VPN or have cookies disabled, etc. Which are great for protecting your privacy, but then obviously privacy means you do have to keep demonstrating you're not a bot.
You might say the problem CloudFlare is causing is lesser than the ones it's solving, but you can't say they're not causing a new, separate problem.
That they're trying counts for brownie points, it's not an excuse to be satisfied with something that still bothers a lot of people. Do better, CloudFlare.
"We have decided to endlessly punish you for using what few tools you have to avoid being exploited online, because it makes our multi-billion dollar business easier. Sucks to be you."
> It has nothing to do with being a multi-billion dollar corp.
Cloudflare is the multi-billion dollar corporation. It has everything to do with that, because they are the primary cause, and their resources and position make them by far the best equipped to solve it.
> Criticizing when there's no other solution isn't very useful, is it?
Of course it is. Without criticism, the growing problem goes unacknowledged and allowed to persist. It should instead be continually called out until it is prioritized, and some of those billions should be spent on researching a solution. (Similarly, a company found to be dumping waste into a river should be held responsible for cleaning up the mess they created. Even if that turns out to be expensive or difficult.)
Expecting a single affected person to solve it for the big corp that caused it is unrealistic. And blaming the victims because they use VPNs or disable cookies is... unhelpful.
CloudFlare is protecting sites from DDoS attacks and out-of-control bots. They're not the ones causing them. If CloudFlare wasn't asking you to prove you're human, many times the site would be down entirely because it couldn't keep up. Or the site would simply shut down because it couldn't afford it.
And this isn't a question of spending some fraction of billions on researching a solution. There fundamentally isn't one, if you understand how the internet works. This is a problem a lot of people would like to solve better, believe me.
So, yes, criticizing Cloudflare here is as useful as criticizing it for not having faster-than-light communication. There's nothing else it can do. It's not "blaming the victims".
I'm going to assume you simply don't have the technical understanding of how the internet works. Because the position you're taking is simply absurd and nonsensical, and there's no way you would write what you're writing otherwise.
Privacy through uniformity, operational security by routine, herd immunity for privacy, traffic normalization, "anonymity set expansion", "nothing to hide" paradox, etc.
I.e., if you use Tor for "normie sites", then the fact that someone can be seen using Tor is no longer a reliable proxy for detecting them trying to see/do something confidential and it becomes harder to identify & target journalists, etc. just because they're using Tor.
Tor Browser has ~1M daily users. Tons of people use it for hitting sites that may be blocked in their country or they want to have some privacy like view pregnancy or health related articles and etc.
In addition to the reasons in sibling comment, this also acts as a filter for low-quality ad-based sites; same reason I close just about any website that gives me a popup about a ToS agreement.
This is essentially the entire IT excuse for going to anything cloud. I see IT engineers all the time justifying that the downtime stops being their problem and they stop being to blame for it. There's zero personal responsibility in trying to preserve service, because it isn't "their problem" anymore. Anyone who thinks the cloud makes service more reliable is absolutely kidding themselves, because everyone who made the decision to go that way already knows it isn't true, it just won't be their problem to fix it.
If anyone in the industry actually cared about reliability and took personal stake in their system being up, everyone would be back on-prem.
Reliability is not even how the cloud got sold to the C Suite. Good God, when my last company started putting things on Azure back in 2015 stuff would break weekly, usually on Monday mornings.
No, the value proposition was always about saving money, turning CapEx into OpEx. Direct quote from my former CEO maybe 9 years ago: We are getting out of the business of buying servers.
Cloud engineering involves architecting for unexpected events: retry patterns, availability zones, multi-region fail over, that sort of thing.
Now - does it all add up to cost savings? I could not tell you. I have seen some case studies, but I also have been around long enough to take those with a big grain of salt.
It's amazing how there's so many cybersecurity incidents now. Bypassing IT will always backfire spectacularly, IT is the people that stop you from dumbing.
The opposite was/is true. If your cloud box can only be used by two people and IT don’t even know about it then IT can never be persuaded to provide the keys to the rest of the company as they were predisposed to doing.
I saw this stuff too many times, and it is precisely why the cloud exploded in use in about 2010.
One notable example was signing keys for builds for distribution actually. And IT had a habit of handing them out to absolutely everyone. Being able to audit who did the signing was done in spite of IT who could, of course, never be persuaded of the merit of any process they don’t own.
I won't discount your IT can be bad, but also if you're keeping something as core to your security as signing keys somewhere your IT can't audit, you are just as bad. And your IT won't be the ones fired when your keys leak.
That might have been true for some kind of organization, but definitely not for every kind. On the other side, there were start-ups that wanted the elasticity and no commitments. But both sides at least partially liked the "it's not on me anymore" feature.
IMHO it adds, but only if you are big enough. Netflix level. At that level, you go and dine with Bezos and negotiate a massive discount. For anyone else, I’d genuinely love to see the numbers that prove otherwise.
> There's zero personal responsibility
Unfortunately, this seems to be the unspoken mantra of modern IT management. Nobody wants to be directly accountable for anything, yet everyone wants to have their fingerprints on everything. A paradox of collaboration without ownership.
Cloud providers have formalized these deals actually. If you promise to spend X amount over Y period, you get Z discounts.
And this is not reserved instances, this is an org level pricing deal. Some have been calling it anti-competitive and saying the regulators need to look at the practice.
> IMHO it adds, but only if you are big enough. Netflix level. At that level, you go and dine with Bezos and negotiate a massive discount. For anyone else, I’d genuinely love to see the numbers that prove otherwise.
It adds if you're smart about using resources efficiently, at any level. And engineer the system to spin up / spin down as customers dictate.
For situations where resources are allocated but are only being utilized a low percentage (even < 50% in some cases), it is not cost effective. All that compute / RAM / disk / network etc. is just sitting there wasted.
I mean in the end it's about making a trade off that makes sense for your business.
If the business can live with a couple of hours downtime per year when "cloud" is down, and they think they can ship faster / have less crew / (insert perceived benefit), then I don't know why that is a problem.
It is a trade-off between convenience and freedom. Netflix vs buying your movies. Spotify vs mp3s. Most tech products have alternatives. But you need to be flexible and adjust your expectations. Most people are not willing to do that
The issue is that real life is not adaptable. Resources and capital are slow.
That's the whole issue with monopolies for example, innit? We envision "ideal free market dynamics" yet in practice everybody just centralizes for efficiency gains.
Right, and my point is that "ideal free market dynamics" conveniently always ignore this failure state that seems to always emerge as a logical consequence of its tenets.
I don't have a better solution, but it's a clear problem. Also, for some reason, more and more people (not you) will praise and attack anyone who doesn't defend state A (ideal equilibrium). Leaving no room to point out state B as a logical consequence of A which requires intervention.
The definition of a monopoly basically resolves to "those companies that don't get pressured to meaningfully compete on price or quality", it's a tautology. If a firm has to compete, it doesn't remain a monopoly. What's the point you're making here?
There absolutely are options but we aren't using them because nobody cares enough about these downsides. bsky is up, with Mastodon you even have choice between tons of servers and setting up your own. Yet, nobody cares enough about the occasional outage to switch. It's such a minor inconvenience that it won't move the needle one bit. If people actually cared, businesses would lose customers and correct the issue.
More like it's time for the pendulum to swing back...
We had very decentralized "internet" with BBSes, AOL, Prodigy, etc.
Then we centralized on AOL (ask anyone over 40 if they remember "AOL Keyword: ACME" plastered all over roadside billboards).
Then we revolted and decentralized across MySpace, Digg, Facebook, Reddit, etc.
Then we centralized on Facebook.
We are in the midst of a second decentralization...
...from an information consumer's perspective. From an internet infrastructure perspective, the trend has been consistently toward more decentralization. Initially, even after everyone moved away from AOL as their sole information source online, they were still accessing all the other sites over their AOL dial-up connection. Eventually, competitors arrived and, since AOL no longer had a monopoly on content, they lost their grip on the infrastructure monopoly.
Later, moving up the stack, the re-centralization around Facebook (and Google) allowed those sources to centralize power in identity management. Today, though, people increasingly only authenticate to Facebook or Google in order to authenticate to some 3rd party site. Eventually, competitors for auth will arrive (or already have ahem passkeys coughcough) and, as no one goes to Facebook anymore anyway, they'll lose grip on identity management.
It's an ebb and flow, but the fundamental capability for decentralization has existed in the technology behind the internet from the beginning. Adoption and acclimatization, however, is a much slower process.
These centralized services do and did solve problems. I'm old enough to remember renting a quarter rack, racking my own server and other infrastructure, and managing all that. That option hasn't gone away, but there are layers of abstraction at work that many people probably haven't and don't want to be exposed to.
Aaand even if we ignore the "benefit" of Cloudflare and AWS outages being blamed on them, rather than you, what does uptime look like for artisanaly hosted services on a quarter rack vs your average services on AWS and Cloudflare?
> Businesses and peoples’ livelihoods are online nowadays
What happened to having a business continuity plan? E.g. when your IT system is down, writing down incoming orders manually and filling them into the system when it's restored?
I have a creeping suspicion that people don't care about that, in which case they can't really expect more than to occasionally be forced into some downtime by factors outside of their control.
Either it's important enough to have contingencies in place, or it's not. Downtime will happen either way, no matter how brilliant the engineers working at these large orgs are. It's just that with so much centralization (probably too much) the blast range of any one outage will be really large.
My wife and I own a small theatre. We can process orders in-store just fine. Our customers can even avoid online processing fees if they purchase in-store. And if our POS system went down, we could absolutely fall back to pencil and paper.
Doesn't change the fact that 99% of our ticket sales happen online. People will even come in to the theatre to check us out (we're magicians and it's a small magic shop + magic-themed theatre - so people are curious and we get a lot of foot traffic) but, despite being in the store, despite being able to buy tickets right then and there and despite the fact that it would cost less to do so ... they invariably take a flyer and scan the QR code and buy online.
We might be kind of niche, since events usually sell to groups of people and it's rare that someone decides to attend an event by themselves right there on the spot. So that undoubtedly explains why people behave like this - they're texting friends and trying to see who is interested in going. But I'm still bringing us up as an example to illustrate just how "online" people are these days. Being online allows you to take a step back, read the reviews, price shop, order later and have things delivered to your house once you've decided to commit to purchasing. That's just normal these days for so many businesses and their customers.
I’m not so sure about that. The pre-internet age had a lot of forced “mental health breaks”. Phone lines went down. Mail was delayed. Trains stalled. Businesses and livelihoods continued to thrive.
The idea that we absolutely need 24/7 productivity is a new one and I’m not that convinced by it. Obviously there are some scenarios that need constant connectivity but those are more about safety (we don’t want the traffic lights to stop working everywhere) than profit.
Just want to correct the record here, as someone who worked at a local CLEC where we took availability quite seriously before the age of the self-defeatist software engineer.
Phone lines absolutely did not go down. Physical POTS lines (Yes, even the cheap residential ones) were required to have around 5 9s of availability, or approximately 5 minutes per year. And that's for a physical medium affected by weather, natural disasters, accidents, and physical maintenance. If we or the LEC did not meet those targets contracts would be breached and worst case the government would get involved.
Okay, as someone who also worked in that era I’ll be pedantic: internal phone systems went down. I experienced it multiple times so I certainly know it happened.
FWIW nothing I said was “self defeatist”, I made it clear I don’t think it’s a good thing. It’s just a simple financial reality that the additional redundancy isn’t worth the extra cost in a lot of situations.
Most businesses are totally fine if they have a few hours of downtime. More uptime is better, but treating an outage like a disaster or an e-commerce site like a power plant is more about software engineer egos than business or customer needs.
If AWS is down, most businesses on AWS are also down, and it’s mostly fine for those businesses.
I’ve worked in cloud consulting for a little over five years. I can say 95% of the time when I discuss the cost and complexity tradeoffs of their websites being down vs going multi region or god forbid “multi cloud”, they shrug and say, it will be fine if they are down for a couple of hours.
This was the same when I was doing consulting inside (ie large companies willing to pay the premium cost of AWS ProServe consultants) and outside working at 3rd party companies.
It's better to have diverse, imperfect infrastructure, than one form of infra that goes down with devastating results.
I'm being semi-flippant but people do need to cope with an internet that is less than 100% reliable. As the youth like to say, you need to touch grass
Being less flippant: an economy that is completely reliant on the internet is one vulnerable to cyberattacks, malware, catastrophic hardware loss
It also protects us from the malfeasance or incompetence of actors like Google (who are great stewards of internet infrastructure... until it's no longer in their interests)
Wealthy, investment-bloated software companies will be fine.
Smaller companies that provide real world services or goods to make a much more meagre living that rely on some of the services sold to them by said software companies will be impacted much more greatly.
Losing a day or two of sales to someone who relies on making sales every day can be a growing hardship.
This doesn’t just impact developers. It’s exactly this kind of myopic thinking that leads to scenarios like mass outages.
My friend, that is just not reality. And it's not just e-commerce t shirt slingers I am talking about here.
You have to realize when software companies tell the world they should rely on their works, they world will do so. And once that occurs, the responsibility is all on the software companies to meet the expectations they built in people!
It's mad that this industry works so hard to claim the trust of millions of people, then shirks it as soon as it's convenient.
> But if you’re down, Spotify is down, social media is down… then “the internet is broken” and you don’t look so bad.
In my direct experience, this isn't true if you're running something even vaguely mission-critical for your customers. Your customer's workers just know that they can't do their job for the day, and your customer's management just knows that the solution they shepherded through their organization is failing.
It's really quite funny, many of the ACTUALLY vital systems to running the world as we know it are running off of very different softwares. Cloudflare appears to have a much higher % of non vital systems running on it than say something like akamai.
If akamai went down i have a feeling you'd see a whole lot more real life chaos.
i also find the sentiment of "well we use a third party so blame them" completely baffeling.
if you run anything even remotely mission critical, not having a plan B which is executable and of which you are in control (and a plan C) will make you look completely incompetent.
There are very, very few events which some people who run mission critical systems accept as force majeur. Most of those are of the scale "national emergency" or worse.
>There are very, very few events which some people who run mission critical systems accept as force majeur. Most of those are of the scale "national emergency" or worse.
And why should anyone be surprised? It's been about 80 years since "The buck stops here."[0] had any real relevance. And more's the pity.
100% this. While in my professional capacity I'm all in for reliability and redundancy, as an individual, I quite like these situations when it's obvious that I won't be getting any work done and it's out of my control, so I can go run some errands to or read a book, or just finish early.
Which "user" are you referring to? Cloudflare users or end product users?
End product users have no power, they can complain to support and maybe get a free month of service, but the 0.1% of customers that do that aren't going to turn the tide and have anything change.
Engineering teams using these services also get "covered" by them - they can finger point and say "everyone else was down too."
Admittedly when I wrote that I was thinking about the recent AWS outage. Anecdotally, I asked friends and family about their experience and they assumed the internet was down. Almost everything at my work runs on Google cloud so we were still running but we observed a notable dip in traffic during the outage all the same.
> it is still bad
No doubt. But there’s a calculation to make, is it bad enough to spend the extra money on mitigations, to hire extra devops folks to manage it all… and in the majority of end user facing cases the answer is no, it isn’t.
Where I've worked and we've been in the cloud I've always promoted just running in one AZ, I run my own things in one Hetzner DC (hel1). I've done hybrid cloud as well and in that case we only have one AZ for the on-premise stuff anyways (plus offsite backup)
That one time when an AZ goes down and your infra successfully fails over to the other two isn't worth it for a lot of my scale companies, ops consultants seem to be chasing high cloud spend to justify their own high cost. I also factor in that I live in Sweden where most infrastructure outages are exceptionally rare.
Ofc it depends on what kind of company you are and what you're providing.
Eh? It's because they are offering a service too good to refuse.
The internet this day is fucking dangerous and murderous as hell. We need Cloudflare just to keep services up due to the deluge of AI data scrapers and other garbage.
More like "don't have choice". It's not like service provider gonna go to competition, because before you switch, it will be back.
Frankly it's a blessing, always being able to blame the cloud that management forced company to migrate to be "cheaper" (which half of the time turns out to be false anyway)
> It also reduces your incentive to change, if “the internet is down” people will put down their device and do something else. Even if your web site is up they’ll assume it isn’t.
I agree. When people talk about the enshittification of the internet, Cloudflare plays a significant role.
Oddly this centralization allows a complete deferral of blame without you even doing anything: if you’re down, that’s bad. But if you’re down, Spotify is down, social media is down… then “the internet is broken” and you don’t look so bad.
It also reduces your incentive to change, if “the internet is down” people will put down their device and do something else. Even if your web site is up they’ll assume it isn’t.
I’m not saying this is a good thing but I’m simply being realistic about why we ended up where we are.