The backstreet San Francisco tech firm that triggered a worldwide internet freakout

In the early morning hours on Tuesday 8 June, thanks to an obscure, San Francisco-based cloud computing firm housed between an ancient scuba diving shop and an only slightly less-ancient US post office, pretty much the entire internet ground to a halt.

The alarm was sounded quickly. More than 20,000 users on Reddit reported problems. Another 2,000 or so said Amazon-related services were having hiccups. Soon came, of course, an official hashtag emerged, #InternetShutdown, as did the meta social media commentary from brands trying to ride the wave of general internet chaos, like candy bar maker KitKat, who told users, “Guess it’s time to Have A Break”.

Companies and governments ranging from PayPal and Amazon, Etsy and Pinterest, the New York Times and CNN, the official gov.uk websites – they all were down. All told, thousands of entities around the world were affected.

Eventually a culprit was discovered: Fastly. The cloud-computing firm helps power the back end of numerous high-profile websites by delivering content across servers around the world.

The company is based in the downtown San Francisco neighbourhood of South Beach, a windswept pocket just west of the GIants baseball stadium. The area is home to the usual discordant mix of post-industrial downtown staples: a crossfit gym, tech companies seemingly named with a random syllable generator, a few seedy apartment buildings that managed to survive gentrification.

According to Fastly, early on Tuesday it began investigating a problem with its POPs, or points of presence, the places where independent digital communications networks connect with each other.

“We identified a service configuration that triggered disruptions across our POPs globally and have disabled that configuration,” the company wrote on an outage dashboard on its website. “Our global network is coming back online.”

About two hours later, the problem was resolved.

But not before millions of dollars of global business came to a halt. According to a study from marketing agency connective3, news publishers alone lost about $300,000 in Google ad revenue, while major e-commerce nodes like PayPal and Amazon lost many multiples more.

All this disruption was from a relatively small cloud company, too, compared to giants like Amazon Web Services.

The whole incident served as a reminder of how interconnected the digital world is, and how disconnected that digital world is from the physical one. Owing to the small matter of a worldwide web shutdown, I hoped in my car and decided to see Fastly for myself.

But a voice on an intercom informed me there wasn’t even anyone physically at the Fastly offices – ”Uh yeah, no. There isn’t anybody to see.” Such is life in the Covid world, where the global economy can rise and fall thanks to the work of coders typing from home in their pajama pants.

At a nearby coffee shop, a friendly and potentially stoned barista named Kevin could not believe the outage originated from a company just down the street.

“Really? No way!” he said.

“It’s because too many people were watching porn,” said a nearby customer. “You need to put an ‘https’ before the website to make it work again,” he added – a little coder humour – before driving off in his Tesla, which was parked in the middle of the road, fully blocking traffic on a one-way alley. A homeless man with no shoes walked by soon after.

In fact, far from a crisis, the incident actually seemed to help Fastly, whose stock price was up more than 11 per cent after the attention. Perhaps the brief outage reminded people of just how vital, vulnerable, and essentially incomprehensible, the modern internet really is. You don’t know what you’ve got until its giving you an error message.

"Incidents like this underline the fragility of the internet and its dependence on a patchwork of fragmented technology. Ironically, this also underlines its inherent strength and how quickly it can recover," Ben Wood, chief analyst at CCS Insight, told Reuters. "The fact that an outage like this can grab headlines around the world shows how rare it is."

Heading off, I noticed one of Fastly’s neighbours was a company called “techsoup Global”. That’s as fitting a name as any for the morning we all had.

Just remember, always save your work. You never know when a kink in the flow of information will ricochet around the world while you’re sleeping.

Speaking to the scope of the outage, Fastly’s dashboard lists the following locations as problem zones: “Asia/Pacific (Auckland (AKL), Brisbane (BNE), Dubai (FJR), Hong Kong (HKG), Melbourne (MEL), Osaka (ITM), Perth (PER), Singapore (SIN), Sydney (SYD), Tokyo (HND), Tokyo (TYO), Wellington (WLG), Singapore (QPG), Tokyo (NRT)), South America (Buenos Aires (EZE), Bogota (BOG), Curitiba (CWB), Rio de Janeiro (GIG), Santiago (SCL), Sāo Paulo (CGH), Sāo Paulo (GRU), Lima (LIM)), North America (Ashburn (BWI), Ashburn (DCA), Ashburn (IAD), Ashburn (WDC), Atlanta (FTY), Atlanta (PDK), Boston (BOS), Chicago (CHI), Chicago (MDW), Chicago (ORD), Chicago (PWK), Columbus (CMH), Columbus (LCK), Dallas (DAL), Dallas (DFW), Denver (DEN), Houston (IAH), Jacksonville (JAX), Kansas City (MCI), Los Angeles (BUR), Los Angeles (LAX), Los Angeles (LGB), Miami (MIA), Minneapolis (MSP), Minneapolis (STP), Montreal (YUL), New York (LGA), Newark (EWR), Palo Alto (PAO), Phoenix (PHX), Portland (PDX), San Jose (SJC), Seattle (SEA), St. Louis (STL), Toronto (YYZ), Vancouver (YVR)), South Africa (Cape Town (CPT), Johannesburg (JNB)), India (Chennai (MAA), Mumbai (BOM), New Delhi (DEL)), and Europe (Amsterdam (AMS), Copenhagen (CPH), Dublin (DUB), Frankfurt (FRA), Frankfurt (HHN), Helsinki (HEL), London (LCY), London (LHR), London (LON), Madrid (MAD), Manchester (MAN), Marseille (MRS), Milan (MXP), Oslo (OSL), Paris (CDG), Stockholm (BMA), Vienna (VIE), Munich (MUC).”