> Senate staff were instructed to respond to the email, which outlined a potential threat for the drill, with their status and location on Capitol Hill or off campus. But the thousands of simultaneous reply-alls that ensued first clogged up inboxes before staff were unable to get into their inboxes altogether.
That's why we test! Imagine if that happened during an actual crisis. Give them a break - that's the point of testing, to discover these glitches.
Note that another negative outcome is that high value targets (e.g., Schumer and McConnell) are emailing every Senate employee with their location, not to mention the email chain also provides aggregated information on where crowds of targets can be found.
Also, email ain't so secure. A sophisticated enemy - and I'm pretty sure the US government actually has some of those - would love that information. Heck, you can find locations just by locating mail traffic.
> The Senate’s email system, run through Microsoft Outlook
I love how out-of-touch HN comments can be sometimes, it’s a breath of fresh air for me. Outlook the mail software (as opposed to the coat of paint applied to hotmail) is used by almost the entire Fortune 500, basically every government at every size, and a decent chunk of the Windows-using western world (which is most of it).
If you’re wondering why you have trouble selling to enterprise and government clients, this type of gap in perception is a big part of it.
Given how wrong the rest of the comment was, confusing mail client and mail server seemed like small potatoes. Especially when it can be argued that mail flows through the client if you squint a bit.
The mail service in Microsoft 365 is called Exchange Online and it‘s based on Microsoft Exchange. Outlook.com (formerly) seems to have been been moved to the same infrastructure as Microsoft 365.
Gee, I wonder why F500s are getting ransomwared left and right :)
I get that giving your drones software that is widely used is the easiest path but ActiveDirectory, Outlook and Windows are basically three of the horsemen of insecure corporate networks. The fourth is probably adware disguised as virus scanners.
Ok I'll bite -- what are they supposed to use instead of ActiveDirectory? We going back to Netware?
Oh you don't want them to use Windows at all? So what, F500s are gonna run on Ubuntu and OpenLDAP? That should be smooth as butter and more secure out of the box /s
Some PsyOps person on here had a term for this, slowing your enemy down by getting them bogged down in unimportant details, but I've forgotten what it was...
Thus one who is skillful at keeping the enemy on the move maintains deceitful appearances, according to which the enemy will act. He sacrifices something, that the enemy may snatch at it.
By holding out baits, he keeps him on the march; then with a body of picked men he lies in wait for him.
Journalism isn't the profession it once was (if it ever was) from major corporations. They're just writing information they are fed by a source and spruce it up a little for views, maybe. There's no investigation going on anymore, there's no source checking generally, at least not like there used to be. The current business model doesn't allow for that in large organizations.
It's Politico, not an IT news source. I think someone on deadline potentially confusing Microsoft's client software and it's MTA is forgivable, particularly given the ever-shifting landscape of MS' branding in this area
You seem to have confused socialism and fascism. A government too entwined with a small number of corporations, to the point that it seems like it's a government for the corporations rather than the people, is generally considered a feature of fascism. A government embracing some sort of anti-corporate and free-and-open solution like OSS would resemble a socialistic policy more.
But yes, I agree, the USG (and other governments) really should embrace Linux/OSS and help contribute to the ecosystem as well (unlike North Korea, for instance, which has a government-maintained Linux distro called Red Flag, but of course doesn't contribute anything at all).
> A government too entwined with a small number of corporations, to the point that it seems like it's a government for the corporations rather than the people, is generally considered a feature of fascism. A government embracing some sort of anti-corporate and free-and-open solution like OSS would resemble a socialistic policy more.
Huh? We know what the socialist policy was. It was that the government was so closely entwined with corporations that they were staffed by government officials and explicitly considered arms of the government. None of the features you mention in your comment represent a distinction between socialism and fascism. They're shared by both systems. They just go farther in socialism than they do in fascism.
I would recommend a refresher on your political economy knowledge.
In fascism, the government worked very closely with corporations, but it was somewhat a two way street (corporations getting juicy contracts because they're friendly with the right people, owners financing the politicians, etc.). In socialism "corporations" are owned by the state/worker's councils/etc. (which is why there are "government employees" there - technically everyone works for the government). But there are no corporation owners working in their interest getting handouts and/or pushing for specific actions and policies in socialism.
> But there are no corporation owners working in their interest getting handouts and/or pushing for specific actions and policies in socialism.
Are you describing any difference other than the use of the word "owner"?
There are people who control the corporations and receive the benefits of their activities. Those people lobby for policies that are good for them. That's how socialism works. It's also how every other system works. Does it matter whether those people prefer to be called "owners" or "secretaries"?
Reply-All cascades are classic entertainment in government work.
When I was in the Air Force some poor Warrant Officer started a Reply-All clusterfuck when he replied to a base-wide email about issued Android phones.
After about 30 Reply-Alls some very senior officer replied (to all) something to the effect of "Nobody else Reply, or your ass is grass" to which some joker from Special Operations replied (all) "Copy, Sir".
Some of the most fun you can have in the public sector.
We had this in college where someone sent "M to mute" and then thousands of people started sending emails with "M" in the body. Nobody knows how many of them were being sarcastic, but it became a meme that we found ways to insert into all sorts of unwanted group communication channels.
When the friends groupchat becomes overly active about a topic only few are interested in (not including me), I joke and reply UNSUBSCRIBE. I think I'm the only one to understand, and definitely the only one to laugh
These are just as fun at large corporations. A bunch of people you've never met, all very mad at each other. (I always join in on these. I simply cannot resist.
One time, someone emailed me privately to say that I should know better. Oh, I do. I just want to watch the email server burn.)
I really don't know why it's so hard to get basic competency in public sector to use bcc or restrict send access to large groups. That said, I enjoyed the drama when an email devolved into requests to stop replying all and requests to be taken off the list.
"A recent international research study allows us to quantify the difference between the broad population and the tech elite. The data was collected from 2011–2015 in 33 countries and was published in 2016 by the OECD (the Organisation for Economic Co-operation and Development, a club of industrialized countries).
In total, 215,942 people were tested, with at least 5,000 participants in most countries. The research aimed to test the skills of people aged 16–65."
USAF doesn't have warrant officers (used to, but the last ones retired last century), but yeah I saw a couple of these in my time, too. Not a lot of fun.
At amazon in 2010, it started with a meeting invite that was accidentally sent to everyone. The meeting was for the Amazon Wallet team for whatever work they were doing at the time. A reply-all storm ensued, and despite several people asking to stop hitting the reply-all button, it continued for several days.
The management had to chime in and sometimes threatened people with punishment if they continued participating to the reply-all storm.
If you ignored the conversation you may not have seen it, but most reply-all storm ended up with someone saying something the line of "we don't care about your wallet".
Anyway, asking people to stop hitting the reply-all button is far from being the latest reply-all on these kind of things...
I finally got around to watching The West Wing. There's a lighthearted small subplot of exactly this in S01E19.
By the way, it is so refreshing nowadays to watch a show about extremely competent imperfect people that legitimately strive to do as much good as they can, as they see it. Likewise with Sorkin's writing, while unrealistically witty and fast, it is a delight watching someone so obviously talented give their all to their work. Granted, as in other Sorkin writing, it does sound a bit like all the characters are just versions of him with different voices. But it seems to me like an intentional stylistic choice, a somewhat heightened reality.
The movie Malice is one of my all-time favorites. 1993, average reviews. Total random movie. Recently I paid attention to the credits: written by Aaron Sorkin. It all makes sense now. It's been ignored for whatever reason but it's amazing.
My other favorite is when corporate comms or HR sends out some calendar invite to a massive number of people, does not use bcc, and then the server crashes when folks go in to the accepted invite on their calendar and you get thousands of calls to 'check availability' flood in.
Have seen that bring the Exchange Server to its knees a few times :)
Way back in the day when I was doing a lot of Exchange consulting work, we would recommend removing permission to send to all except for one or two specific accounts. This was to avoid reply-all meltdowns, for security (viruses couldn't send to all), and also from an HR / internal communication perspective. We would recommend something similar for department-level groups, but often with a bit more freedom.
The latter is best. I think another sensible action is to remove “all” from the reply. If you really want to send to all, you have to enter it yourself.
I've seen this happen a few times, usually someone accidentally spams their entire address book, or even better a bunch of listserv addresses. Then you get the "reply all" messages to "take me off this list" or "unsubscribe" which just adds fuel to the fire.
The last storm I was in had people replying all to tell people to not reply all, followed by people replying all to shame the people who replied telling people not to reply all.
Followed by others replying all asking how to mute the conversation, and others replying all with instructions. Multiple times. Which prompted more people to reply all commenting on the increasing recursion of the absurdity of the whole thing.
So you'd get fed up and mute the chain and think you'd think you'd killed the process, but any time someone changed the email subject line and replied all again it came back like a bad penny. Which caused more people to angrily reply all with vulgarities, which caused supervisors to reply all that their behavior was being noted and disciplinary meetings would be had.
And then one day, someone replied-all to one of the child email chains with the entire script of The Bee Movie. Which launched more of all of the above. And did the same thing to every other child email chain, sporadically, for a few months. I think it took over half a year for the last reply-all to quiet down.
I think at least one of those chains lives on for posterity in one of my folders, and now you tempt me...
One of the privileges of previously being an Exchange admin was being able to send the "Do not reply all" message and then setting up a rule on the server to kill all further messages.
But it's also why I had a script to lock down any distribution list with more than 100 members to authorized senders only.
Back in the days before ransomware and RATs and APTs and ubiquitous cloud-hosted corporate email, when viruses were accidentally introduced rather than spear-phished, state/criminal-gang-sponsored existential threats to a company, I had the idea (without the ability to manifest it) of a virus that would infect an MS Exchange instance and set everyone's Out of Office on and all incoming email to be forwarded to every other mailbox in the company.
Not sure if it is or was feasible, but I still like the simplicity of an automation-stack doing the really hard work of exponential amplification.
It's 2023, we have systems processing trillions of transactions per minute, and still a few thousand emails can "melt" an email server? That's testament to how little consideration the mail stack gets.
Well, kinda. Due to features like BCC headers, each recipient is supposed to get their own unique copy of every email, so email servers can end up spending a lot of time writing tens of thousands of copies of every mail sent to the list to disk. And then of course they have to sync that to all of the redundant servers, to support HA.
I don’t entirely disagree, but while deltas can be small emails are usually pretty small themselves. It might be a single 4k write to each mailbox either way. And remember that appending to a file requires the OS to do a read plus a write, while adding a file to a directory requires a write plus an update to the directory itself (another read+write at minimum). Both require allocating sectors, which means updating the freelist(s), Of course there are filesystems like ZFS that can batch up all of those writes so that they become linear writes instead of random writes, but that’s not available if you‘re running Exchange.
I have no doubt that email servers could be better optimized, but only because I have that optimistic belief about all software as a general rule. (The probability that we have already found the most optimal way to write any non–trivial program you happen to examine is pretty low.) On the other hand, I doubt that Exchange has made _no_ progress at all over the decades.
The fact that their email server fell over from the load probably has more to do with poor choice of hardware than any software flaw. Their Exchange server apparently fell over while sending email to “thousands” of people, while the Bedlam DL3 incident at Microsoft involved a mailing list with 13k people on it. In 1997. Maybe that order of magnitude difference means that the Senate needs to buy an NVME disk. (Of course that ignores the fact that asking thousands of people to send you their location via email is pure stupidity; this is the Senate we’re talking about.)
Presumably they are more interested in information like which room someone is in than their position on Earth. I doubt that indoors the location accuracy of phone GPS would be accurate enough to be useful, particularly in buildings where many locations are designed to be secure both from physical attacks and espionage (probably a lot of thick metal plates in random walls). And even without interference issues, GPS elevation accuracy is pretty bad. Finally, you're only tracking the location of the person's phone, which in an emergency could easily get left behind somewhere.
None of this is to say an email chain is the optimal solution though.
The gov could write an app, installed on every senator's phone, that tracks their location as needed.
Perhaps the senators might be against such invasive surveillance? Then I'd hope their voting record on such bills matches their privacy concerns when they're the ones affected.
Remember the days before mailing list distributers filtered out of office replies? It was fairly common to open your mail in the morning and have to delete several hunderds of these auto replies and dozens of replies from people complaining about it adding to the bulk.
This happened to us several years ago, a basic reply-all storm across probably 30k employees. The icing on the cake though - when the storm finally calmed down a few hours later, the original sender then tried to recall the message. The recall notification went out to everyone and sparked it back up again.
And here I thought the oxy-moronic reply-all pavlovian response had been trained out of humans. I've been fortunate enough to work on teams where I haven't witnessed this in at least a decade. Of course its in our technically inept government where this happens.
Of all things happening in a West Wing episode (although to the White House, not the Senate) to still be topical 25 years after airing, this would not have been high on my list. I'm not sure to consider myself overly optimistic or overly naive...
Same thing happened at Microsoft, and Facebook. I'm sure it's happened anywhere with large email lists. Servers should probably just reject replies to lists large lists.
That's why we test! Imagine if that happened during an actual crisis. Give them a break - that's the point of testing, to discover these glitches.
Note that another negative outcome is that high value targets (e.g., Schumer and McConnell) are emailing every Senate employee with their location, not to mention the email chain also provides aggregated information on where crowds of targets can be found.
Also, email ain't so secure. A sophisticated enemy - and I'm pretty sure the US government actually has some of those - would love that information. Heck, you can find locations just by locating mail traffic.
> The Senate’s email system, run through Microsoft Outlook
It's 2023 and someone is still writing that?