To summarize something from a little over a year after I joined there: Cloudflare was building out a way to ship logs from its edge to a central point for customer analytics and serving logs to enterprise customers. As I understood it, the primary engineer who built all of that out, Albert Strasheim, benchmarked the most likely serialization options available and found Cap'n Proto to be appreciably faster than protobuf. It had a great C++ implementation (which we could use from nginx, IIRC with some lua involved) and while the Go implementation, which we used on the consuming side, had its warts, folks were able to fix the key parts that needed attention.
Anyway. Cloudflare's always been pretty cost efficient machine wise, so it was a natural choice given the performance needs we had. In my time in the data team there, Cap'n Proto was always pretty easy to work with, and sharing proto definitions from a central schema repo worked pretty well, too. Thanks for your work, Kenton!
Albert here :) Decision basically came down to the following: we needed to build a 1 million events/sec log processing pipeline, and had only 5 servers (more were on the way, but would take many months to arrive at the data center). So v1 of this pipeline was 5 servers, each running Kafka 0.8, a service to receive events from the edge and put it into Kafka, and a consumer to aggregate the data. To squeeze all of this onto this hardware footprint, I spent about a week looking for a format that optimized for deserialization speed, since we had a few thousand edge servers serializing data, but only 5 deserializing. Capnproto was a good fit :)
I also think his thesis isn't to celebrate poverty but to consider that development (or modernity, globalization, pick a term) comes with its own problems.
(Yes to the earlier post - had a similar experience with an early metrics system for Twilio in 2012-2013)
The 1st issue is that ZMQ implementations generally have an IO thread handling all socket IO. REQ/REP patterns tend to work out just fine, because the client is going to wait for a reply from the server. But things like PUSH/PULL become...harder to follow. You can push things in the client, but how do you know when they've flushed out of the IO thread and made it to the server? Those sorts of things matter when shutting down a process, running integration tests, etc. In contrast, with an HTTP request, you're basically always doing REQ/REP, and so you know when the data has been pushed.
The 2nd issue is that ZMQ implementations tend to be harder to observe / operate than a more common path like HTTP requests. (HTTP requests can go through a load balancer, have standard response codes, use headers for authorization, etc.) For these reasons, I've tended to avoid ZMQ ever after -- it seems like often, you're best off either using REST or GRPC if by HTTP, or raw TCP if it's purely a data push kind of operation (e.g. forwarding structured logs, with framing, to a remote TLS endpoint).
You don't have to agree with her politics, but I think Peggy Noonan had the right answer on this one: it's a show and there's no good to be found in taking part.
Just say no and hit send.
> In February 2018 Nicholas Thompson and Fred Vogelstein of Wired wrote a deeply reported piece that mentioned the 2016 meeting. It was called so that the company could “make a show of apologizing for its sins.” A Facebook employee who helped plan it said part of its goal—they are clever at Facebook and knew their mark!—was to get the conservatives fighting with each other. “They made sure to have libertarians who wouldn’t want to regulate the platform and partisans who would.” Another goal was to leave attendees “bored to death” by a technical presentation after Mr. Zuckerberg spoke.
Chloramine is responsible for the classic "swimming pool smell", as the chlorination in the pool reacts with urine, sweat, and other nitrogen-containing organics. Chloramine is a less effective disinfectant than free chlorine, and more irritating to swimmers, and diffuses out of the water more slowly, so this is why you shouldn't pee in the pool, and why you should rinse off in a shower for a minute before entering. Mainly, just don't pee in the pool.
If you smell "swimming pool" where no pool exists, it might be prudent to leave the area immediately and take a 1000 mg vitamin C tablet, before making any attempts to discover the source of the smell. (Taking vitamin C after the damage is already done won't help.) If the odor is especially strong, or if it causes any irritation to your nose and eyes, evacuate, and do not return. Call your emergency services number and tell them you smelled a strong chlorine odor.
Don't combine bleach cleaners with ammonia cleaners, ever. Don't even store them in the same place. Also avoid mixing bleach with acids, such as vinegar.
Chlorine can react with about anything that contains ammonia or amines to form chloramine, so shampoos, lotions, shaving foams, or conditioners could also be responsible. Amino acids are named thus for their amine groups, so anything containing proteins or protein fragments will react. Amine oxides and triethanolamine are used as surfactants in cosmetics. Sweat is unlikely to be the culprit. Check the ingredients in your cosmetics.
I've not spent enough time on Pearl, but Migual Hernan and Jamie Robins have a free textbook on causal inference at https://www.hsph.harvard.edu/miguel-hernan/causal-inference-... ; parts of it were pretty useful for a causal inference assignment I did earlier this year.
It'd be hard to understand without feeling it for yourself, but in actual instrument conditions (IMC), it's easy for your sense of balance to mislead you into thinking you're level, when in fact you're turning.
There's some physiology to explain this, but one key point is that once you're in a coordinated turn, even a descending one, your weight is still going straight down the vertical axis of the seat. So, you'll think you're doing fine, and possibly even pull the turn tighter (the graveyard spiral), and fixate on things other than the attitude indicator and your plummeting altitude.
Michel Foucault also wrote about this essay. His essay starts thus:
> Today when a periodical asks its readers a question, it does so in order to collect opinions on some subject about which everyone has an opinion already; there is not much likelihood of learning anything new. In the eighteenth century, editors preferred to question the public on problems that did not yet have solutions. I don't know whether or not that practice was more effective; it was unquestionably more entertaining.
> In any event, in line with this custom, in November 1784 a German periodical, Berlinische Monatschrift published a response to the question: Was ist Aufklärung? And the respondent was Kant.
* Hour restrictions for passenger transport are significantly more stringent than for freight.
* Southern Air's fleet consists 737s and 777s -- not small planes.
* These plans have to fly over and into major metropolitan areas.
* Many parts of flying can be automated, but in the end, and for aircraft this size, you probably want a human operator in the plane for the sheer reason that they are better equipped to handle unplanned emergencies.
It's a tough call whether flying freight or regional airlines pays worse -- but in either case, you're anywhere from a quarter to half of what a software engineer of similar years experience in the bay area would make. The hours are in a lot of ways worse -- mostly because you're away from home a lot of the time, and you have really limited control of your schedule. (As someone who's worked as an on-call engineer for ten years, and who also flies, I'd take the on-call responsibilities over flying freight any day of the week.)
I'd agree that the article lacks for details. But, a relatively quick reading through things like airliners.net will give you an idea of the kind of hours and pay these guys typically work -- it's not an easy job. If we care about their safety -- and our own, given the number of freight aircraft flying over us every night -- we shouldn't write them off as just asking for money they don't deserve.
I think most people will agree that the $200k+ starting salary in the bay area is an anomaly. Everyone is doing everything to "fix" this problem.
If we care about safety, we prohibit them from doing things that are unsafe. If management tries to make them do things that are prohibited, we should make it easy for people to say no without repercussions.
You're right. But as always in history, these changes can't come by asking companies nicely. You either need a strong union that is willing to strike (and Southern actually has one) and/or federal regulators.
Why are hour restrictions different for freight than passenger? Adapting this would probably already change much. And a strike at Prime Air for a few weeks during holiday season should also help improving conditions.
Anyway. Cloudflare's always been pretty cost efficient machine wise, so it was a natural choice given the performance needs we had. In my time in the data team there, Cap'n Proto was always pretty easy to work with, and sharing proto definitions from a central schema repo worked pretty well, too. Thanks for your work, Kenton!