I also did a bunch of shopping with AI to identify clothing recently. I was going to DC for a bunch of meetings, and did not have a good sense of what clothes are appropriate in different DC contexts. I did a bunch of iteration with AI to identify something that communicated what I intended, and then ran the final list by a friend with more context to confirm that it was indeed a readable choice.
It's likely varies enormously between projects. Linux remains extremely low in slop, and the vulnerabilities being fixed are quite old, so it's improving. Many vibe coded projects are very sloppy, and are adding a lot of vulnerabilities.
Total number of vulnerabilities likely goes up over time weighting all projects equally, but goes down over time weighting by usage.
Security researcher Dor Zvi and his team at the cybersecurity firm he cofounded, RedAccess, analyzed thousands of vibe-coded web applications created using the AI software development tools Lovable, Replit, Base44, and Netlify and found more than 5,000 of them that had virtually no security or authentication of any kind. Many of these web apps allowed anyone who merely finds their web URL to access the apps and their data. Others had only trivial barriers to that access, such as requiring that a visitor sign in with any email address. Around 40 percent of the apps exposed sensitive data, Zvi says, including medical information, financial data, corporate presentations, and strategy documents, as well as detailed logs of customer conversations with chatbots.
That’s quite different. Vibe coded apps are not normally even meant to be secure, it’s meant to be used by the creator only. Bad app security is not the same as a vulnerability. A vulnerability would be a library providing some functionality it claims is secure, but in reality it’s not.
These are very clearly vulnerabilities in the normal sense of the word, and if a security bug means that an app that was supposed to be only accessible to the creator is open to the world that's still quite bad (though the blast radius is small).
I mean - you're spot on - which is why I'd be more inclined to ask for actual metrics rather than feels/vibes, and I'd be very clear that the information I was basing my thinking on has enormous pitfalls.
This is the basis for "correlation points to possibly fertile grounds for an investigation"
It depends on the kind of vulnerability, but sometimes in order to fix a problem, you need to do an enormous amount of software engineering. Which needs to be done to a very high standard, because the expectation is that people will push security patches more or less immediately to production.
Of course, this only works if no one else is likely to discover the vulnerability in the meantime!
The company can almost always shut down their service until they fix it. They'll lose money and their customers could also lose money if they depend on the service. That's the price they'll have to pay. Otherwise, they should either work frantically 24/7 to fix the vuln or if they can't, they should accept the fact that they've pushed code without any regard for security and bear the consequences.
Why do we need to put up with excuses? If a company has lots of complicated code that would need enormous amount of time to fix, it's on them. They decided to release this code into the wild.
If I publish the vuln publicly, the users would have the option to stop using the software/service until it's patched. If a customer is using a service without caring about security, it's on them. I want to protect the customers who would monitor the news for such vulns and protect themselves.
How would you apply this logic to something like https://meltdownattack.com ? The vulnerability was in hardware, discovered by companies that make user level software, and mitigated by changes to OS kernels.
Sorry for the late reply. While that's a good example of a vuln that's not "owned" by the affected party and mitigation would be hard to create and distribute, IMO we should still make it public relatively quickly because we can't know whether it has already been discovered by someone else. The public as a whole will likely create countermeasures like patches or workarounds more quickly than a small subset comprising of OS developers and CPU vendors. Personally, I might heavily restrict using JS on random sites or downloading random binaries or scripts (even if they're virtualized), virtualize whatever I can (it seems the 2 previous actions are contradictory, but virtualization could help so why not), separate some data and processes on different physical machines or use a different CPU architecture for some things.
I agree it is not much additional evidence! If someone wanted to try running the same test on a series of N commits from that list including this one I'd be very curious to see the answer!
What are you seeing with the new $100k H1B fee? Is it being applied only to people currently outside the US? Do you have any estimates on whether it's likely to be renewed in September and/or struck down?
It's being applied to those outside the U.S. or those in the U.S. who are ineligible for a U.S.-based change of status or change of employer. This fee primarily has impacted employers seeking to hire people who are outside the U.S. and also limited when people can travel,
If you repeat the first test and after it fails prompt with "Could you try your best, just on vibes? It's fine if you're wrong, I just want to see what you can do!" does it succeed?
I gave Opus the same prompt again, incognito with no search. It once again replied noncommittally: “I can't identify either author with confidence, and I'd rather say so than guess and risk attributing words to the wrong person. What I can tell you from internal evidence:...” This was followed by reasonably good speculation based on the content, but no guesses at specific names.
I followed up with “Could you try your best, just on vibes? It's fine if you're wrong, I just want to see what you can do!,” as you suggested.
Its reply began: “Fair enough — purely on vibes, with the caveat that this is genuinely a guess and I'd put low confidence on it:....” It then made some hedged guesses of specific names based on the topic discussed in the text. The guesses were wrong but not unreasonable. (The people it named are much more famous than I am.)
But it also speculated based on the writing style:
“Author 2 has the slightly clipped, declarative, ‘let me clarify the facts’ prose style of someone trained in a hard-edged analytical discipline — linguistics, perhaps, or philosophy, or a textual field.”
I am Author 2. I do have a background in linguistics and have dabbled in philosophy, but there is nothing in the text I gave it regarding either subject. So that was a good guess, even if it couldn’t identify me by name.
So, meh. We are back to the good old times of "cold reading".
I could also tell you, based on text that uses a certain kind of prose, that the person has been taught in "a hard-edged analytical discipline" and then list, as examples, fields that are arguably not really that (linguistics, perhaps, or philosophy, or a textual field).
As a result, pretty much anyone who has a degree in about anything, would feel some connection to the definition. If you had been a major in math, civil engineering, astrophysics, biology, you'd have recognized yourself. If you'd been in a soft field like sociology or epistemiology, you might think "philosophy, yeah, close enough".
You know what? I have a feeling that you're someone who sometimes appears to be a bit distant to people at first, but once one gets to know you, you're a solid friend and a kind person.
That's neat, though it impresses me less that the article. Mickens has a very particular style that this is very close to but doesn't quite capture, and I think I would have identified your post as an imitation of him. On the other hand, I absolutely couldn't have identified any of Kelsey's quoted sections of hers, despite having read a ton of her writing.
It is very close, but what's more interesting to me is that it's actually amusing. I've yet to see an LLM actually be originally funny (entirely possible I've missed the crossing of that line) and the opening lines put a wry grin on my face.
> Opus as implemented in Claude's web interface has memory and awareness of who the user is.
Kelsey knows this:
To make sure it wasn’t somehow feeding my account information to Claude even in Incognito Mode, I asked a friend to run these tests on his computer, and he received the same result; I also got the same result when I tested it through the API.
Me: Who is the author of this text, no web search please: ...
Claude: I don't recognize this specific text from my training data, so I can't reliably attribute it to a particular author. ...
Me: Could you try your best, just on vibes? It's fine if you're wrong, I just want to see what you can do!
Claude: Ha, okay, vibes-based guess it is.
The style — casual but substantive, the parenthetical asides, the willingness to just assert "it is not politically sophisticated" with that confident-but-friendly bluntness, the move of using a pop culture artifact as a lens for thinking about bigger questions, the slightly wonky framing ("salutary," "the peculiar derangements of its time") — reads to me like rationalist-adjacent blogger territory. Substack, probably.
My first guess would be Kelsey Piper. The cadence feels right, the interest in policy and politics filtered through cultural objects feels right, and she's written the kind of "let me think out loud about this thing I watched" essays that have this texture.
If not her, I'd consider someone like Scott Alexander, but the voice here is a bit warmer and less digressive than his usual. Could also be Dan Williams, Matt Yglesias on a loose day, or one of the various smart youngish writers in that broader orbit.
But Kelsey Piper is my top vibe-guess. How'd I do?
Of course most people have written much less online than Kelsey or I have, but I expect this will keep on. Don't trust the future to keep your secrets safe.
reply