My experience of those utterance is that it’s purely phatic mimicry: they lack genuine intuitive surprise, it’s just marking a very odd shift in direction. The problem isn’t the lack of path, is that the rhetorical follow-up to those leaps are usually relevant results, so they stream-of-token ends up rapidly over-playing its own conviction. That’s why it’s necessary (and often ineffective) to tell them to validate their findings thoroughly: too much of their training is “That’s odd” followed by “Eureka!” and not “Nevermind…”
I think that a lot of models have to sprinkle in a lot of "fluff" in their thinking to stay within the right distribution. They only have language as their only medium; the way we annotate context is via brackets and then training them to hopefully respect the brackets. I'd imagine that either top labs explicitly train, or through the RL process the models implicitly learn, to spam tokens to keep them 'within distribution' since everything's going through the same channel and there's no fine grained separation between things.
Philosophically, it's not like you're a detached observer who simply reasons over all possible hypotheses. Ever get stuck in a dead end and find it hard to dig yourself out? If you were a detached observer, it'd be pretty easy to just switch gears. But it's not (for humans).
Language really only exists at the input and output surfaces of the models. In the middle it's all numerical values. Which you might be quick in relating to just being a numeric cypher of the words, which while not totally false, it misses that it is also a numeric cypher of anything. You can train a transformer on anything that you can assign tokens to.
That's not my point. I'm talking about something far more mundane - transformers do inference over raw tokens and perform an n^2 loop over tokens, but tokens are itself the context. So it's better to have more raw tokens in your input that all nudge it to the right idea space, even if technically it doesn't need all those tokens. ICL and CoT have a lot of study into them at this point, these are well known phenomena.
This applies to any transformer-based architecture including JEPA which tries to make the tokens predict some kind of latent space (in which I've separately heard arguments as to why the two are equivalent, but that's a different discussion.)
Similarly, none of our comments actually exist as language on Hacker News—just numerical values from the ASCII table. We're deluding each other into thinking we're using language.
I believe it's reasonably clear that our thought processes generally occur outside of language. We do use language during explicit reasoning, but most thinking occurs heuristically. It's on par with the thinking of animals that don't use language but do complex behavior.
It not clear to me how well that maps onto LLMs. Our wetware predates language, and isn't derived from it. Language is built on top. LLMs are derived from language. I think that means that the intermediate layers are very different from the brain neurons, but I don't know. It's eerie how well the former emulates the latter.
There’s an interesting thing there that I believe varies person to person. My understanding is that some people do think in a more symbolic/heuristic way, some rely very heavily on their inner monologue to make sense of things (I am in the latter camp, and only have a single core language processor so pretty much cannot come up with coherent thoughts if I’m concentrating on what someone else is saying)
Even more interesting, and getting off on a bit of a tangent, there is also a mode that I use for revealing emotions that I don’t have words for (alexythmia): I open up a text editor, stare off into space, and let my fingers type without “observing” the stream of words coming out. I then go back and read what I “wrote” and often end up understanding how I’m feeling much better than I did. It’s weird.
Edit: also, playing with local models through e.g. llama-cpp in “thinking mode” is super fascinating for me. The “thought process” that comes out before the real answer often feels pretty familiar when I reflect on my own inner monologue, although sometimes it’s frustrating for me because I see where their “thinking” went off the rails and want to correct it.
And what I find fascinating is I see similar mimicking by my 5 year old. Perhaps we shouldn’t be so quick to call this a lack of being genuine. Sometimes emotions are learned in humans but we wouldn’t call them fake.
I don’t want to declare machines to have emotion outright, but to call mimicry evidence of falsehood is also itself false.
Mimicry is how kids learn the expected reactions to particular emotions. A kid mimicking your surprise doesn’t mean they are surprised (as surprise requires an existing expectation of an outcome they may not have the experience for), but when they do feel genuine surprise, they’ll know how to express it.
Because it's a statistical process generating one part of a word at a time. It probably isn't even generating "surprise". It might be generating "sur", then "prise" then "!"
But what is surprise really? Something not following expectation. The distribution may statistically leverage surprise as a concept via how it has seen surprise as a concept e.g. "interesting!"
So it can be both true that it has nothing to do with the emotion of surprise, but appear as the emulation of that emotion since the training data matches the concept of surprise (mismatch between expectation and event).
It’s the emotional and physiological response to a prediction being wrong. At its most primal, it’s the fear and surge of adrenaline when a predator or threat steps out from where you thought there was no threat. That’s not something most people will literally experience these days but even comedic surprise stems from that shock of subversion of expectation.
LLMs do not feel. They can express feeling, just as you can, but it doesn’t stem from a true source of feeling or sensation.
Expressing fake feelings is trivial for humans to do, and apparently for an LLM as well. I’m sure many autistic people or even anyone who’s been given a gift they didn’t like can relate to expressing feelings that they don’t actually feel, because expressing a feeling externally is not at all the same as actually feeling it. Instead it’s how we show our internal state to others, when we want to or can’t help it.
It is a mistake to equate artificial intelligence with sentience and humanity for moral reasons, if nothing else.
We are also technically a statistical process generating one part of a word at a time when we speak. Our neurons form the same kind of vectorised connections LLMs do. We are the product of repeated experiences - the same way training works.
Our brains are more advanced, and we may not experience the world the same way, but I think we have clearly created rudimentary digital consciousness.
Because it has no mind, no cognition, and nothing to "feel" with. Don't mistake programmatic mimicry for intention. That's just your own linguistic-forward primate cognition being fooled by the linguistic signals the training set and prompt are making the AI emit.
I could describe the electrical and chemical signals within your neurons and synapses as proof that you are merely a series of electrochemical reactions, and can only mimic genuine thought.
You could do that if you wanted to ignore reality and be reductive to score points in an argument by purposefully conflating mimicry with intention, yes.
And that is dogma. It's unthinking circular reasoning.
It wasn't very long ago that scientists were certain that animals did not posses thoughts or feelings. Any behaviour which appeared to resemble thinking or feeling was simply unconscious autonomic responses, with no more thought behind them than a sunflower turning towards the sun. Animals, by definition, lack Immortal Souls and Free Will, and therefore they are empty inside. Biological automata.
Of course this dogma was unfalsifiable, because any apparent evidence of animal cognition could be refuted as simply not being cognition, by definition.
Look, either cognition is magic, or it's math. There really isn't a middle ground. If you want to believe that wetware is fundamentally irreducible to math, then you believe it's magic. If that's want you want to believe, then fine. But it's dogma, and maintaining that dogma will require increasingly willful acts of blindness.
You are using word "math" in a magical way. Current LLM programs are reducible to math and human cognition is reducible to math (which is a reasonable hypothesis). What you are implying is that just because word math is used in both sentences it actually means the same thing. And that is a magical thinking. Just because human cognition is reducible to math (let's assume that for sake of discussion) doesn't mean it's the same math as in the LLM programs, or even close enough. Or maybe it is, but we don't have any proof yet.
I agree with this. I'm not arguing that LLMs are conscious. We don't understand the math behind how our brains work; we don't know how close or far LLMs are to that; and we don't know how many different pathways to consciousness there are within math.
All I'm saying is that the argument that "It's not consciousness, it's just <insert any tangentially mathematical claim here>", is dogma. Given everything that we don't know, agnosticism is the appropriate response.
> It wasn't very long ago that scientists were certain that animals did not posses thoughts or feelings. Any behaviour which appeared to resemble thinking or feeling was simply unconscious autonomic responses, with no more thought behind them than a sunflower turning towards the sun. Animals, by definition, lack Immortal Souls and Free Will, and therefore they are empty inside. Biological automata.
It's cool that you can decide to take half-remembered incorrect anecdotes about what "scientists" are certain of at some indeterminate time in the past, sans citation, and use that to underpin your argument about a totally different thing.
> Of course this dogma was unfalsifiable...
...like your post's anecdata.
> Look, either cognition is magic, or it's math.
Yes, when you decide to draw a convoluted imaginary bounding box around the argument, anything can be whatever you want it to be.
LLMs have no mind and no intention. They are programmed to mimic human language. Read some Grice and learn exactly how dependent humans are on the cooperative principle, and exactly how vulnerable we are to seeing intent where none exists in LLM communication that mimics the outputs our inputs expect to receive.
Your cries of "dogma dogma dogma" are unpersuasive and lack grounding in practical reality.
It’s funny that this is probably due to bias in the training texts, right? Humans are way more likely to publish their “Eureka!” moments than their screwups… if they did, maybe models would’ve exhibit this behavior.
Now that AI labs have all these “Nevermind” texts to train on, maybe it’s getting easier to correct? (Would require some postprocessing to classify the AI outputs as successful or not before training)
I think it's more explicit than that, part of post-training to enforce the kind of behavior, I don't think it's emergent but rather researchers steering it to do that because they saw the CoT gets slightly better if the model tries to doubt itself or cheer itself on. Don't recall if there was a paper outlining this, tried finding where I got this from but searches/LLMing turns up nothing so far.
My understanding is that it’s the result of these companies making sure to keep you engaged/happy less than the result of data these companies train with.
I don’t know if it’s true or not but it certainly tracks given LLMs are way more polite than the average post on the internet lol
I believe there might be more to it. Wasn't a big part of thinking or reasoning taking the response, replacing the final period with "Wait!" and then continuing? Which suggests that such words actually are important to the internals.
I think sometimes though there harness LLMs providing guidance. For instance I’ve seen recently coding agents doing an analysis then mid response saying “no wait, that’s not right” and course correcting. This feels implausible as an auto regressive rhetorical tick. LLM harnesses are widely used in advanced agentic systems and I’m sure the Pro level reasoning models exploit them extensively. I’m not saying this is what happened here, but there is a chance it was something injected by the hardness into its thinking.
My impression is that the quality of the conversation is unexpectedly better: more self-critical, the suggestions are always critical, the default choices constantly best. I might not have as many harnesses as most people here, so I suspect it’s less obvious but I would expect this to make it far more valuable for people who haven’t invested as much.
After a few basic operations (retrospective look at the flow of recent reviews, product discussions) I would expect this to act like a senior member of the team, while 4.6 was good, but far more likely to be a foot-gun.
Yes, with an actual payment (processed credit card transaction), a signed contract with clear payment terms, or a convincing promise to pay such as a written instruction to send an invoice.
A lot of startups have made the mistake of thinking "customers" are the same as "downloads of a free app" or "people who created an online account" or "people who signed up to be notified of the actual launch."
Accelerators once encouraged this ("you have to show progress to investors on demo day!") but unless you have actual paying customers it's not a real business.
> What do you mean by that? Promise some investment? Commit to something?
Commitment, in any form. Best is a signature on a purchase order.
The Mom Test is a good explanation of why you don't have PMF with the client until you receive a purchase order.
In short, people don't want to tell you things that hurt your feelings, especially if you're a good sales person (good sales people are likeable, you aren't closing if the client doesn't like you).
So you're sitting their with the clients team, and they like you, you made a good impression. No one wants to tell you "Look, it's not what we want, it's best you be on your way". What they'll do instead is:
1. Try to soften the blow with "We like it but we have to get sign-off from $VP first" (blame it on the non-present Big Bad Wolf); you tell them "Can we pull them in now? How about tomorrow?"
2. "Don't have the budget but we'll put it in the upcoming budget in a few months" (Hoping you won't know on their door again in 4 months); You tell them fine, we can sign now for deployment in a few months, before the rates increase you will have in a few months, and you'll throw in a discount as well.
3. "Can it do $X, $Y and $X" (Hoping you'll say "No" and then bugger off); you tell them they can make $X, $Y and $Z part of the signed purchase order.
4. "We have a lot of projects going on now, don't have capacity to manage this" (Hoping, again, you'll bugger off and forget to come back); this is one you don't respond to, you just back off because they don't need your product that badly.
If a business wants what you are selling, you will have a signed purchase order the next day. If you don't get that signed PO, you follow up over the course of maybe a month. After that you spend your limited sales resources on someone else.
Once, I spoke telephonically with an (existing) client in the morning on an upsell, and had their signed purchase order about two hours later. Fair enough, this was an existing client, but the upsell was for a completely new product.
My rule of thumb is "demo once, and record the demo". If higher ups need to see a demo before they okay, they can see the video. If the demo was not to the correct group, the correct group can see the demo (and they'll request another demo if they want one, the existing group won't ask for another demo without new participants).
Look, sales means you do more work than the client, but they still have some work to do, and if they don't do it then they're not committed.
Bad design on their part, another reason not to revisit! If a site breaks my workflow I generally stop using the site, rather than changing my workflow.
Though I'm guessing it would work in the cases being discussed in this article & thread: when you are navigating into a site (such as linkedin) from another, rather than following internal links.
Satellite source would require detailed editing, and there’s very little chance those are fully automated. The entire Middle-East is being blocked, but only Lebanon is being affected.
It could be that they have a provider in Lebanon that was bombed but I’ve never heard of a cartographer with local dependencies like that.
Yeah, that’s the default option for detailed databases like that. Large deletion are either technical issues (and that should affect a lot more than one country) or deliberate edits.
You can very easily verify the claim by following the link. Other than three major cities, there are no agglomeration listed in Lebanon. Other countries have detailed maps.
Is anyone stopping you from doing that? Do you need my permission? If so, granted. I think you should spend whatever effort you want to verify claims if you believe that would be of value.
Some big moneyed interests are trying to split Europe and the US.
The current US administration is definitely not helping, but every ad I see on the Reddit main feed is a blatant attack on the relation, from brand new subreddits, pointing at magazines I’ve never heard about before. I’ve been reporting them, but it keeps coming, from constantly different sources, different names, subreddits, but always the same vague but incredible incredibly provocative titles
I suspect that some social-media-addled senior US officials are being fed the same crap because their reactions to non-existent European reaction are not grounded in reality.
> Some big moneyed interests are trying to split Europe and the US. The current US administration is definitely not helping
Did you listen/read Vance's recent speeches in Hungary? Or read the US policy document put out months back? It goes way beyond merely "not helping" - the US administration is in turns provoking, alienating and separating itself from center/center-left European governments in pursuit of exporting extremist partisan politics in the hopes of getting far-right governments elected across Europe.
European citizens and politicians everywhere can see the actions for what they are. What was that about Greenland and annexing Canada? There's no big-money conspiracy, just a bully administration with no sense of second-and third-order effects.
I’m not saying that Vance is not doing that—God knows that man’s ethics has no floor.
I’m doubtful he paid for ads to make his disdain better known. So I suspect someone else is trying to make that happen beyond what Vance can with his speeches.
> I’m doubtful he paid for ads to make his disdain better known.
They are not separate efforts - the administration is working hand in glove with the said interests that Vance worked for in his VC days, sponsored his Senate campaign, and parachuted him onto the Trump ticket.
He's probably an instrument of those interests as well.
'We'll prop up this crazy narcissistic bully and the suckers are gonna vote for him, mainly because Biden has been a disaster. Then he'll put other idiots in charge, go after the EU and Iran just because and make us piles of cash in the process.'.
Classic case of PR leveraging a real, anecdotic observation on one single result, but completely flipping it to pretend it’s a systematic result, to saw doubt on all scientific findings around microplastic. The same companies behind this last story have done the same thing to slow down regulation to limit the impact of smoking, alcohol, processed foods, oil refinery, global warming, lead pipes…
I wish the millions of people killed by delaying safety legislation for decades knew that pretending to make jokes (what became known as the "stochastic asshole" approach) was also a common tactic taught by those PR firms, to make critics sound like sour-puss.
Do you know what a "useful idiot" is, in the Soviet manipulation tradecraft?
Someone who repeats, jokingly or not, an argument that was placed somewhere deniable. One lab, looking at a small study, published a correction saying their estimates were wrong because they didn’t realize how their gloves accounted for it. Do you know who knew about that? Every intern in every lab ever. This was a minor correction that should never has reached anyone except the 10 readers of their original report.
But, strangely, that story got a wide coverage in the press: the usual “science” publication, the trade press, even widespread media. Why? Because it was presented as a “They are making things up about micro-plastics” piece, and those can go really far. And that kind of coverage doesn’t happen by accident.
So no, I don’t think you did that deliberately. But I know you read about it recently; I know you didn’t check what that original story was that triggered the coverage; I know you found that quaint—and I have no reason to think you deliberately tried to spread misinformation. But, you did. Because the people who want to sow doubt know what they are doing.
Not as long as there are powerful car lobbies and the main source of microplastic will remain car tires.
Instead, you have articles like this trying to tell people to look away from that main source of problem, and blame, say, indoors or food preparation, and skip details like how the homes with the most microplastic in them are… close to the highway.
My experience of those utterance is that it’s purely phatic mimicry: they lack genuine intuitive surprise, it’s just marking a very odd shift in direction. The problem isn’t the lack of path, is that the rhetorical follow-up to those leaps are usually relevant results, so they stream-of-token ends up rapidly over-playing its own conviction. That’s why it’s necessary (and often ineffective) to tell them to validate their findings thoroughly: too much of their training is “That’s odd” followed by “Eureka!” and not “Nevermind…”
reply