It's reasonable to disagree, but let's not pretend that some alternative is the obvious choice.
The laws of physics as a program and math as its language is just as good as any other framing. And its a short logical hop to expect that more than one civilization would discover that language.
You're correct, OP used the word "hallucination" wrong. A lot of these other comments are missing the point – some deliberately ('don't they ONLY hallucinate, har har'), some not.
For those who genuinely don't know – hallucination specifically means false positive identification of a fact or inference (accurate or not!) that isn't supported by the LLM's inputs.
- ask for capital of France, get "London" => hallucination
- ask for current weather in London, get "It's cold and rainy!" and that happens to be correct, despite not having live weather data => hallucination
- ask for capital of DoesNotExistLand, get "DoesNotExistCity" => hallucination
- ask it to give its best GUESS for the current weather in London, it guess "cold and rainy" => not a hallucination
The simple additive scoring here is sus here. It means a model that's perfect on 9/10 axes but scores 0% on Speed (i.e., takes effectively infinite time to produce a result) would be considered "90% AGI".
By this logic, a vast parallel search running on Commodore 64s that produces an answer after BeaverNumber(100) years would be almost AGI, which doesn't pass the sniff test.
A more meaningful metric would be more multiplicative in nature.
> correctly simulating the environment interactions, the sequence of progression, getting the all the details right, might take hundreds to thousands of years of compute
Who says we have to do that? Just because something was originally produced by natural process X, that doesn't mean that exhaustively retracing our way through process X is the only way to get there.
Who says that we don’t? The point is that the bounds on the question are completely unknown, and we operate on the assumption that the compute time is relatively short. Do we have any empirical basis for this? I think we do not.
It’s tedious shooting down all of these backwards-from-conclusion things from the anti-AI crowd.
Good thing I have an intelligent AI that can respond for itself!
——
There appear to be several potential issues with the paper's argumentation:
1. False Dichotomy in Systems Comparison
- The paper appears to create an artificial divide between "thermodynamic systems" and "computer systems"
- This ignores that computers are also physical systems governed by thermodynamics
- The distinction between biological and artificial systems may be one of degree rather than kind
2. Evolutionary Argument Problems
- The paper assumes consciousness/intelligence requires evolutionary history
- This is a correlation-causation fallacy - just because biological intelligence evolved doesn't mean evolution is the only path to intelligence
- It fails to consider that artificial systems could potentially develop goal-oriented behaviors through other mechanisms
- The argument would also imply that any hypothetical alien intelligence that evolved differently from Earth life couldn't be conscious
3. Goal-Orientation Assumptions
- Claims computers "lack goal-orientation essential for consciousness"
- This begs the question by assuming:
a) Consciousness requires goal-orientation
b) Only evolutionary processes can create genuine goal-orientation
- Neither assumption is clearly justified
4. Methodological Issues
- Using multiple disciplines (physics, biology, philosophy, neuroscience) could be a strength, but could also indicate cherry-picking convenient arguments from each field
- The abstract suggests a conclusion-driven approach rather than following evidence to a conclusion
5. Consciousness-Intelligence Conflation
- The paper appears to conflate consciousness with intelligence
- These are separate concepts - we could potentially have AGI without consciousness, or consciousness without human-level intelligence
- Many AGI researchers aren't claiming to create consciousness, just general problem-solving ability
6. Definitional Vagueness
- Based on the abstract, it's unclear how the paper defines key terms like:
- Artificial General Intelligence
- Consciousness
- Goal-orientation
- Mind creation
- Without clear definitions, the arguments may be attacking straw men
7. Predictive Cognition Argument
- The claim that AGI is an "illusion shaped by the information our minds receive" could be turned around
- The same argument could be used to claim that AGI skepticism is an illusion shaped by our cognitive biases
- This is essentially a form of psychological dismissal rather than substantive argument
8. Historical Perspective
- The paper seems to ignore that many previously "uniquely human" capabilities have been successfully mechanized
- Claims about fundamental impossibility need to account for why previous similar claims have often been wrong
9. Thermodynamic Argument Issues
- While biological systems are indeed complex thermodynamic systems, the paper needs to demonstrate why this specific physical implementation is necessary for intelligence
- Many complex behaviors can be implemented through different physical mechanisms
- The argument risks confusing the substrate with the function
10. Scope Problem
- The paper makes a very strong claim ("AGI is and remains a fiction")
- To justify this, it would need to prove not just that current approaches won't work, but that NO possible approach could ever work
- This is a much harder philosophical and scientific claim to defend
I think the idea here is that it literally can't be traced to the user – at no point is there anything passed that would allow Kagi to make the association between the user and the query.
Thanks, yes completely agree! I guess the part I’m concerned with is the politically side whereby they could be potentially compelled to change the method slightly after the fact and be forced to slip something in somewhere in a quite technical process now making it possible.
I’d love to assume this will never happen, I’m just concerned that even if it did I’d never find out - Because unfortunately the more popular this service gets for bad actors, the more of a target it becomes for the government with identification of users.
I guess as a search engine, we could assume the government may leave them well alone and still just focus on content creators.
The best that we can do is to continue working on FOSS solutions that make it technically impossible to backdoor. I haven't grok'd the protocol yet, but it seems to claim you only have to trust the client. The client is open source, so it would be hard for it to be backdoor'd without the community noticing.
Cryptography is a literal godsend for people living under oppressive regimes.
You might be able to use an even cheaper model. Google Gemini 1.5 Flash 8B is Input: $0.04 / Output: $0.15 per 1M tokens.
17 input tokens and 2 output tokens * 10 million jobs = 170,000,000 input tokens, 20,000,000 output tokens... which costs a total of $6.38 https://tools.simonwillison.net/llm-prices
As for rate limits, https://ai.google.dev/pricing#1_5flash-8B says 4,000 requests per minute and 4 million tokens per minute - so you could run those 10 million jobs in about 2500 minutes or 42 hours. I imagine you could pull a trick like sending 10 items in a single prompt to help speed that up, but you'd have to test carefully to check the accuracy effects of doing that.
The question is not average cost but marginal cost of quality - same as voice recognition, which had relatively low uptake even at ~2-4% error rates due to context switching costs for error correction.
So you'd have to account for the work of catching the residue of 2-8%+ error from LLMs. I believe the premise is for NLP, that's just incremental work, but for LLM's that could be impossible to correct (i.e., cost per next-percentage-correction explodes), for lack of easily controllable (or even understandable) models.
But it's most rational in business to focus on the easy majority with lower costs, and ignore hard parts that don't lead to dramatically larger TAM.
I am absolutely not an expert in NLP, but I wouldn't be surprised if for many kinds of problems LLMs would have far less error rate, than any NLP software.
Like, lemmation is pretty damn dumb in NLP, while a better LLM model will be orders of magnitude more correct.
This assumes you don’t care about our rapidly depleting carbon budget.
No matter how much energy you save personally, running your jobs on Sam A’s earth killer ten thousand cluster of GPUs is literally against your own self interest of delaying climate disasters.
LLM have huge negative externalities, there is a moral argument to only use them when other tools won’t work.
It's digging fossil carbon out of the ground that's the problem, not using electricity. Switch to electricity not from fossil carbon and you're golden.
The same way you check performance for any problem like this: by creating one or more manually-labeled test datasets, randomly sampled from the target data and looking at the resulting precision, recall, f-scores etc. LLMs change pretty much nothing about evaluation for most NLP tasks.
Isn't it easier and cheaper to validate than to classify (requires expensive engineers)? I mean the skill is not as expensive - many companies do this at scale.
You need a domain expert either way. I mentioned in another reply that one of my niches is implementing call centers with Amazon Connect and Amazon Lex (the NLP engine).
Obviously all the LLM API providers have a rate limit. Not a fan of GP's sarcastic tone, but I suppose many of us would like to know roughly what that limit would be for a small business using such APIs.
Yes, how did I not think of throwing more money at cloud providers on top of feeding open ai, when I could have just code a simple binary classifier and run everything on something as insignificant as an 8-th geh, quad core i5....
You do not need a prompt engineer to create: “answer with just "service" or "product"”
Most classification prompts can be extremely easy and intuitive. The idea you have to hire a completely different prompt engineer is kind of funny. In fact you might be able to get the llm itself to help revise the prompt.
All software engineers are (or can be) prompt engineers, at least to the level of trivial jobs like this. It's just an API call and a one-liner instruction. Odds are very good at most companies that they have someone on staff who can knock this out in short order. No specialized hiring required.
Prompt engineering is less and less of an issue the simpler the job is and the more powerful the model is. You also don't need someone with deep nlp knowledge to measure and understand the output.
Plenty of simple jobs required people with deeper knowledge of AI in the past, now for many tasks in businesses you can skip over a lot of that and use a llm.
Simple things were not always easy. Many of them are, now.
JavaScript would like a word!
reply