I used to think the same way wrt Nvidia stock when it tanked - compute is clearly diminishing returns. Tech companies subsequently announced capex equal to or greater than expected in compute. I smacked myself on the forehead when I realized - I'd been think too much like an engineer. Tech CEOs badly want to believe they have an edge over every upstart from San Francisco to Shanghai. Unlimited spending on compute gives them that reassurance. In fact, the more threatened they feel, the more they spend to cling onto it.
Kids have security blankets. Tech CEOs have security compute clusters.
This is the danger of being informed only by sensational headlines. Nvidia's stock has fully recovered and is again near an all-time high. You seem to be generalizing about "Tech CEOs" — but in this case, GPUs are the advantage. They are necessary to achieve the outcome, and yet they are severely supply constrained. It's smart to overpay now.
Apple did something similar with NAND storage for the iPad mini. They took a bet that could have been wrong. It was not wrong. Competitors had a hard time because of it.
load isn't what causes degradation, it's heat and as someone who has mined crypto for years I'm aware that there are a lot of things that can be done to run hardware quite hard and keep thermals low. Whether or not that is what is being done, I have no idea. A GPU mining crypto for 5 years kept below 65C (rather easily done) is going to have far more life left than a GPU in some kids gaming PC that spikes frequently to 85C for even a year.
Everything is near an "all time high." Microstrategy stock is hovering near an all-time-high, and they're just a company that buys up Bitcoin and wastes some of it. Meme coins are floating up to all-time-highs. Stop using asset prices to justify anything people are doing, they're fully decoupled from anything happening below.
I don't think I was informed by sensational headlines. I was well into talking to people I knew about how DS's performance relative to compute was a game changer much before the stock crash.
It's not binary where you either have compute or not. You definitely do need GPUs, but there's already masses of compute, I believe it doubles every ten months or so just from Nvidia's chips. Many factors make it a very irrational decision
1) Companies were spending hundreds of billions collectively on AI capex. Meta alone was 75 billion projected this year. This is an extraordinary bet, given that the most revenue any AI company makes is a few billion by OpenAI.
2) When DS came out, it was a huge validation of the moatless idea. These SOTA companies have no moat, at best they are spending tens of billions to maintain a few months edge.
3) DS was also a huge validation of the compute saturation idea - that SOTA models were always massively efficient. At best it was traded for iteration speed.
4) Many other more technical arguments - Jevons paradox, data exhaustion (synthetic data can only be generated for a fixed set of things), apparent diminishing returns (performance relative to compute, the denominator has been exponential but the numerator logarithmic)
So on one hand you have these SOTA models which are becoming free. On the other hand you have this terrible business model. I strongly suspect that AI will go the way of Meta's Metaverse - a staggering cash burn with no realistic path to profitability.
It's one thing to invest in a new technology with tangible benefits to your product. It's another to spend vastly, vastly more into vague promises of AGI. To put it into perspective, Meta will spend on AI capex in a few months of 2025 as much as Apple spent on NAND in total. What advantage is there to be had with SOTA models? You do 20% better on some AIME/IQ/competitive coding benchmark, which still translates atrociously to real world issues.
But Nvidia will be very successful because these companies frankly have lost a lot of the plot and are FOMOing like mad. I still have memories of the 2013 AI gold rush where every tech company was grabbing anything with AI in them, which is how Google got DeepMind. They are being enormously rewarded by it by the stock market with Meta's price 6x since it's lows.
It is entirely possible that LLMs end up serving some useful purpose, but don't end up being great businesses.
I can think of a million different software services that have some value to users, but don't have some multi-trillion dollar revenue stream flowing from them.
There is an idea that these LLM companies are going to be able to insert their agents into the labour market and capture some percentage of the savings that firms realize from laying off their human workforce. Given the competitive environment, it is far more likely that these LLMs become an undifferentiated commodity good, and their value to the economy gets captured elsewhere. Currently the only value being captured is at the infrastructure level, and that is all predicated on a series of assumptions around software business models that have not materialized yet.
Kids have security blankets. Tech CEOs have security compute clusters.