EMPIRE OF AI: Dreams and Nightmares in Sam Altman’s OpenAI
Karen Hao
Penguin Press, 2025
482 pages
Karen
Hao’s Empire of AI is a deep dive into the rise of OpenAI, the
organization that released what quickly became the world’s most popular Large
Language Model (LLM) ChatGPT.[1] At center stage in Hao’s book is
OpenAI CEO Sam Altman, the current darling of Silicon Valley. Her narrative
revolves around efforts to oust Altman from OpenAI in November 2023 and his
ability to brush off such attempts by turning what was once a non-profit
research lab into a standard Silicon Valley tech corporation.
Empire
of AI paints a detailed yet engaging portrait of the AI industry,
one that shows how a combination of scientific talent, ideological zeal,
ruthless business and an abundance of good fortune helped the company morph
into the innovation poster boy it is today. It includes interviews with workers
in historically oppressed countries who perform much of the critical work that
enables AI to function. Its coverage of the environmental impact of AI and the
efforts of major firms like Google to cover-up environmental abuses is valuable.[2]
It also
features much material about Altman himself. Hao recounts many junctures in the
evolution of OpenAI in which he used personal charm to seduce scientists and
investors before an unexpected decision left them feeling used and betrayed. Empire
of AI describes the many falling-outs Altman has had with ex-OpenAI
employees and investors such as Elon Musk, OpenAI’s ex-lead scientist Ilya
Sutskeker, and Dario Amodei, who would later found rival AI firm Anthropic.
Hao lays
out the web of relationships that enabled Altman in the first place. Most
notable was Peter Thiel, who has acted as a mentor for over a decade, imparting
a good deal of his techno-libertarian ideology in the process. She argues
convincingly that the elitist ideology of a handful of Silicon Valley
billionaires is guiding AI development much more so than scientific or social
concerns. Hao recounts how politicians in Congress are eagerly offering support
in the form of massive infrastructure scaling, deregulation and public funding
despite warnings from a multitude of academic researchers.
One of
the book’s strengths is its clear and concise explanation of technical
controversies among AI researchers and how they relate to speculation about
AI’s capabilities. Hao does a good job of dismantling utopian claims that AI
systems will soon outpace human intelligence in every domain.
All tech
products are the result of shared scientific knowledge of all humanity as well
as, in many cases, direct government funding. Invariably, after development
they pass into the hands of a narrow slice of billionaires. Hao shows how AI is
the culmination of this ongoing privatization process. She explains how
generative AI systems like OpenAI’s ChatGPT are trained on effectively huge
amounts of stolen data, whether it be proprietary computer code, e-books or
academic articles.
Whereas
a heist of this magnitude would lead to criminal consequences for you or me, AI
companies enjoy the backing of powerful world governments. Although Hao doesn’t
mention it, it cannot help but recall the tragic case of Aaron Swartz who
killed himself on the eve of sentencing as he faced up to 35 years in jail. His
crime was pirating academic articles from the JSTOR database - small change
compared to OpenAI’s open looting.
Overall,
anyone seeking an even-handed and up-to-date treatment of the industry. But it
is not without certain weaknesses
The role
of AI in the economy, and the amazing hype that has accompanied it at every
step of the way, are not discussed. There is also little treatment of perhaps
the most plainly insidious aspect of AI development: its use in a military
context and for mass surveillance.
But
perhaps the most significant weakness, politically speaking, is the book’s
central argument that some features AI industry resemble classic 19th-century
imperialism. This ascribes a role for AI and big tech that is utterly
exceptional in world-historical terms. As is often the case with radical
liberal criticism, the effect is to conflate symptoms with the underlying
disease and thus let modern imperialism off the hook.
This may
seem like nit picking of an otherwise strong book. But they are essential for a
Marxist understanding of AI and its politico-economic role.
What is
AI
First, a
quick aside on AI and how it became such a source of excitement and
controversy. Its definition is often slippery, but it generally refers to
computer algorithms capable of simulating human cognitive abilities. Artificial
Intelligence was first coined in 1956, its fortunes rising and falling in line
with the ups and downs of technological development in the ensuing decades.
Older
techniques from the 2000s and 2010s weren’t called AI at the time but probably
should be now. For example, Google’s search algorithm or the knowledge graphs
behind the logistics network of companies like Amazon have had a massive
impact, but aren’t usually thought of as AI. primarily for economic and
cultural reasons rather than scientific. These technologies matured before the
“thawing” of the last AI winter around about 2012. It should be noted that
there have been already been two “AI winters” since 1956, both setting in
because AI had failed to come good on short term promises of financial returns,
leading investors to jump ship. These cycles, highly disruptive in terms of
scientific research, are prime examples of how damaging the anarchy of production
and research can be in an ultra-financialized capitalist world system, even on
its supposed home turf of innovation.
In the
2020s, the new rage is generative AI techniques (Gen AI), the tech behind
ChatGPT and other chatbots. These learn from vast troves of data and then
generate outputs typically in text, image or audio form. Despite the
renaissance of terminology like AI and Artificial General Intelligence (AGI) –
more on that later – Gen AI is just the latest link in a long chain of
developments in digital technologies in the last few decades.
Gen AI
is based on neural nets with architectures loosely – much looser than is often
admitted – inspired by the neural architecture of the human brain. While neural
methods had been popular in the 1980s, they were too computationally expensive
for the hardware at the time while data was also hard to come by. While there were
some genuine algorithmic advances in the 2010s that laid the framework for
modern models, they have primarily been made possible by massive improvements
in computer chip technology and the availability of previously unimaginable
amounts of data on the Internet.
LLMs
have been successful in capturing the public’s imagination with their capacity
for amusing and even shocking interactions with human users. LLMs comfortably
pass the Turing Test [3],
once regarded as the premier test of whether a machine can match or exceed
human intelligence. However, Turing’s test is based on an overly-linguistic
view of human intelligence, and passing it does not show AI is effective at
reasoning or interacting with the physical world.
![]() |
"Stochastic Parrots at work" IceMing & Digit / https://betterimagesofai.org / |
Tech
leaders cite Gen AI’s supposed world-shattering novelty and insist that, unlike
previous tech innovations and the burst bubbles they precipitated, Gen AI won’t
fail to deliver on its promises. The uncritical amplification of these claims
has become a modus operandi for tech journalists. Willingly or not, they
all seem to have forgotten the precedents for the current hype around AI, not
just within the tech world but within the narrower AI field itself. This is not
to say there is no novelty with current techniques, but that their capacity to
change the world beyond recognition is almost deliriously overblown, leading to
unreal stock market valuations and seemingly endless speculation about AI’s
impact.
Whether
or not AI systems will ever exhibit “true” intelligence, there is little
question on a practical level concerning their ability to perform some of the
tasks we are trained to do at school or university or in the workplace. However
overhyped, there is no doubt that Gen AI tech does have significant economic
value. As with earlier episodes like the dotcom bubble of 2000, genuine value
was involved, even if not quite as much as tech CEOs and investment bankers
believed.
AI and the Economy
Empire
of AI closes with OpenAI’s announcement of its transformation into
a for-profit corporation in late December 2024 – apparently timed for the
holiday period to reduce public scrutiny. Its press release declared:
We once
again need to raise more capital than we’d imagined. The world is moving to
build out a new infrastructure of energy, land use, chips, datacenters, data,
AI models, and AI systems for the 21st century economy. We seek to evolve in
order to take the next step in our mission, helping to build the AGI economy
and ensuring it benefits humanity.[4]
But what
does an AGI economy mean? Elsewhere, Hao explains how Artificial General
Intelligence (AGI) is a notoriously contentious concept[5]. Most of the time, it refers to an
AI that is smarter or more capable than a human being. This isn’t very
enlightening since we don’t have reliable metrics for human intelligence and
those we do have are not suitable for measuring AI (IQ tests and exams rest of
a series of assumptions about human capabilities that do not carry over to AI.)
If AGI
means doing a human job at many times the speed and scale, then it could be
argued we’ve had super-intelligent AI systems for a long time (The efficiency
and scale of Google Search is incomparable to someone trying to find a document
by hand in a database.) But if it means replacing human beings by being better
at absolutely everything they do, then it is clearly nowhere near fruition.
Generating text, images, and computer code may one day replace a computer
programmer or an academic researcher (although human oversight will still be
required), but it isn’t going to clean a sewer or cook a meal anytime soon.
That AGI
is ill-defined does not stop tech CEOs from touting its imminence to the public
and lawmakers alike. To prevent China from getting there first, we must
deregulate tech and massively fund AGI development – or so the argument goes.
That’s
the stick. The carrot is that AGI is going to radically transform the economy.
The argument is that AI advancements will make labor so productive that we will
no longer have enough jobs and the vast majority of people will have to find
other things to find meaning in life - even if it is currently impossible to
find meaning in anything other than work.
Such
claims are not unprecedented. In 1930, as the world reeled from the Wall Street
Crash, John Maynard Keynes wrote an essay titled “Economic Possibilities for
Our Grandchildren” [6]
that predicted that, within two generations, the average person would work a
15-hour-week. Even during the Industrial Revolution figures such as John Stuart
Mill and David Ricardo warned of mass unemployment as a result of technological
innovation. Yet the job base since the 19th century has grown many
times over.
Today, the
great debate in the AI commentariat is whether AI is so powerful that it will
replace most if not all cognitive tasks. Unlike previous technological
innovations this would primarily impact white-collar jobs which require
literacy skills or specialized knowledge such as software developers.
The
broad answer is, yes, AI is that powerful. The ability of LLMs to generate
computer code, visuals, audio and text in seconds with small amounts of human
input means it can massively increase productivity in jobs which produce
content of this kind. Opponents of AI will, rightly, charge that this will lead
to a major reduction in the quality and significant safety concerns.[7] However, for better or worse, this did not stop lower quality mass production
superseding artisans during the Industrial Revolution.
While it is clear AI will impact some jobs, accurately measuring just how many is much more difficult. Much current hype is based on anecdotal evidence about major staff reductions. But such claims are hard to judge other than a few areas in which the impacts are exceptionally well documented (e.g., the mass replacement of customer service handlers with AI voice programs). Otherwise, it’s entirely possible that employers are using AI to justify layoffs that were already in the works and that other workers are being forced to pick up the slack. Rather than innovation, the result is little more than an old-fashioned speed-up.
AI researchers have struggled to come up with more objective measures. Reports from Anthropic [8] and Microsoft [9] base their estimates on the queries users submit to current chatbots. The Microsoft study concludes that jobs like sales representatives and customer service representatives (which employ 1.142 million and 2.859 million people respectively) are directly in the line of fire.
One of the most successful areas of AI application seems to be in software development. AI systems like Anthropic’s Claude 4.1 Opus and the recently released GPT5 are capable of coding entire applications or websites based on natural language descriptions by a user. However, recent evidence [10] suggests that, for more experienced coders, the use of such tools reduces productivity. While the coding ability of these models is impressive for users with little or no coding experience, they can quickly become a hindrance when applied to more specialized problems or are large code banks.
It
should also be acknowledged that AI’s ability to replace a job doesn’t mean it
will since such replacement requires knowledge and capital investment. In other
words, exposure doesn’t always equal replacement. Economists are often more
skeptical. For instance, Nobel Prize winning economist Daron Acemoglu predicted
in 2024 that total productivity gains from AI will not exceed 0.71 percent
across next 10 years [11],
a forecast based, moreover, on rather generous estimates of AI’s capabilities
produced by OpenAI researchers in 2023.[12] He expects the impact on jobs to be modest overall, all things being equal.
AI and the continued rise of the stock market
A key
component of the supposed drive toward Altman’s “AGI economy” is the “scaling
hypothesis,” which has led to a phenomenon known as a “hyperscaling” business
strategy. The argument is that all we need to reach AGI is more data, more
processing power, and hence more government-backed data centers across the
globe.
This has
led to speculation about “emergent” AI capabilities, the idea that once a
certain quantity of processing takes place, capabilities like reasoning arise
without the model being explicitly trained to develop them.
This is
a case where the supposedly scientific explanation of how to develop AI
dovetails with major corporate efforts to gain a monopoly positions over
rivals. While early LLMs improved drastically as their data base expanded,
there is little evidence that more and more data will lead to substantially
better models beyond a certain point. Indeed, a plateau seems to have been
reached in mid-2024. Since then, much of the progress has come from
improvements in model architecture or data quality (e.g., the Chinese based
DeepSeek’s V3 R1 model that shocked world markets earlier this year as well as
OpenAI’s recently released GPT5 model).
Nevertheless,
the unquestioned belief in this hypothesis is what is driving AI companies to
build growing numbers of bigger and bigger data centers in order to create
bigger and bigger models. The upshot is a “hyperscaling” arms race. This is
leading to the construction of gargantuan data centers across the US, polluting
major areas and drying out water supplies.[13] Hao’s own reporting on data centers in northern Chile and Uruguay shows the
devastation that this bigger-and-better philosophy has caused.
The
scramble for data centers is the primary driver of huge valuations of chip
companies NVIDIA (now the world’s most valuable company with a market cap of
$4.28 trillion) and AMD- who provide the chips used to construct the giant
computers housed in these buildings. Between them, they have a near-monopoly on
the graphic processing units (GPUs) needed to train and run large AI models.
Traditional
big-time players like Meta, Google, Microsoft, and Apple are competing to
integrate AI into their products. The aim is to leverage AI systems throughout
the economy to improve productivity and reduce labor costs. This promise has
become a critical prop for world capitalism in the past three years. According
to DataTrek, without the so-called Magnificent Seven stocks comprising the
major tech companies and NVIDIA, the S&P 500 would scarcely have grown
across 2023 and 2024 (it actually increased in value by 24.2 and 23.3 percent
respectively across those years).[14]
According to Paul Kedrosy [15],
half of the US’s economic growth in the last year came from data-center
construction.
This has
helped drive the stock market to new heights without a clear route to
profitability. AI research labs like Anthropic and OpenAI have burnt through
cash since the release of their LLMs, with many commentators speculating the
models actually cost far more to run on current hardware than revenue from
subscription services. Yes, these costs are likely to significantly decline,
but OpenAI’s revenue is still fairly limited compared to ad revenue brought in
by Google or Facebook.
This didn’t stop OpenAI
from valuing itself at $500 billion in a recent initial public offering Its
aspirations for a future monopoly on advanced AI products seem unlikely due to
the speed with which other industry players are able to catch up to or even
leap ahead. OpenAI’s revenue is estimated to be at most $5billion and it
continues to make gargantuan loses on all of its service offerings. In a
bombastic but informative summary of the crazy numbers involved in the AI
industry Ed Zitron argues,
“What is
missing is any real value generation. Again, I tell you, put aside any
feelings you may have about generative AI itself, and focus on the actual
economic results of this bubble. How much revenue is there? Why is there no
profit? Why are there no exits? Why does big tech, which has sunk hundreds
of billions of dollars into generative AI, not talk about the revenues
they’re making? Why, for three years straight, have we been asked to “just wait
and see,” and for how long are we going to have to wait to see it?”[16]
Microsoft,
Apple and Google’s investment in data centers and AI research seems to be more
about keeping existing users in platforms like Microsoft Office, iOS or the
Google Suite rather than creating brand new products. Similarly, Musk’s massive
investment in AI is aimed at attracting users to X, with the hope it can be
transformed into an everything app analogous to WeChat in China. Whoever the
eventual winner(s) are in this race, it is not clear how this will boost
profits drastically - except for eventual ability to jack up prices for
consumers for the same services that already exist.
One
striking rationale for tech stocks continued rise comes from none other than
Peter Thiel, who recently conceded that AI is viewed as a last chance saloon
before markets face the reality of general economic stagnation. As he told the
New York Times Interesting Things podcast, “The fact that we’re only talking
about A.I. — I feel that is always an implicit acknowledgment that but for
A.I., we are in almost total stagnation.” And the economic data supports this
point of view as well.
Following
the disappointing release of GPT5 there are some signs the AI bubble may be
about to burst. Bloomberg asked “is the AI winter finally upon us” [17] and the Washington post wondered if
the AI Industry was “off course”.[18]
Perhaps most ominously, however, on August 18 Altman himself declared AI was in
a bubble, “Are we in a phase where investors as a whole are overexcited about
AI? My opinion is yes.”[19]
Clouds are certainly forming.
Should
workers be opposed to AI?
Even though precise answers about AI’s ability to automate jobs are hard to come by, we do not need precise measures, luckily, to sketch out important political questions concerning potential job displacement.
AI-fueled
job displacement is first and foremost a political question. Similar to
outsourcing production to low-wage economies or investment in machinery during
the Industrial Revolution, using AI to increase productivity and lay off
workers is a way for companies to increase their profits. As in the SAG-AFTRA
strike last year, workers in many industries will face a choice between
opposing AI or losing their livelihood.
In the
abstract, workers should be in favor of all productivity-raising technologies.
Even before AI, Keynes was right to think that his grandchild’s 15-hour work
week would be technologically feasible. AI’s ability to replicate a number of
cognitive tasks only makes the irrationality of having people work endless
hours more obvious. With the correct utilization of AI and other recent
innovations, our collective work and private lives could look very different
indeed, starting, not least, with being a good deal shorter. In an ideal world,
workers fighting layoffs would call less for an end of AI and more for the use
and development of AI to be in their own hands.
Unfortunately,
we do not live in an ideal world, and will undoubtedly be used in the near-term
to assault jobs and working conditions while enhancing the ability of managers
to spy on workers.
One critical factor is that even a limited introduction of AI tools will prepare the ground for further replacement. The more workers use these tools, the more data they provide on how to do their job, which is exactly what is needed to train newer models to replace an ever larger share of the tasks they perform. AI agents that take over computer desktops to perform multi-stage tasks are trained on data recorded from office workers – although so far they have proven to be less successful than initially hoped.
![]() |
"Wheel of Progress" Leo Lau & Digit / https://betterimagesofai.org |
A more
successful example of this has already come through driver less cars, which for
years have been trained on data taken recorded from millions of human drivers.
Now the driver less taxi service Waymo is growing rapidly in San Francisco,
overtaking Lyft’s market share in the city in recent months. Of course, an
efficient public transport system would render most taxi services superfluous
without any AI magic, but that is a discussion for a different day.
In
upcoming struggles, this may mean opposing AI’s introduction to a workplace
completely for a time as a way to defend livelihoods. This should go along with
a clear message that the issue is not AI, but corporate use of it to attack
workers. Those opposing AI will undoubtedly be called Luddites and enemies of
progress – a worn out cliche raised by bourgeois media outlets whenever workers
oppose job cuts or wage freezes imposed in the name of technological
innovation. However, they ought not to be deterred, and Marxists should not
cede ground on this question.
It
should also be kept in mind that whether AI really can replace a job or not
will often be irrelevant to layoffs across many industries. As happened in the
jobs massacre in the American auto industry in the aftermath of the 2007
financial crash, when automation was used as an excuse to cut tens of thousands
of jobs and restructure the leading US auto producers from top to bottom, AI
will undoubtedly be used in the context of a new crisis as a convenient
justification for layoffs regardless of its impact on productivity.
Although
AI is an impressive technology with many potentially uses, it is a technology
massively shaped by the ideology of the ruling elite, in particular the
emerging fascist-libertarian culture of Silicon Valley tech billionaires. Marx
argued that technological developments reflect the interests of capital, and AI
is no exception.
But the
methods behind AI could be leveraged for a huge expansion of economic
productivity the world over. The underpinning of modern AI innovations are
statistical methods of almost unimaginable scale. At the moment, these are
leveraged primarily to recommend products on Amazon, determine social media
feeds, and, in the case of LLMs, predict the next word (or pixel). At the
moment these algorithms are leveraged primarily for profit (or in the case of
LLMs some vague imagined future profitability). Such methods – and the hardware
also monopolized by big tech - could clearly be leveraged to make possible
economic planning and productive efficiency such as is required to overcome
global warming and poverty.
Even if
AI never reaches the vague yet dizzy heights promised by tech billionaires, in
the long run it has the potential to massively increase productivity across
society. However, the benefits of this will only ever be reserved for a narrow
elite unless the technology and its future development is put in the hands of
those who develop and use it: the working class.
AI’s
military and surveillance uses
Another
significant issue that is not covered in Hao’s book is the use of AI for
military and surveillance purposes. Again, it could be argued convincingly that
this is outside the purview of Hao’s book. However, it is a critical aspect of
the AI industry and throws light on the industry’s increasing links to the
state.
Upon
Altman’s reinstatement to OpenAI after a failed attempt to oust him in 2023, a
new board of directors including ex-Secretary of the Treasury Larry Summers and
retired US Army General Paul M. Seasoned, solidified OpenAI’s government ties.
Under the Biden administration, and now increasingly under Trump, AI has been
identified as a top national security priority. The big players in the
industry, despite their previous association with Democratic politics, have
continued to enjoy preferential treatment under Trump.
In the
days after his inauguration – in which groveling tech leaders famously lined up
to pay homage to the new president – Trump hosted Altman at the White House to
announce the $500-billion Stargate initiative. By dollar value, this is the
largest infrastructure project in US history. Over the next four years, the
project aims to massively expand the US data processing capabilities by
building a huge wave of data centers. Since Hao’s book has been published,
OpenAI, Anthropic, Meta, and X have each signed $200-million dollar contracts
with the US government for their services.
While
the advantages of such applications are yet to be seen, what existing AI
systems have shown is the deadly result of further distancing the combatant
from the consequences of his or her actions. This has been seen more tragically
in Israel’s bombardment of Palestinians in Gaza. A 2024 report by +972 magazine
explains how the Israeli Defense Forces introduced a range of AI systems to
generate bombing targets in Gaza.[20]
Using
traditional target identification methods, Israel would have run out of targets
to bomb within a few weeks of October 7. According to +972’s sources the new
Lavender AI system analyzed surveillance data from Gaza and identified 37,000
new individuals as “junior Hamas officers” who then became targets for the
Israeli bombs. Another AI system, grotesquely named “Where’s Daddy,”
automatically inform human monitors when targets returned to their families in
the evening so that the Israeli Air Force could launch a bombing raid.
It is
clear that AI is a further development of the tendency of military technology
to distance the combatant from the consequences of their actions. A 972+ source
who has worked with Lavender told the magazine it reduces the human role in
target selection to rubber stamping the system’s recommendations – offloading
moral responsibility for the decision to target an official and their entire
family from a human analyst to an algorithm.
Without
understanding the particular architecture, it is of course difficult to know
how targets are generated. However, with large AI systems based on neural nets,
the factors that go into a recommendation are often a black box, even for the
system designers, leading to a phenomenon known as “hallucinations.” This is
where incorrect information is included in the output. Many factors completely
unrelated to involvement with Hamas may well have played a role in target
selection. Precision bombing has always been a fraudulent notion and
introducing AI technology into target selection hardly reduces “collateral
damage.”
The
IDF’s AI systems are undoubtedly some of the most advanced in the world;
indeed, their continuous surveillance of their own population and Palestinians
in the West Bank and Gaza has provided a treasure trove of data with which to
train such models. The development of equivalent AI systems in imperialist
armies is undoubtedly underway the world over, in many cases inspired by
“combat-tested” systems like Lavender.
Beyond
generating new bombing targets, AI is seen as a critical tool by militaries for
its capacity to process data and inform decision making in superhuman time. The
promise is that the analysis and synthesis of battlefield data necessary for
tactical decisions can be achieved in seconds, giving a massive speed advantage
over conventional foes. The industry leader in developing these technologies is
US-based firm Palantir, whose latest advert ran with the slogan, “battles are
won before they begin.”[21]
Palantir’s
stock valuation has increased 23 times since the start of 2023 was founded by
Peter Thiel among others. This is no doubt due to the general hype around AI
technologies and the massive injection of public money into rearmament. But it
is also due to Palantir’s close relationship with the civilian and military
arms of the Trump administration. It has signed a number of contracts with
various sections of the US government in the last year including a $10-billion
contract with the Pentagon on July 30. Its remit includes using generative AI
and new drone technologies to develop cutting-edge military products.
Palantir
is also on the leading edge of efforts to leverage AI to bolster surveillance
inside imperialist countries. The firm has led the Trump administration's
efforts to gut government agencies and increase executive power over the
functions of the US government. One of the many controversies surrounding Elon
Musk’s Department of Government Efficiency was its ability to gain access to
critical government databases, including Social Security data and Treasury
records, which means the data of every American has effectively been stolen.
Palantir has been contracted by the Trump administration to combine these data
sources into a single database on all Americans. Additionally, it is working
with Immigration and Customs Enforcement “to track migrant movements in
real-time.”[22]
Under
the guise of efficiency, the US government and Palantir are effectively trying
to produce surveillance technology that will be able to link individuals to
their bank accounts and home addresses behind the backs of the courts and
Congress, data that was previously separated within the government to avoid
such concentrations of power.
Not only
will this enable Trump to spy on and intimidate opposition within the US, but it
is also a boon for Palantir as a private company. Essentially, it has been
handed some of the most extensive data in the world to develop new AI systems
for free. Under the auspices of AI development and efficiency the US government
has thus stripped itself of some of its most valuable assets and placed them in
the hands of Palantir.
![]() |
"Surveillance View A" Comuzi / https://betterimagesofai.org / Image by BBC |
The
company’s reach is not confined to the US. In the UK, where the company is
headed by Lewis Mosley the grandson of late British fascist Oswald Mosley,
Palantir won a £330-million contract to rework the National Health Service’s
data infrastructure. Apparently, this system has been rejected by hospital
staff across the country as unsuitable for the purposes. This has led the
Labour government to bring in KPMG to somehow turn around staff attitudes at
the cost of £8 million. Such are the “efficiencies” of privatization.[23]
The
distinction between the “surveillance capitalism” industry, to use the term
coined by Shoshana Zuboff [24], and government surveillance is
diminishing. Far from regulating these industries, the state is handing the
keys to data infrastructure that would cost millions on the private market
None of
which poses radically new questions for Marxists. Rather, it poses old
questions within a new framework. The use of AI on the battlefield threatens to
massively increase the efficiency of an army’s capacity for death and
destruction. Along with the massive rearmament being undertaken by imperialist
powers internationally, the battlefield use of AI, in particular its use for
target selection and “preventive strikes” must be strongly opposed.
The same
goes for mass surveillance. The data people produce belongs to them by rights,
yet has been misappropriated by private companies to maximize advertising
profits. Its – often illegal – use by governments for surveillance has also
been a reality for decades. AI may not fundamentally revolutionize how
surveillance is carried out [25],
but it does promise to enable spying at an even larger scale than was
previously possible.
The AI
Industry as an “Empire”
So far,
we have discussed important aspects of the AI Industry that are excluded from
Hao’s book. Now we will turn to an issue that is included. This is her central
metaphor of AI as an empire. She explains:
Over
the years, I’ve found only one metaphor that encapsulates the nature of what
these AI power players are: empires. During the long era of European
colonialism, empires seized and extracted resources that were not their own and
exploited the labor of the people they subjugated to mine, cultivate, and
refine those resources for the empires’ enrichment. They projected racist,
dehumanizing ideas of their own superiority and modernity to justify- and even
entice the conquered into accepting – the invasion of sovereignty, the theft,
and the subjugation. They justified their quest for power by the need to
compete with other empires: In an arms race, all bets are off. All this
ultimately served to entrench each empire’s power and to drive its expansion
and progress.[26]
She
continues:
The
empires of AI are not engaged in the same overt violence and brutality that
marked this history. But they, too, seize and extract previous resources to
feed their vision of artificial intelligence: the work of artists and writers;
the data of countless individuals posting about their experiences and
observations online; the land, energy and water required to house and run
massive datacenters and supercomputers. So too do the new empires exploit the
labor of people globally to clean, tabulate, and prepare that data for spinning
into lucrative AI technologies. They project tantalizing ideas of modernity and
posture aggressively about the need to defeat other empires to provide cover
for, and to fuel, invasions of privacy, theft, and the cataclysmic automation
of large swaths of meaningful economic opportunities.[27]
Hao’s
book makes clear that OpenAI and other big tech players like Google, Microsoft,
Meta and X all do a great deal of horrible things reminiscent of 19th-century
empires.
![]() |
"Frontiers of AI" Hanna Barakat / https://betterimagesofai.org/ |
The
comparison isn’t wholly false, but it is superficial and misleading. The
analogy reflects a long-standing tendency among left-liberal critiques to
present the crimes of capitalism as peculiar to certain industries. Reports,
books and films – often with extremely valuable material - on the horrific
conditions for workers, damage to consumers, and environmental destruction in
this or that branch of production abound. At various points in the last two
decades, rare-earth metal mining, garment production, the arms industry, and
even fish farming have come under attack as exceptionally evil industries.
However,
critiques limited to evil industries or individuals, as repugnant as they may
be, end up promoting illusions in capitalism’s capacity for reform. The
argument, sometimes explicit and other times implied, is that the solution is
not a radical change in the organization of society, but a few more regulations
to rein in a given industry’s excesses.
This is
how Hao concludes her introduction:
But the
empires of AI won’t give up their power easily. The rest of us will need to
wrest back control of this technology’s future. And we’re at a pivotal moment
when that’s still possible. Just as empires of old eventually fell to more
inclusive forms of governance, we, too, can shape the future of AI together.
Policymakers can implement strong data privacy and transparency rules and
update intellectual property protections to return people’s agency over their
data and work. Human rights organizations can advance international labor norms
and laws to give data labelers guaranteed wage minimums and humane working
conditions as well as to shore up labor rights and guarantee access to
dignified economic opportunities across all sectors and industries.[28]
In the
conclusion to the book, titled “How the empire falls,” Hao expands on this
view, proposing a three-pronged framework for “dissolving empire”[29]
by undermining the industry’s monopoly knowledge, resources, and influence. Here
she suggests a number of reasonable measures including funding to support
independent evaluations of AI models and alternative approaches to AI development,
forcing companies to disclose training data, stronger labor protections, and board-based
education about how AI systems work as an “antidote to mysticism and mirage of
AI hype.”[30]
Having
laid out the growing strength of the AI industry and the subservience of
elected representatives in the US, Kenya, Chile and Uruguay, these suggestions
strike the reader as highly limited. Critically, they do not object to AI technology
itself being held as private property- which again, Hao’s book exposes as an absurdity,
given the technology’s reliance on data garnered from centuries of humanity’s
shared labor stored on the Internet and algorithms developed by researchers
over decades.
Hao’s
suggestions leave us with a “call your local representative” type of Democratic
Party protest politics. The implication is essentially that “If only our politicians
knew how horrible AI is” then they would undoubtedly act against it. However,
as Hao makes clear, the AI industry has been the sweetheart of Democratic and
Republican administrations alike and neither party is ever going to take on an
industry that is propping up the US stock market.
That
politicians will not even entertain the limited measures suggested by Hao is an
indication that a much more radical approach is needed. Empire of AI
makes clear is that the guiding lights of all of the decisions made in the
production of ChatGPT and other companies’ models was how to minimize costs and
scale as quickly as possible to try to gain monopoly position. Indeed, her
analyses of the ideology driving individuals like Altman and Thiel show that
this race for monopoly is their explicit aim. If this requires hyper-exploiting
desperate workers in Kenya or Venezuelan immigrants and destroying access to
fresh water across Latin America and the US, then so be it.
Hao’s
detailed work ultimately leaves us with an account of the AI industry that is much
less exceptional than the author realizes. While the industry may be among the
most extreme examples of corporate greed and thievery, the general tendencies
are of a piece with most major 21st century industries from food
production to auto.
The
question of how to eliminate the rampant exploitation in the AI industry, as
with many others, is ultimately one of taking on the world capitalist system
itself. AI, the hardware, and the algorithms behind it ought to be public
utilities, run according to the needs and interests of the great mass of the
world population. This will require a much more thoroughgoing transformation of
society than a few policy tweaks.
Conclusion
The main
aim of this review has been to critique the limited political conclusions and
oversights of an otherwise very valuable work. Despite claims that AI is
somehow completely novel, the political and social questions it poses are not
fundamentally different from those raised in the face of previous technological
developments from electricity, the production line, and the Internet.
That is
not to say each one of these technologies does not have its own particular
challenges and opportunities for revolutionaries. But in the most general
terms, opposing mass job displacement, the extension of the bourgeois state
further and further into private life and politics, and the threat of more
deadly military systems have strong historical parallels.
What is
clear is that appealing to policymakers and human rights organizations is not
enough. AI is a modern globalized and highly exploitative capitalist industry.
The empire Hao speaks of is not OpenAI, Google, or Facebook, but the entire
apparatus of the modern imperialist states, with America far ahead of its
rivals, that work to advance the interests of every major corporation,
investment fund and bank above all else.
At the same time modern AI systems are technological development and a step further in the concentration of technology and data into the hands of major corporations and banks. As discussed above, they are not just based on clever algorithms but on all artifacts of human culture on the Internet, which have been developed by billions of people over the course of human history. They have only been made possible due to the uncountable labor of masses of working people, and it is they who should control and benefit from these systems.
[1]
Technically, models like GPT4
or o3 are LLMs and ChatGPT is an extra layer on top that allows users to have
conversational chats with the model, but most people refer to ChatGPT as a
single AI tool.
[2] According
to a Goldman Sachs report (https://www.goldmansachs.com/insights/articles/gs-research/generational-growth-ai-data-centers-and-the-coming-us-power-surge/report.pdf)
from last year the data centers needed to develop and run AI systems, as well
as tech’s other digital services, will account for 8 percent by 2030 compared
to 3 percent in 2022. The report argues that generative AI is likely to be the
key driver of this increase in demand, accounting for 200 TWh of power demand
by 2030.
[4] Empire
of AI, p.407 (quoting https://openai.com/index/why-our-structure-must-evolve-to-advance-our-mission/)
[5]
In the aftermath of the disappointing release of OpenAI’s flagship GPT5 model,
Altman has recently walked back his efforts to associate himself and the OpenAI
brand with AGI, telling
CNBC last week “it’s not a super useful term.” It was only 8 months ago that Altman
proclaimed in a blog post, “We are now confident we know how to
build AGI as we have traditionally understood it.”
[11] https://economics.mit.edu/sites/default/files/2024-04/The%20Simple%20Macroeconomics%20of%20AI.pdf
[13] A
recent investigation by More Perfect Union (https://www.youtube.com/watch?v=3VJT2JeDCyw)
on Elon Musk’s efforts to build the world’s biggest data centers in Memphis
gives an insight into the devastating impact this infrastructure is having on
communities.
[14] https://www.marketwatch.com/story/the-entire-stock-market-is-being-carried-by-these-four-ai-stocks-6e69fbbd
[24] https://www.theguardian.com/books/2019/oct/04/shoshana-zuboff-surveillance-capitalism-assault-human-automomy-digital-privacy
[25] For
example, the arrests of leaders of the Baader-Meinhof group, a left-wing
terrorist group, in 1971 occurred after a special police unit used a computer
to estimate where members most likely lived in Berlin and then did deep
searches in those areas.
[26] Empire
of AI, p.16
[27] Ibid
p.17
[28] Ibid
p.19
[29] Ibid
p.419
[30] Ibid
p.421
17 comments:
A good article. Agreed that the military implications are highly under-appreciated. The automation of genocide in Gaza is only the beginning - imperialism has caught on that drones & AI will change warfare, and they're now moving to rapidly develop and implement these technologies. The danger to humanity from this tech will exceed by far that posed by nuclear weapons.
I just want to comment on one part, which is your (and Hao's) claim that "their [AI's] capacity to change the world beyond recognition is almost deliriously overblown." You downplay the concepts of AGI and super-intelligence as ill-defined or unrealistic and liken the current hype to past eras. This is profoundly mistaken.
It is true that there is not much new in the techniques. The large language models of today are nothing more than giant stacks of machinery that was invented in the 50s and 60s. It is also true that there is hype about the short-term economic effects, which has led to a massive bubble. And finally, it’s true that the current techniques are not sufficient, by themselves, to generalize in the way that human intelligence does.
However, it is not true that “emergent” AI capabilities are mere “speculation” or that “scaling” is nothing but an “unquestioned belief.” Both are empirical facts and essential to understanding what’s going on. The process of learning can be thought of as a process of self-organization. A simple learning rule, applied many times over a brain with many neurons can lead to an incredible amount of complexity. Two things are true at once: AI is not doing everything that the human brain does, but it IS doing part of it. All you have to do is interact with a chatbot for a while to understand that these things are not just smoke and mirrors, there is something remarkable going on.
The human brain is material, and as such, its basic laws can be discovered and reproduced. Moreover, the human brain is not the final product of evolution; it can and will be surpassed. So while the immediate consequences of AI are over-hyped, the not-so-distance consequences are actually still under-hyped.
We (or rather completely unaccountable tech monopolies) are in the process of building machine intelligence. They are moving as fast as they can to build powerful systems that no one understands, and that will ultimately pose an existential threat to humanity, possibly quite soon. That is not some kind of illusion that can be dismissed with the wave of a hand, as Hao does, it is a threat taken seriously by leading figures in the field.
You say, the tech-capitalists „are moving as fast as they can to build powerful systems that no one understands, and that will ultimately pose an existential threat to humanity, possibly quite soon. That is not some kind of illusion that can be dismissed with the wave of a hand, as Hao does, it is a threat taken seriously by leading figures in the field.“
I think here you run the risk of conceding ground to those „leading figures in the field“ whose irrationalist ideas are criticized in the article. While I can agree that the consequences of the current breakthroughs in A.I. technology are under-appreciated, it would be wrong to portray the intelligent machine as a sort of demi-god towering above the contradictions of private property and the nation-state system, „posing an existential threat to all of humanity“. Insofar as an existential threat is posed, it is because the capitalists are hellbent on posing it. It is they who inscribe their ideology into the machine and thus conjur up the spectres of „AI-triggered world war“ or „AI-organized genocide“. In this sense, I do think one must oppose the notion that technology is „neutral“ or „value-free“. The machine that acts on command of the class enemy has always been a terrible foe. But technology will never escape the contradictions that have produced it. The real gains slumbering within each technological achievement will be fully realized only to the degree that the working class takes hold of it.
@Anonymous By leading figures, I didnt have in mind the Sam Altmans or Elon Musks — who could rightly be described as “irrationalist” and even deranged — but rather the serious scientists, people like Geoffrey Hinton, Yoshua Bengio, Richard Sutton, etc. “But technology will never escape the contradictions that have produced it.” Are you so sure? What materialist argument persuades you that a machine intelligence is impossible (or at least something for the far future), or that AI can only ever be a tool, without its own goals? One must base themselves on a careful study of the science. Reinforcement learning provides the concepts for understanding agentic or goal-driven machines. How one gets from the current AI to “general intelligence” is, of course, a mystery, but the basic outlines can begin to be made out.
Thank you for your comment Peter.
In terms of the stock market and venture capitalist firm investment, I think it is fair to say that AI company valuations are deliriously overblown - at least in the short term. Much as was the case with the Dotcom bubble, it can be simultaneously true that the underlying technology has value but that value is drastically overestimated.
In my view, concepts such as AGI and super-intelligence have two issues. The first is no one has really come up with a way of comparing human capabilities and AI capabilities (this is even true of most narrow jobs tasks let alone intelligence in general). The second is that has any AI system ever operated without a human-in-the-loop? Even the most sophisticated LLMs still take instructions from a human and are walked through an interaction by a human interlocutor. From an economic viewpoint it is valid and interesting to compare human vs AI + human, but from the more abstract and ambitious notions of AGI and Superintelligence is there any test we can do with current AI that is actually an unambiguous comparison of AI and humans (where the former does not have a human in the cockpit)? This is why I think it is fair to be skeptical of the label of ‘intelligence’ at all for things like ChatGPT, they are really sophisticated tools that are always operated by a human- otherwise we would have to label other tool inventions that improve human output such as writing or musical instruments as "intelligent" as well. These greatly improved the capabilities of the human user of these tools, but they were inoperable independent of users.
Your point about the future potential of deep learning techniques seems to me to be an empirical question. My impression is that while the scaling law has led to drastic increases in AI capabilities there are 1) continued hard stops inherent in deep learning it can’t seem to get over, most prominently hallucinations, and 2) scaling laws reflect performance on NLP task benchmarks such as MMLU and do not reflect progress toward ‘intelligence’ in general.
1) can be partially overcome through quality assurances in training data, or the integration of symbolic systems or source-databases but in longer term tasks where chains of logical inference are required this continues to be a problem for AI systems.
For 2) it is important to note the relationship between the development of Natural Language Processing techniques and benchmarks to evaluate them. While increases in performance have been huge in the last 5 years, this performance has been limited NLP evaluations. In other words, AI developers are developing models to achieve higher performance on relatively narrow benchmarks and then proclaiming progress toward intelligence in general on the basis of their success. NLP tasks are a huge part of the modern service based and highly digitized economy and will be massively impacted by AI, but this has no more baring on progress toward ‘human’ intelligence in general than the calculator did when it automated arithmetic.
(1/2)
Some 'emergent' capabilities are certainly compelling and there is a debate ongoing as to why they arise. Do they come for free with a machine that can mimic human language? Or is it the case that LLMs training data is so large that many examples of the task are actually included in its training data?
For example, a compelling case of ‘emergent’ capabilities is LLM’s capacity to translate texts between two languages. LLMs mostly outcompete older deep learning models such as Google’s BERT on translation benchmarks. Putting the technical details to one side, BERT is trained explicitly to translate between two languages, whereas LLMs are trained to predict the next token (or word). At first glance, LLMs ability to translate between two texts seems nearly magical. However, when we consider that the internet has a huge number of pages of identical text in various different languages it becomes harder to maintain the claim that LLMs were not trained in someway to translate. It can be argued that while it was not the intention of the AI developers to create a translation model, they did so by including so much translation data in the training date. This is not to say this necessarily explains all ‘emergent’ capabilities in LLMs, but it is an important detail which suggests that just scaling up data and training time over and over will not necessarily lead to more and more ‘emergent’ abilities, which is often implicitly or explicitly claimed by AI CEOs and researchers.
Similarly to point 2 above, even the most sophisticated ‘emergent’ abilities are going to be limited to digitalised NLP tasks. In many cases LLMs already perform these tasks at a superhuman level, but again this doesn’t mean they are progressing toward human-level intelligence in general.
One more point on deep learning techniques and the analogy to the human mind. You are right that in principle ‘the human brain is material’ and ‘its basic laws can be discovered and reproduced’ but the question is have they been reproduced? Or is there evidence we are close? Granted, it is true that for decades symbolist approaches to AI dominated the field. In philosophy the corollary of this approach, which I take to be language of thought has always had a bit of whiff of idealism about them in my view. Nevertheless, the successful return to connectionist approaches does not necessarily mean the field has returned to a more materialist outlook and broken the back of the problem of replicating human intelligence. While deep learning techniques are analogous to the neural structure of the brain at the most general level, beyond that the analogy breaks down pretty quickly. In the human brain there are multiple types of neuron, a variety of different electro-chemical processes, diverse connections. Similarly, techniques like transformers or back-propagation which have been crucial in recent developments are unrelated to known processes in the brain (although our limited understanding of the latter makes any adjudication on this issue complex).
I believe it is possible to warn about the significant economic, military and surveillance aspects of AI while at the same time pushing back against claims to ‘superintelligence’ or ‘AGI’ championed by tech CEOs and some AI researchers. I think the main point I make in response to Hao’s book also applies here: the direction of travel behind AI and the public discourse surrounding it has much more to do with the peculiar features of the capitalist economy in the age of global tech monopolies that it arose from than it does about a new age of history where machines are more intelligent than, and therefore an existential threat to, human-beings.
Interested to hear more about what you think about these responses as you raise important points.
(2/2)
This was responding to your initial comment Peter, but I think what I said in the response is related the point you make about 'general outlines' of a path toward human intelligence as well. It depends, how general your use of 'general' is I suppose. It's clear that both connectionist and symbolist approaches do simulate something like what's going on in the human brain at some very abstract level- but it isn't clear that refining such a simulation is a clear route to human-like intelligence.
Hi Sam. I find most of what you say here reasonable, but I do disagree with much of it.
I agree on the existence of a stock market bubble.
I don’t think one needs to have a perfect way of comparing capabilities or a perfect definition of intelligence for the concepts of AGI and super-intelligence to be useful. Human intelligence possesses a generality that is missing in other animals and AI, and that is precisely why the latter are still limited to being tools. The notion of AGI is important because it emphasizes this difference and that it represents a critical threshold. The notion of super-intelligence is important because it emphasizes that human intelligence is not the absolute limit of what is achievable in nature. Whether we want to call existing AI intelligent or not is probably irrelevant. The question, as you note, is whether it truly represents a step toward general intelligence, however that is to be defined (more on this later).
I have to disagree with you on scaling laws. The nature and limitation of scaling laws is not so mysterious when we understand what LLMs are actually doing. An LLM is nothing more than a prediction machine. It builds a model of whatever slice of the world it is trained on, which can be language, or it can be vision, robotics, etc. It is passive prediction, i.e. not based on experience or interaction (that is the domain of reinforcement learning). The fact that LLMs do better as you scale them up is just a consequence of a larger/richer model of the world being more capable than a smaller/less detailed one. To the extent that something more than modeling the world is necessary for intelligence, the scaling laws fail to deliver. However, since learning to model the world is a component part of an intelligent system, any future AI will exhibit similar kinds of scaling laws.
Emergent capabilities are also maybe not so surprising, because it actually is quite natural that a larger model can do things that a smaller model can’t, and that the transition from one regime to another can be sharp, i.e. a sort of punctuated equilibrium. I don’t think this has anything to do with language in particular, but is a more general phenomenon. Do such abilities come from memorizing the training data? That seems likely, but would only indicate the limitations of the current methods and not neural networks in general.
“Hallucination” is an unavoidable property of the way LLMs are trained. One trains them to be prediction machines, and then, in the crudest way imaginable, fine tunes them with a bit of reinforcement learning. This isn’t a fundamental limitation of neural networks. Human-generated data, on the other hand, is a limitation only so long as AIs are not primarily learning from their own experiences.
(1/2)
I don’t know, of course, how close we are to reproducing the “laws of motion” of intelligence, but I do think we can be confident that deep learning makes use of *some* of these principles. These are very simple ideas: represent everything, whether language or anything else, as matter in motion in a high-dimensional space, build a model of the world by stacking simpler models, etc. All of it is done in a simple and pretty much unavoidable way, a much more powerful way than symbol manipulation. The brain must be doing something analogous, and the great variety of biology, all the various types of neurons, etc are probably not essential.
Again, one thing to notice about these systems is how crude they are. We do the most obvious thing, yet they still end up quite capable. That is a strong indicator that much can be achieved with relatively simple architectures, and that there is a lot of room for improvement. These are early days.
What is missing from AI? Better generalization, long term/ episodic memory, the ability to continually learn, more powerful forms of recurrence, advances in reinforcement learning, etc. None of these issues seem particularly insurmountable. Given the clearly vast potential of deep learning to build arbitrary levels of complexity into a brain via a process of self-organization, i.e. “learning,” it is easy to imagine that one or two important insights applied at scale would probably give rise to a qualitative leap, and potentially very rich behavior.
So while I agree that it is vital to talk about all of the near-term implications, and to counter the techno-fascists that are steering the ship, it looks to me like there is no fundamental obstacle between here and AGI, and as little as a few major scientific insights could be all that is needed. A problem I have with some of the discourse on the left is the idea that, because we don’t really understand where this is going and everything is shrouded in uncertainty, we should downplay the possibility of AGI as hype that plays into the hands of the capitalists. I say: the total lack of understanding is very concerning! Anyone who makes a positive claim that AGI is far off must understand that the burden of proof is now fully on them.
(2/2)
To make it easier to follow I am going to respond to some of the points you make Peter by quoting them and responding underneath.
“I don’t think one needs to have a perfect way of comparing capabilities or a perfect definition of intelligence for the concepts of AGI and super-intelligence to be useful. Human intelligence possesses a generality that is missing in other animals and AI, and that is precisely why the latter are still limited to being tools. The notion of AGI is important because it emphasizes this difference and that it represents a critical threshold.”
--
I agree that ‘generality’ is an important component of the differences between human cognition and AI- however, it isn’t the only one, for example, agency and some of the other aspects of human cognition like episodic memory etc. many of which you list in your previous comment. Nonetheless, I think it is still a dangerous term because of its ambiguity and because it raises associations with sci-fi tropes like Hal from Space Odessey, Ava from Deus Ex-Machina and all the way back to Shelley’s Frankenstein’s monster.
I do think the comparison to human intelligence is crucial for grounding the discussion away from such tropes. For instance Musk describes AGI as an “AI that is smarter than the smartest human” and predicts it will arrive in 2026. But ‘smarter’ in what sense? He seems to mean that a human using Grok can outperform a professor on some (any?) test. Okay, but in that case AGI has been around for a very long time… armed with a calculator a highschooler using a calculator could outperform the world’s leading mathematician at arithmetic, a human using Google Search is massively more capable than someone going through library stacks or newspaper ads manually. So whatever the definition of AGI is, we should strive to make it consistent and clear.
I think this tension around AGI definition has been around since the beginning of AI as a field. Different researchers seem to have had different definitions of AI. Marvin Minsky, for example, described AI as “the science of making machines do things that would require intelligence if done by men.” This is subtly yet crucially different from John McCarthy’s definition of AI as “the science and engineering of making intelligent machines.” Whether or not McCarthy and Minsky thought this distinction was significant they convey a completely different vision about the goals of AI research and whether one would even seek to achieve AGI. On the first definition, machines that achieve AGI in numerous domains are completely uncontroversial, like the calculator or Google Search examples above. However, on the second definition any AI system, whether better than humans or not is ‘intelligent’ in the human sense. My sense is that Minsky’s definition is the right one and leads one to conclude that talk of AGI is unhelpful – why would we want to recreate a general human like intelligence when we can create a host of machines that perform narrow tasks at a much greater speed and scales than humans. AI development is and has always been about creating tools that reduce the amount of labour required to perform a given task, not about recreating full or partial intelligence and claims about the latter lead to speculation and scenario-building that is dislocated from the actual state of the technology. In this sense, AI isn’t so different from the general trend toward greater and greater mechanisation in economic production that has typified the history of capitalist development. 1/?
“The nature and limitation of scaling laws is not so mysterious when we understand what LLMs are actually doing. An LLM is nothing more than a prediction machine. It builds a model of whatever slice of the world it is trained on, which can be language, or it can be vision, robotics, etc. It is passive prediction, i.e. not based on experience or interaction (that is the domain of reinforcement learning).”
--
You may, quite understandably, respond to the above that Grok outperforming a human on any test is a lot more general than the calculator or search example, and this is certainly true. As you say, given enough data in a particular domain neural AI systems can become highly performant at tasks within that domain. Training LLMs on the huge subsections of Internet makes it very effective (often times equal or better than expert humans) at tasks found on the Internet: writing, translating, programming etc. However, I would argue that being trained on the internet is still massively narrower than the rich day-to-day experience humans (and animals) are continuously exposed to. It isn’t massively clear that this is easy to scale up, particularly across multiple modalities of human experience.
Computer Vision provides ample evidence for this. While in narrow and controlled domains it is very easy to get models to outperform humans, simple variations in light or signal interference (easily dealt with by humans) can be catastrophic. Like other neural systems applying them to test data outside of their training data leads to huge degradation in performance- the “Islands of Automation” section of this piece is particularly interesting for how this plays out in the practice of Radiology (https://www.worksinprogress.news/p/why-ai-isnt-replacing-radiologists). 2/?
"“Hallucination” is an unavoidable property of the way LLMs are trained. One trains them to be prediction machines, and then, in the crudest way imaginable, fine tunes them with a bit of reinforcement learning. This isn’t a fundamental limitation of neural networks. Human-generated data, on the other hand, is a limitation only so long as AIs are not primarily learning from their own experiences."
My understanding is that ‘hallucinations’ are fundamental (to neural models of any kind in isolation (I know I introduced the term in the last comment but I actually prefer the term confabulation because AI models are not deviating from some grounding world model). See: https://en.wikipedia.org/wiki/Confabulation_(neural_networks). Indeed, neural mechanisms in the human-brain presumably confabulate signals all the time but we are mostly able to correct these occurrences through our continuous interaction with the external world. You raise the centrality of this issue in your statement that “AI are not primarily learning from their own experiences.” On this point I agree.
However, how and whether it is even possible for AI to have human like experiences of their interaction of the world (or if some digitized proxy is ever capable of reaching the levels of fidelity of the input humans gather from perception) are completely open question. So far, AI is only capable of learning from data that is digitized and then recognizing patterns between the 1s and 0s we’ve assigned to images or words or sounds etc (I know this is a massive over-simplification but I am just trying to illustrate 1) that simulating the full extent of human perception and interaction with the world is much much more complex then training a gen AI in one big go on how ever many trillion words or images 2) there may be principled differences between human ‘analog’ perception and computations carried out over ‘digitized’ representations of words, images, audio). 3/?
"It looks to me like there is no fundamental obstacle between here and AGI, and as little as a few major scientific insights could be all that is needed. A problem I have with some of the discourse on the left is the idea that, because we don’t really understand where this is going and everything is shrouded in uncertainty, we should downplay the possibility of AGI as hype that plays into the hands of the capitalists. I say: the total lack of understanding is very concerning! Anyone who makes a positive claim that AGI is far off must understand that the burden of proof is now fully on them."
--
I disagree with the approach here. I actually think the burden of proof lies with those arguing there are no principled barriers to AGI (however that is defined) as to why things like episodic memory, some proxy of human like experience, continual learning etc. seem like they so obviously follow from next-word prediction based on neural nets trained at a massive scale. It should also be noted that models for how to divide up the various cognitive systems in the human brain and how they interact together are highly variable. I did some work on working memory and visual memory models a couple of years ago and there’s not really broad consensus on how to delimit it from long-term memory, short-term memory and iconic memory – and whether or it not it works in the same ways across various modalities of perception.
The correct position is not that these things are impossible, but they we shouldn’t be preparing for their immediate advent without pretty strong evidence backing that. What we should be preparing for is the economic impact on the huge number of people that involve repetitive tasks carried out on computer – which particularly in developed economies is a huge percent of overall employment (we should still be cautious here, studies on impacts of AI adoption aren’t as conclusive as the general narrative around AI would have you believe e.g., Becker et al.’s study of coding tools on productivity https://arxiv.org/abs/2507.09089). The economic/environmental impact of the huge investment in data centers. The fact that the technology as it exists could be massively beneficial across the whole economy but is warped to the interests and ideology of a tiny band of tech CEOs etc etc.
As for the more fundamental question of whether artificial intelligence in the strongest sense of the term is possible (where AI stops being a tool used by humans and becomes some kind of being in its own right), it seems to me that there is nothing to suggest modern methods are any closer to this now than they were 50 years ago or indeed at the onset of the industrial revolution. While it may be idealist to suggest that such an eventuality is a certain impossibility, it doesn’t mean the correct materialist analysis is to put a plus sign where idealists place a negative. The truth is we don’t really know how the human mind works or the peculiarities about how and when it evolved (theories like the ‘cognitive revolution’ happening 70,000 years ago with the overnight advent of language supported by Chomsky are pretty conjectural in my opinion). It may not be necessary to replicate the exact history of our own mind’s evolution to recreate human intelligence, but I’ve not yet seen a clear conception of any other route to that result. In the absence of such a road-map it isn’t something Marxists should be immediately concerned about in my opinion. 4/4
An additional comment just on your remarks abouts about warnings from experts in the field. Warnings from Hinton and Bengio are well-noted (and I actually think they raise concerns about the direction of AI research that are valid independently of the AGI question). However, it isn't clear at all that the majority of AI researchers believe AGI is imminent or the path to get there is obvious. This year's AAAI survey (report here: https://aaai.org/wp-content/uploads/2025/03/AAAI-2025-PresPanel-Report-FINAL.pdf) reported that 76% of 500 respondents (mostly academics) assert that “scaling up current AI approaches” to yield AGI is “unlikely” or
“very unlikely” to succeed." While I imagine many of them would be less skeptical than I am about the concept of AGI overall, it isn't the case there is a wide consensus among leading researchers that AGI is just around the corner.
Very interesting discussion. I agree with Sam and especially appreciate the following:
"The truth is we don’t really know how the human mind works or the peculiarities about how and when it evolved (theories like the ‘cognitive revolution’ happening 70,000 years ago with the overnight advent of language supported by Chomsky are pretty conjectural in my opinion)."
Chomsky's theories about a magical development 70K years ago are absurd since it's clear that language and intelligence developed gradually over some four million years of hominid development. Some sort of threshold was crossed with Homo Erectus, which appeared some two million years and apparently possessed some kind of proto-language. But while I'm a big fan of sci-fi series like "Humans" and "Real Humans" as well as "Ex Machina," the puzzle remains how tech can possibly replicate such a rich and lengthy biological process. Indeed, technology and biology seem in many ways to be opposite.
Post a Comment