Karen Hao’s Empire of AI: a liberal critique of capitalist technological progress

EMPIRE OF AI: Dreams and Nightmares in Sam Altman’s OpenAI

Karen Hao

Penguin Press, 2025

482 pages


Karen Hao’s Empire of AI is a deep dive into the rise of OpenAI, the organization that released what quickly became the world’s most popular Large Language Model (LLM) ChatGPT.[1] At center stage in Hao’s book is OpenAI CEO Sam Altman, the current darling of Silicon Valley. Her narrative revolves around efforts to oust Altman from OpenAI in November 2023 and his ability to brush off such attempts by turning what was once a non-profit research lab into a standard Silicon Valley tech corporation.

Empire of AI paints a detailed yet engaging portrait of the AI industry, one that shows how a combination of scientific talent, ideological zeal, ruthless business and an abundance of good fortune helped the company morph into the innovation poster boy it is today. It includes interviews with workers in historically oppressed countries who perform much of the critical work that enables AI to function. Its coverage of the environmental impact of AI and the efforts of major firms like Google to cover-up environmental abuses is valuable.[2]

It also features much material about Altman himself. Hao recounts many junctures in the evolution of OpenAI in which he used personal charm to seduce scientists and investors before an unexpected decision left them feeling used and betrayed. Empire of AI describes the many falling-outs Altman has had with ex-OpenAI employees and investors such as Elon Musk, OpenAI’s ex-lead scientist Ilya Sutskeker, and Dario Amodei, who would later found rival AI firm Anthropic.

Hao lays out the web of relationships that enabled Altman in the first place. Most notable was Peter Thiel, who has acted as a mentor for over a decade, imparting a good deal of his techno-libertarian ideology in the process. She argues convincingly that the elitist ideology of a handful of Silicon Valley billionaires is guiding AI development much more so than scientific or social concerns. Hao recounts how politicians in Congress are eagerly offering support in the form of massive infrastructure scaling, deregulation and public funding despite warnings from a multitude of academic researchers.

One of the book’s strengths is its clear and concise explanation of technical controversies among AI researchers and how they relate to speculation about AI’s capabilities. Hao does a good job of dismantling utopian claims that AI systems will soon outpace human intelligence in every domain.

All tech products are the result of shared scientific knowledge of all humanity as well as, in many cases, direct government funding. Invariably, after development they pass into the hands of a narrow slice of billionaires. Hao shows how AI is the culmination of this ongoing privatization process. She explains how generative AI systems like OpenAI’s ChatGPT are trained on effectively huge amounts of stolen data, whether it be proprietary computer code, e-books or academic articles.

Whereas a heist of this magnitude would lead to criminal consequences for you or me, AI companies enjoy the backing of powerful world governments. Although Hao doesn’t mention it, it cannot help but recall the tragic case of Aaron Swartz who killed himself on the eve of sentencing as he faced up to 35 years in jail. His crime was pirating academic articles from the JSTOR database - small change compared to OpenAI’s open looting.

Overall, anyone seeking an even-handed and up-to-date treatment of the industry. But it is not without certain weaknesses

The role of AI in the economy, and the amazing hype that has accompanied it at every step of the way, are not discussed. There is also little treatment of perhaps the most plainly insidious aspect of AI development: its use in a military context and for mass surveillance.

But perhaps the most significant weakness, politically speaking, is the book’s central argument that some features AI industry resemble classic 19th-century imperialism. This ascribes a role for AI and big tech that is utterly exceptional in world-historical terms. As is often the case with radical liberal criticism, the effect is to conflate symptoms with the underlying disease and thus let modern imperialism off the hook.

This may seem like nit picking of an otherwise strong book. But they are essential for a Marxist understanding of AI and its politico-economic role.

 

What is AI

First, a quick aside on AI and how it became such a source of excitement and controversy. Its definition is often slippery, but it generally refers to computer algorithms capable of simulating human cognitive abilities. Artificial Intelligence was first coined in 1956, its fortunes rising and falling in line with the ups and downs of technological development in the ensuing decades.

Older techniques from the 2000s and 2010s weren’t called AI at the time but probably should be now. For example, Google’s search algorithm or the knowledge graphs behind the logistics network of companies like Amazon have had a massive impact, but aren’t usually thought of as AI. primarily for economic and cultural reasons rather than scientific. These technologies matured before the “thawing” of the last AI winter around about 2012. It should be noted that there have been already been two “AI winters” since 1956, both setting in because AI had failed to come good on short term promises of financial returns, leading investors to jump ship. These cycles, highly disruptive in terms of scientific research, are prime examples of how damaging the anarchy of production and research can be in an ultra-financialized capitalist world system, even on its supposed home turf of innovation.

In the 2020s, the new rage is generative AI techniques (Gen AI), the tech behind ChatGPT and other chatbots. These learn from vast troves of data and then generate outputs typically in text, image or audio form. Despite the renaissance of terminology like AI and Artificial General Intelligence (AGI) – more on that later – Gen AI is just the latest link in a long chain of developments in digital technologies in the last few decades.

Gen AI is based on neural nets with architectures loosely – much looser than is often admitted – inspired by the neural architecture of the human brain. While neural methods had been popular in the 1980s, they were too computationally expensive for the hardware at the time while data was also hard to come by. While there were some genuine algorithmic advances in the 2010s that laid the framework for modern models, they have primarily been made possible by massive improvements in computer chip technology and the availability of previously unimaginable amounts of data on the Internet.

LLMs have been successful in capturing the public’s imagination with their capacity for amusing and even shocking interactions with human users. LLMs comfortably pass the Turing Test [3], once regarded as the premier test of whether a machine can match or exceed human intelligence. However, Turing’s test is based on an overly-linguistic view of human intelligence, and passing it does not show AI is effective at reasoning or interacting with the physical world.

"Stochastic Parrots at work"
IceMing & Digit / https://betterimagesofai.org / 

Tech leaders cite Gen AI’s supposed world-shattering novelty and insist that, unlike previous tech innovations and the burst bubbles they precipitated, Gen AI won’t fail to deliver on its promises. The uncritical amplification of these claims has become a modus operandi for tech journalists. Willingly or not, they all seem to have forgotten the precedents for the current hype around AI, not just within the tech world but within the narrower AI field itself. This is not to say there is no novelty with current techniques, but that their capacity to change the world beyond recognition is almost deliriously overblown, leading to unreal stock market valuations and seemingly endless speculation about AI’s impact.

Whether or not AI systems will ever exhibit “true” intelligence, there is little question on a practical level concerning their ability to perform some of the tasks we are trained to do at school or university or in the workplace. However overhyped, there is no doubt that Gen AI tech does have significant economic value. As with earlier episodes like the dotcom bubble of 2000, genuine value was involved, even if not quite as much as tech CEOs and investment bankers believed.

 

AI and the Economy

Empire of AI closes with OpenAI’s announcement of its transformation into a for-profit corporation in late December 2024 – apparently timed for the holiday period to reduce public scrutiny. Its press release declared:

We once again need to raise more capital than we’d imagined. The world is moving to build out a new infrastructure of energy, land use, chips, datacenters, data, AI models, and AI systems for the 21st century economy. We seek to evolve in order to take the next step in our mission, helping to build the AGI economy and ensuring it benefits humanity.[4]

But what does an AGI economy mean? Elsewhere, Hao explains how Artificial General Intelligence (AGI) is a notoriously contentious concept[5]. Most of the time, it refers to an AI that is smarter or more capable than a human being. This isn’t very enlightening since we don’t have reliable metrics for human intelligence and those we do have are not suitable for measuring AI (IQ tests and exams rest of a series of assumptions about human capabilities that do not carry over to AI.)

If AGI means doing a human job at many times the speed and scale, then it could be argued we’ve had super-intelligent AI systems for a long time (The efficiency and scale of Google Search is incomparable to someone trying to find a document by hand in a database.) But if it means replacing human beings by being better at absolutely everything they do, then it is clearly nowhere near fruition. Generating text, images, and computer code may one day replace a computer programmer or an academic researcher (although human oversight will still be required), but it isn’t going to clean a sewer or cook a meal anytime soon.

That AGI is ill-defined does not stop tech CEOs from touting its imminence to the public and lawmakers alike. To prevent China from getting there first, we must deregulate tech and massively fund AGI development – or so the argument goes.

That’s the stick. The carrot is that AGI is going to radically transform the economy. The argument is that AI advancements will make labor so productive that we will no longer have enough jobs and the vast majority of people will have to find other things to find meaning in life - even if it is currently impossible to find meaning in anything other than work.

Such claims are not unprecedented. In 1930, as the world reeled from the Wall Street Crash, John Maynard Keynes wrote an essay titled “Economic Possibilities for Our Grandchildren” [6] that predicted that, within two generations, the average person would work a 15-hour-week. Even during the Industrial Revolution figures such as John Stuart Mill and David Ricardo warned of mass unemployment as a result of technological innovation. Yet the job base since the 19th century has grown many times over.

Today, the great debate in the AI commentariat is whether AI is so powerful that it will replace most if not all cognitive tasks. Unlike previous technological innovations this would primarily impact white-collar jobs which require literacy skills or specialized knowledge such as software developers.

The broad answer is, yes, AI is that powerful. The ability of LLMs to generate computer code, visuals, audio and text in seconds with small amounts of human input means it can massively increase productivity in jobs which produce content of this kind. Opponents of AI will, rightly, charge that this will lead to a major reduction in the quality and significant safety concerns.[7] However, for better or worse, this did not stop lower quality mass production superseding artisans during the Industrial Revolution.

While it is clear AI will impact some jobs, accurately measuring just how many is much more difficult. Much current hype is based on anecdotal evidence about major staff reductions. But such claims are hard to judge other than a few areas in which the impacts are exceptionally well documented (e.g., the mass replacement of customer service handlers with AI voice programs). Otherwise, it’s entirely possible that employers are using AI to justify layoffs that were already in the works and that other workers are being forced to pick up the slack. Rather than innovation, the result is little more than an old-fashioned speed-up.

AI researchers have struggled to come up with more objective measures. Reports from Anthropic [8] and Microsoft [9] base their estimates on the queries users submit to current chatbots. The Microsoft study concludes that jobs like sales representatives and customer service representatives (which employ 1.142 million and 2.859 million people respectively) are directly in the line of fire.

One of the most successful areas of AI application seems to be in software development. AI systems like Anthropic’s Claude 4.1 Opus and the recently released GPT5 are capable of coding entire applications or websites based on natural language descriptions by a user. However, recent evidence [10] suggests that, for more experienced coders, the use of such tools reduces productivity. While the coding ability of these models is impressive for users with little or no coding experience, they can quickly become a hindrance when applied to more specialized problems or are large code banks.

It should also be acknowledged that AI’s ability to replace a job doesn’t mean it will since such replacement requires knowledge and capital investment. In other words, exposure doesn’t always equal replacement. Economists are often more skeptical. For instance, Nobel Prize winning economist Daron Acemoglu predicted in 2024 that total productivity gains from AI will not exceed 0.71 percent across next 10 years [11], a forecast based, moreover, on rather generous estimates of AI’s capabilities produced by OpenAI researchers in 2023.[12] He expects the impact on jobs to be modest overall, all things being equal.

 

AI and the continued rise of the stock market

A key component of the supposed drive toward Altman’s “AGI economy” is the “scaling hypothesis,” which has led to a phenomenon known as a “hyperscaling” business strategy. The argument is that all we need to reach AGI is more data, more processing power, and hence more government-backed data centers across the globe.

This has led to speculation about “emergent” AI capabilities, the idea that once a certain quantity of processing takes place, capabilities like reasoning arise without the model being explicitly trained to develop them.

This is a case where the supposedly scientific explanation of how to develop AI dovetails with major corporate efforts to gain a monopoly positions over rivals. While early LLMs improved drastically as their data base expanded, there is little evidence that more and more data will lead to substantially better models beyond a certain point. Indeed, a plateau seems to have been reached in mid-2024. Since then, much of the progress has come from improvements in model architecture or data quality (e.g., the Chinese based DeepSeek’s V3 R1 model that shocked world markets earlier this year as well as OpenAI’s recently released GPT5 model).

Nevertheless, the unquestioned belief in this hypothesis is what is driving AI companies to build growing numbers of bigger and bigger data centers in order to create bigger and bigger models. The upshot is a “hyperscaling” arms race. This is leading to the construction of gargantuan data centers across the US, polluting major areas and drying out water supplies.[13] Hao’s own reporting on data centers in northern Chile and Uruguay shows the devastation that this bigger-and-better philosophy has caused.

The scramble for data centers is the primary driver of huge valuations of chip companies NVIDIA (now the world’s most valuable company with a market cap of $4.28 trillion) and AMD- who provide the chips used to construct the giant computers housed in these buildings. Between them, they have a near-monopoly on the graphic processing units (GPUs) needed to train and run large AI models.

Traditional big-time players like Meta, Google, Microsoft, and Apple are competing to integrate AI into their products. The aim is to leverage AI systems throughout the economy to improve productivity and reduce labor costs. This promise has become a critical prop for world capitalism in the past three years. According to DataTrek, without the so-called Magnificent Seven stocks comprising the major tech companies and NVIDIA, the S&P 500 would scarcely have grown across 2023 and 2024 (it actually increased in value by 24.2 and 23.3 percent respectively across those years).[14] According to Paul Kedrosy [15], half of the US’s economic growth in the last year came from data-center construction.

This has helped drive the stock market to new heights without a clear route to profitability. AI research labs like Anthropic and OpenAI have burnt through cash since the release of their LLMs, with many commentators speculating the models actually cost far more to run on current hardware than revenue from subscription services. Yes, these costs are likely to significantly decline, but OpenAI’s revenue is still fairly limited compared to ad revenue brought in by Google or Facebook.

This didn’t stop OpenAI from valuing itself at $500 billion in a recent initial public offering Its aspirations for a future monopoly on advanced AI products seem unlikely due to the speed with which other industry players are able to catch up to or even leap ahead. OpenAI’s revenue is estimated to be at most $5billion and it continues to make gargantuan loses on all of its service offerings. In a bombastic but informative summary of the crazy numbers involved in the AI industry Ed Zitron argues,

“What is missing is any real value generation. Again, I tell you, put aside any feelings you may have about generative AI itself, and focus on the actual economic results of this bubble. How much revenue is there? Why is there no profit? Why are there no exits? Why does big tech, which has sunk hundreds of billions of dollars into generative AI, not talk about the revenues they’re making? Why, for three years straight, have we been asked to “just wait and see,” and for how long are we going to have to wait to see it?”[16]

Microsoft, Apple and Google’s investment in data centers and AI research seems to be more about keeping existing users in platforms like Microsoft Office, iOS or the Google Suite rather than creating brand new products. Similarly, Musk’s massive investment in AI is aimed at attracting users to X, with the hope it can be transformed into an everything app analogous to WeChat in China. Whoever the eventual winner(s) are in this race, it is not clear how this will boost profits drastically - except for eventual ability to jack up prices for consumers for the same services that already exist.

One striking rationale for tech stocks continued rise comes from none other than Peter Thiel, who recently conceded that AI is viewed as a last chance saloon before markets face the reality of general economic stagnation. As he told the New York Times Interesting Things podcast, “The fact that we’re only talking about A.I. — I feel that is always an implicit acknowledgment that but for A.I., we are in almost total stagnation.” And the economic data supports this point of view as well.

Following the disappointing release of GPT5 there are some signs the AI bubble may be about to burst. Bloomberg asked “is the AI winter finally upon us” [17] and the Washington post wondered if the AI Industry was “off course”.[18] Perhaps most ominously, however, on August 18 Altman himself declared AI was in a bubble, “Are we in a phase where investors as a whole are overexcited about AI? My opinion is yes.”[19] Clouds are certainly forming.

 

Should workers be opposed to AI?

Even though precise answers about AI’s ability to automate jobs are hard to come by, we do not need precise measures, luckily, to sketch out important political questions concerning potential job displacement.

AI-fueled job displacement is first and foremost a political question. Similar to outsourcing production to low-wage economies or investment in machinery during the Industrial Revolution, using AI to increase productivity and lay off workers is a way for companies to increase their profits. As in the SAG-AFTRA strike last year, workers in many industries will face a choice between opposing AI or losing their livelihood.

In the abstract, workers should be in favor of all productivity-raising technologies. Even before AI, Keynes was right to think that his grandchild’s 15-hour work week would be technologically feasible. AI’s ability to replicate a number of cognitive tasks only makes the irrationality of having people work endless hours more obvious. With the correct utilization of AI and other recent innovations, our collective work and private lives could look very different indeed, starting, not least, with being a good deal shorter. In an ideal world, workers fighting layoffs would call less for an end of AI and more for the use and development of AI to be in their own hands.

Unfortunately, we do not live in an ideal world, and will undoubtedly be used in the near-term to assault jobs and working conditions while enhancing the ability of managers to spy on workers.

One critical factor is that even a limited introduction of AI tools will prepare the ground for further replacement. The more workers use these tools, the more data they provide on how to do their job, which is exactly what is needed to train newer models to replace an ever larger share of the tasks they perform. AI agents that take over computer desktops to perform multi-stage tasks are trained on data recorded from office workers – although so far they have proven to be less successful than initially hoped.

"Wheel of Progress"
Leo Lau & Digit / https://betterimagesofai.org

A more successful example of this has already come through driver less cars, which for years have been trained on data taken recorded from millions of human drivers. Now the driver less taxi service Waymo is growing rapidly in San Francisco, overtaking Lyft’s market share in the city in recent months. Of course, an efficient public transport system would render most taxi services superfluous without any AI magic, but that is a discussion for a different day.

In upcoming struggles, this may mean opposing AI’s introduction to a workplace completely for a time as a way to defend livelihoods. This should go along with a clear message that the issue is not AI, but corporate use of it to attack workers. Those opposing AI will undoubtedly be called Luddites and enemies of progress – a worn out cliche raised by bourgeois media outlets whenever workers oppose job cuts or wage freezes imposed in the name of technological innovation. However, they ought not to be deterred, and Marxists should not cede ground on this question.

It should also be kept in mind that whether AI really can replace a job or not will often be irrelevant to layoffs across many industries. As happened in the jobs massacre in the American auto industry in the aftermath of the 2007 financial crash, when automation was used as an excuse to cut tens of thousands of jobs and restructure the leading US auto producers from top to bottom, AI will undoubtedly be used in the context of a new crisis as a convenient justification for layoffs regardless of its impact on productivity.

Although AI is an impressive technology with many potentially uses, it is a technology massively shaped by the ideology of the ruling elite, in particular the emerging fascist-libertarian culture of Silicon Valley tech billionaires. Marx argued that technological developments reflect the interests of capital, and AI is no exception.

But the methods behind AI could be leveraged for a huge expansion of economic productivity the world over. The underpinning of modern AI innovations are statistical methods of almost unimaginable scale. At the moment, these are leveraged primarily to recommend products on Amazon, determine social media feeds, and, in the case of LLMs, predict the next word (or pixel). At the moment these algorithms are leveraged primarily for profit (or in the case of LLMs some vague imagined future profitability). Such methods – and the hardware also monopolized by big tech - could clearly be leveraged to make possible economic planning and productive efficiency such as is required to overcome global warming and poverty.

Even if AI never reaches the vague yet dizzy heights promised by tech billionaires, in the long run it has the potential to massively increase productivity across society. However, the benefits of this will only ever be reserved for a narrow elite unless the technology and its future development is put in the hands of those who develop and use it: the working class.

 

AI’s military and surveillance uses

Another significant issue that is not covered in Hao’s book is the use of AI for military and surveillance purposes. Again, it could be argued convincingly that this is outside the purview of Hao’s book. However, it is a critical aspect of the AI industry and throws light on the industry’s increasing links to the state.

Upon Altman’s reinstatement to OpenAI after a failed attempt to oust him in 2023, a new board of directors including ex-Secretary of the Treasury Larry Summers and retired US Army General Paul M. Seasoned, solidified OpenAI’s government ties. Under the Biden administration, and now increasingly under Trump, AI has been identified as a top national security priority. The big players in the industry, despite their previous association with Democratic politics, have continued to enjoy preferential treatment under Trump.

In the days after his inauguration – in which groveling tech leaders famously lined up to pay homage to the new president – Trump hosted Altman at the White House to announce the $500-billion Stargate initiative. By dollar value, this is the largest infrastructure project in US history. Over the next four years, the project aims to massively expand the US data processing capabilities by building a huge wave of data centers. Since Hao’s book has been published, OpenAI, Anthropic, Meta, and X have each signed $200-million dollar contracts with the US government for their services.

While the advantages of such applications are yet to be seen, what existing AI systems have shown is the deadly result of further distancing the combatant from the consequences of his or her actions. This has been seen more tragically in Israel’s bombardment of Palestinians in Gaza. A 2024 report by +972 magazine explains how the Israeli Defense Forces introduced a range of AI systems to generate bombing targets in Gaza.[20]

Using traditional target identification methods, Israel would have run out of targets to bomb within a few weeks of October 7. According to +972’s sources the new Lavender AI system analyzed surveillance data from Gaza and identified 37,000 new individuals as “junior Hamas officers” who then became targets for the Israeli bombs. Another AI system, grotesquely named “Where’s Daddy,” automatically inform human monitors when targets returned to their families in the evening so that the Israeli Air Force could launch a bombing raid.

It is clear that AI is a further development of the tendency of military technology to distance the combatant from the consequences of their actions. A 972+ source who has worked with Lavender told the magazine it reduces the human role in target selection to rubber stamping the system’s recommendations – offloading moral responsibility for the decision to target an official and their entire family from a human analyst to an algorithm.

Without understanding the particular architecture, it is of course difficult to know how targets are generated. However, with large AI systems based on neural nets, the factors that go into a recommendation are often a black box, even for the system designers, leading to a phenomenon known as “hallucinations.” This is where incorrect information is included in the output. Many factors completely unrelated to involvement with Hamas may well have played a role in target selection. Precision bombing has always been a fraudulent notion and introducing AI technology into target selection hardly reduces “collateral damage.”

The IDF’s AI systems are undoubtedly some of the most advanced in the world; indeed, their continuous surveillance of their own population and Palestinians in the West Bank and Gaza has provided a treasure trove of data with which to train such models. The development of equivalent AI systems in imperialist armies is undoubtedly underway the world over, in many cases inspired by “combat-tested” systems like Lavender.

Beyond generating new bombing targets, AI is seen as a critical tool by militaries for its capacity to process data and inform decision making in superhuman time. The promise is that the analysis and synthesis of battlefield data necessary for tactical decisions can be achieved in seconds, giving a massive speed advantage over conventional foes. The industry leader in developing these technologies is US-based firm Palantir, whose latest advert ran with the slogan, “battles are won before they begin.”[21]

Palantir’s stock valuation has increased 23 times since the start of 2023 was founded by Peter Thiel among others. This is no doubt due to the general hype around AI technologies and the massive injection of public money into rearmament. But it is also due to Palantir’s close relationship with the civilian and military arms of the­ Trump administration. It has signed a number of contracts with various sections of the US government in the last year including a $10-billion contract with the Pentagon on July 30. Its remit includes using generative AI and new drone technologies to develop cutting-edge military products.

Palantir is also on the leading edge of efforts to leverage AI to bolster surveillance inside imperialist countries. The firm has led the Trump administration's efforts to gut government agencies and increase executive power over the functions of the US government. One of the many controversies surrounding Elon Musk’s Department of Government Efficiency was its ability to gain access to critical government databases, including Social Security data and Treasury records, which means the data of every American has effectively been stolen. Palantir has been contracted by the Trump administration to combine these data sources into a single database on all Americans. Additionally, it is working with Immigration and Customs Enforcement “to track migrant movements in real-time.”[22]

Under the guise of efficiency, the US government and Palantir are effectively trying to produce surveillance technology that will be able to link individuals to their bank accounts and home addresses behind the backs of the courts and Congress, data that was previously separated within the government to avoid such concentrations of power.

Not only will this enable Trump to spy on and intimidate opposition within the US, but it is also a boon for Palantir as a private company. Essentially, it has been handed some of the most extensive data in the world to develop new AI systems for free. Under the auspices of AI development and efficiency the US government has thus stripped itself of some of its most valuable assets and placed them in the hands of Palantir.

"Surveillance View A"
Comuzi / https://betterimagesofai.org / Image by BBC

The company’s reach is not confined to the US. In the UK, where the company is headed by Lewis Mosley the grandson of late British fascist Oswald Mosley, Palantir won a £330-million contract to rework the National Health Service’s data infrastructure. Apparently, this system has been rejected by hospital staff across the country as unsuitable for the purposes. This has led the Labour government to bring in KPMG to somehow turn around staff attitudes at the cost of £8 million. Such are the “efficiencies” of privatization.[23]

The distinction between the “surveillance capitalism” industry, to use the term coined by Shoshana Zuboff [24], and government surveillance is diminishing. Far from regulating these industries, the state is handing the keys to data infrastructure that would cost millions on the private market

None of which poses radically new questions for Marxists. Rather, it poses old questions within a new framework. The use of AI on the battlefield threatens to massively increase the efficiency of an army’s capacity for death and destruction. Along with the massive rearmament being undertaken by imperialist powers internationally, the battlefield use of AI, in particular its use for target selection and “preventive strikes” must be strongly opposed.

The same goes for mass surveillance. The data people produce belongs to them by rights, yet has been misappropriated by private companies to maximize advertising profits. Its – often illegal – use by governments for surveillance has also been a reality for decades. AI may not fundamentally revolutionize how surveillance is carried out [25], but it does promise to enable spying at an even larger scale than was previously possible.

 

The AI Industry as an “Empire”

So far, we have discussed important aspects of the AI Industry that are excluded from Hao’s book. Now we will turn to an issue that is included. This is her central metaphor of AI as an empire. She explains:

Over the years, I’ve found only one metaphor that encapsulates the nature of what these AI power players are: empires. During the long era of European colonialism, empires seized and extracted resources that were not their own and exploited the labor of the people they subjugated to mine, cultivate, and refine those resources for the empires’ enrichment. They projected racist, dehumanizing ideas of their own superiority and modernity to justify- and even entice the conquered into accepting – the invasion of sovereignty, the theft, and the subjugation. They justified their quest for power by the need to compete with other empires: In an arms race, all bets are off. All this ultimately served to entrench each empire’s power and to drive its expansion and progress.[26]

She continues:

The empires of AI are not engaged in the same overt violence and brutality that marked this history. But they, too, seize and extract previous resources to feed their vision of artificial intelligence: the work of artists and writers; the data of countless individuals posting about their experiences and observations online; the land, energy and water required to house and run massive datacenters and supercomputers. So too do the new empires exploit the labor of people globally to clean, tabulate, and prepare that data for spinning into lucrative AI technologies. They project tantalizing ideas of modernity and posture aggressively about the need to defeat other empires to provide cover for, and to fuel, invasions of privacy, theft, and the cataclysmic automation of large swaths of meaningful economic opportunities.[27]

Hao’s book makes clear that OpenAI and other big tech players like Google, Microsoft, Meta and X all do a great deal of horrible things reminiscent of 19th-century empires.

"Frontiers of AI"
Hanna Barakat  / https://betterimagesofai.org/

The comparison isn’t wholly false, but it is superficial and misleading. The analogy reflects a long-standing tendency among left-liberal critiques to present the crimes of capitalism as peculiar to certain industries. Reports, books and films – often with extremely valuable material - on the horrific conditions for workers, damage to consumers, and environmental destruction in this or that branch of production abound. At various points in the last two decades, rare-earth metal mining, garment production, the arms industry, and even fish farming have come under attack as exceptionally evil industries.

However, critiques limited to evil industries or individuals, as repugnant as they may be, end up promoting illusions in capitalism’s capacity for reform. The argument, sometimes explicit and other times implied, is that the solution is not a radical change in the organization of society, but a few more regulations to rein in a given industry’s excesses.

This is how Hao concludes her introduction:

But the empires of AI won’t give up their power easily. The rest of us will need to wrest back control of this technology’s future. And we’re at a pivotal moment when that’s still possible. Just as empires of old eventually fell to more inclusive forms of governance, we, too, can shape the future of AI together. Policymakers can implement strong data privacy and transparency rules and update intellectual property protections to return people’s agency over their data and work. Human rights organizations can advance international labor norms and laws to give data labelers guaranteed wage minimums and humane working conditions as well as to shore up labor rights and guarantee access to dignified economic opportunities across all sectors and industries.[28]

In the conclusion to the book, titled “How the empire falls,” Hao expands on this view, proposing a three-pronged framework for “dissolving empire”[29] by undermining the industry’s monopoly knowledge, resources, and influence. Here she suggests a number of reasonable measures including funding to support independent evaluations of AI models and alternative approaches to AI development, forcing companies to disclose training data, stronger labor protections, and board-based education about how AI systems work as an “antidote to mysticism and mirage of AI hype.”[30]

Having laid out the growing strength of the AI industry and the subservience of elected representatives in the US, Kenya, Chile and Uruguay, these suggestions strike the reader as highly limited. Critically, they do not object to AI technology itself being held as private property- which again, Hao’s book exposes as an absurdity, given the technology’s reliance on data garnered from centuries of humanity’s shared labor stored on the Internet and algorithms developed by researchers over decades.  

Hao’s suggestions leave us with a “call your local representative” type of Democratic Party protest politics. The implication is essentially that “If only our politicians knew how horrible AI is” then they would undoubtedly act against it. However, as Hao makes clear, the AI industry has been the sweetheart of Democratic and Republican administrations alike and neither party is ever going to take on an industry that is propping up the US stock market.

That politicians will not even entertain the limited measures suggested by Hao is an indication that a much more radical approach is needed. Empire of AI makes clear is that the guiding lights of all of the decisions made in the production of ChatGPT and other companies’ models was how to minimize costs and scale as quickly as possible to try to gain monopoly position. Indeed, her analyses of the ideology driving individuals like Altman and Thiel show that this race for monopoly is their explicit aim. If this requires hyper-exploiting desperate workers in Kenya or Venezuelan immigrants and destroying access to fresh water across Latin America and the US, then so be it.

Hao’s detailed work ultimately leaves us with an account of the AI industry that is much less exceptional than the author realizes. While the industry may be among the most extreme examples of corporate greed and thievery, the general tendencies are of a piece with most major 21st century industries from food production to auto.

The question of how to eliminate the rampant exploitation in the AI industry, as with many others, is ultimately one of taking on the world capitalist system itself. AI, the hardware, and the algorithms behind it ought to be public utilities, run according to the needs and interests of the great mass of the world population. This will require a much more thoroughgoing transformation of society than a few policy tweaks.

Conclusion

The main aim of this review has been to critique the limited political conclusions and oversights of an otherwise very valuable work. Despite claims that AI is somehow completely novel, the political and social questions it poses are not fundamentally different from those raised in the face of previous technological developments from electricity, the production line, and the Internet.

That is not to say each one of these technologies does not have its own particular challenges and opportunities for revolutionaries. But in the most general terms, opposing mass job displacement, the extension of the bourgeois state further and further into private life and politics, and the threat of more deadly military systems have strong historical parallels.

What is clear is that appealing to policymakers and human rights organizations is not enough. AI is a modern globalized and highly exploitative capitalist industry. The empire Hao speaks of is not OpenAI, Google, or Facebook, but the entire apparatus of the modern imperialist states, with America far ahead of its rivals, that work to advance the interests of every major corporation, investment fund and bank above all else.

At the same time modern AI systems are technological development and a step further in the concentration of technology and data into the hands of major corporations and banks. As discussed above, they are not just based on clever algorithms but on all artifacts of human culture on the Internet, which have been developed by billions of people over the course of human history. They have only been made possible due to the uncountable labor of masses of working people, and it is they who should control and benefit from these systems.

 

 

 

 



[1] Technically, models like GPT4 or o3 are LLMs and ChatGPT is an extra layer on top that allows users to have conversational chats with the model, but most people refer to ChatGPT as a single AI tool.

[2] According to a Goldman Sachs report (https://www.goldmansachs.com/insights/articles/gs-research/generational-growth-ai-data-centers-and-the-coming-us-power-surge/report.pdf) from last year the data centers needed to develop and run AI systems, as well as tech’s other digital services, will account for 8 percent by 2030 compared to 3 percent in 2022. The report argues that generative AI is likely to be the key driver of this increase in demand, accounting for 200 TWh of power demand by 2030.

[5] In the aftermath of the disappointing release of OpenAI’s flagship GPT5 model, Altman has recently walked back his efforts to associate himself and the OpenAI brand with AGI, telling CNBC  last week “it’s not a super useful term.” It was only 8 months ago that Altman proclaimed in a blog post, “We are now confident we know how to build AGI as we have traditionally understood it.”

[13] A recent investigation by More Perfect Union (https://www.youtube.com/watch?v=3VJT2JeDCyw) on Elon Musk’s efforts to build the world’s biggest data centers in Memphis gives an insight into the devastating impact this infrastructure is having on communities.

[25] For example, the arrests of leaders of the Baader-Meinhof group, a left-wing terrorist group, in 1971 occurred after a special police unit used a computer to estimate where members most likely lived in Berlin and then did deep searches in those areas.

[26] Empire of AI, p.16

[27] Ibid p.17

[28] Ibid p.19

[29] Ibid p.419

[30] Ibid p.421



Print Friendly and PDF
Share:

17 comments:

Peter Ross said...

A good article. Agreed that the military implications are highly under-appreciated. The automation of genocide in Gaza is only the beginning - imperialism has caught on that drones & AI will change warfare, and they're now moving to rapidly develop and implement these technologies. The danger to humanity from this tech will exceed by far that posed by nuclear weapons.

I just want to comment on one part, which is your (and Hao's) claim that "their [AI's] capacity to change the world beyond recognition is almost deliriously overblown." You downplay the concepts of AGI and super-intelligence as ill-defined or unrealistic and liken the current hype to past eras. This is profoundly mistaken.

It is true that there is not much new in the techniques. The large language models of today are nothing more than giant stacks of machinery that was invented in the 50s and 60s. It is also true that there is hype about the short-term economic effects, which has led to a massive bubble. And finally, it’s true that the current techniques are not sufficient, by themselves, to generalize in the way that human intelligence does.

However, it is not true that “emergent” AI capabilities are mere “speculation” or that “scaling” is nothing but an “unquestioned belief.” Both are empirical facts and essential to understanding what’s going on. The process of learning can be thought of as a process of self-organization. A simple learning rule, applied many times over a brain with many neurons can lead to an incredible amount of complexity. Two things are true at once: AI is not doing everything that the human brain does, but it IS doing part of it. All you have to do is interact with a chatbot for a while to understand that these things are not just smoke and mirrors, there is something remarkable going on.

The human brain is material, and as such, its basic laws can be discovered and reproduced. Moreover, the human brain is not the final product of evolution; it can and will be surpassed. So while the immediate consequences of AI are over-hyped, the not-so-distance consequences are actually still under-hyped.

We (or rather completely unaccountable tech monopolies) are in the process of building machine intelligence. They are moving as fast as they can to build powerful systems that no one understands, and that will ultimately pose an existential threat to humanity, possibly quite soon. That is not some kind of illusion that can be dismissed with the wave of a hand, as Hao does, it is a threat taken seriously by leading figures in the field.

Anonymous said...

You say, the tech-capitalists „are moving as fast as they can to build powerful systems that no one understands, and that will ultimately pose an existential threat to humanity, possibly quite soon. That is not some kind of illusion that can be dismissed with the wave of a hand, as Hao does, it is a threat taken seriously by leading figures in the field.“

I think here you run the risk of conceding ground to those „leading figures in the field“ whose irrationalist ideas are criticized in the article. While I can agree that the consequences of the current breakthroughs in A.I. technology are under-appreciated, it would be wrong to portray the intelligent machine as a sort of demi-god towering above the contradictions of private property and the nation-state system, „posing an existential threat to all of humanity“. Insofar as an existential threat is posed, it is because the capitalists are hellbent on posing it. It is they who inscribe their ideology into the machine and thus conjur up the spectres of „AI-triggered world war“ or „AI-organized genocide“. In this sense, I do think one must oppose the notion that technology is „neutral“ or „value-free“. The machine that acts on command of the class enemy has always been a terrible foe. But technology will never escape the contradictions that have produced it. The real gains slumbering within each technological achievement will be fully realized only to the degree that the working class takes hold of it.

Peter Ross said...

@Anonymous By leading figures, I didnt have in mind the Sam Altmans or Elon Musks — who could rightly be described as “irrationalist” and even deranged — but rather the serious scientists, people like Geoffrey Hinton, Yoshua Bengio, Richard Sutton, etc. “But technology will never escape the contradictions that have produced it.” Are you so sure? What materialist argument persuades you that a machine intelligence is impossible (or at least something for the far future), or that AI can only ever be a tool, without its own goals? One must base themselves on a careful study of the science. Reinforcement learning provides the concepts for understanding agentic or goal-driven machines. How one gets from the current AI to “general intelligence” is, of course, a mystery, but the basic outlines can begin to be made out.

Sam Tissot said...

Thank you for your comment Peter.

In terms of the stock market and venture capitalist firm investment, I think it is fair to say that AI company valuations are deliriously overblown - at least in the short term. Much as was the case with the Dotcom bubble, it can be simultaneously true that the underlying technology has value but that value is drastically overestimated.

In my view, concepts such as AGI and super-intelligence have two issues. The first is no one has really come up with a way of comparing human capabilities and AI capabilities (this is even true of most narrow jobs tasks let alone intelligence in general). The second is that has any AI system ever operated without a human-in-the-loop? Even the most sophisticated LLMs still take instructions from a human and are walked through an interaction by a human interlocutor. From an economic viewpoint it is valid and interesting to compare human vs AI + human, but from the more abstract and ambitious notions of AGI and Superintelligence is there any test we can do with current AI that is actually an unambiguous comparison of AI and humans (where the former does not have a human in the cockpit)? This is why I think it is fair to be skeptical of the label of ‘intelligence’ at all for things like ChatGPT, they are really sophisticated tools that are always operated by a human- otherwise we would have to label other tool inventions that improve human output such as writing or musical instruments as "intelligent" as well. These greatly improved the capabilities of the human user of these tools, but they were inoperable independent of users.

Your point about the future potential of deep learning techniques seems to me to be an empirical question. My impression is that while the scaling law has led to drastic increases in AI capabilities there are 1) continued hard stops inherent in deep learning it can’t seem to get over, most prominently hallucinations, and 2) scaling laws reflect performance on NLP task benchmarks such as MMLU and do not reflect progress toward ‘intelligence’ in general.

1) can be partially overcome through quality assurances in training data, or the integration of symbolic systems or source-databases but in longer term tasks where chains of logical inference are required this continues to be a problem for AI systems.

For 2) it is important to note the relationship between the development of Natural Language Processing techniques and benchmarks to evaluate them. While increases in performance have been huge in the last 5 years, this performance has been limited NLP evaluations. In other words, AI developers are developing models to achieve higher performance on relatively narrow benchmarks and then proclaiming progress toward intelligence in general on the basis of their success. NLP tasks are a huge part of the modern service based and highly digitized economy and will be massively impacted by AI, but this has no more baring on progress toward ‘human’ intelligence in general than the calculator did when it automated arithmetic.
(1/2)

Sam Tissot said...


Some 'emergent' capabilities are certainly compelling and there is a debate ongoing as to why they arise. Do they come for free with a machine that can mimic human language? Or is it the case that LLMs training data is so large that many examples of the task are actually included in its training data?

For example, a compelling case of ‘emergent’ capabilities is LLM’s capacity to translate texts between two languages. LLMs mostly outcompete older deep learning models such as Google’s BERT on translation benchmarks. Putting the technical details to one side, BERT is trained explicitly to translate between two languages, whereas LLMs are trained to predict the next token (or word). At first glance, LLMs ability to translate between two texts seems nearly magical. However, when we consider that the internet has a huge number of pages of identical text in various different languages it becomes harder to maintain the claim that LLMs were not trained in someway to translate. It can be argued that while it was not the intention of the AI developers to create a translation model, they did so by including so much translation data in the training date. This is not to say this necessarily explains all ‘emergent’ capabilities in LLMs, but it is an important detail which suggests that just scaling up data and training time over and over will not necessarily lead to more and more ‘emergent’ abilities, which is often implicitly or explicitly claimed by AI CEOs and researchers.

Similarly to point 2 above, even the most sophisticated ‘emergent’ abilities are going to be limited to digitalised NLP tasks. In many cases LLMs already perform these tasks at a superhuman level, but again this doesn’t mean they are progressing toward human-level intelligence in general.

One more point on deep learning techniques and the analogy to the human mind. You are right that in principle ‘the human brain is material’ and ‘its basic laws can be discovered and reproduced’ but the question is have they been reproduced? Or is there evidence we are close? Granted, it is true that for decades symbolist approaches to AI dominated the field. In philosophy the corollary of this approach, which I take to be language of thought has always had a bit of whiff of idealism about them in my view. Nevertheless, the successful return to connectionist approaches does not necessarily mean the field has returned to a more materialist outlook and broken the back of the problem of replicating human intelligence. While deep learning techniques are analogous to the neural structure of the brain at the most general level, beyond that the analogy breaks down pretty quickly. In the human brain there are multiple types of neuron, a variety of different electro-chemical processes, diverse connections. Similarly, techniques like transformers or back-propagation which have been crucial in recent developments are unrelated to known processes in the brain (although our limited understanding of the latter makes any adjudication on this issue complex).

I believe it is possible to warn about the significant economic, military and surveillance aspects of AI while at the same time pushing back against claims to ‘superintelligence’ or ‘AGI’ championed by tech CEOs and some AI researchers. I think the main point I make in response to Hao’s book also applies here: the direction of travel behind AI and the public discourse surrounding it has much more to do with the peculiar features of the capitalist economy in the age of global tech monopolies that it arose from than it does about a new age of history where machines are more intelligent than, and therefore an existential threat to, human-beings.

Interested to hear more about what you think about these responses as you raise important points.

(2/2)

Sam Tissot said...

This was responding to your initial comment Peter, but I think what I said in the response is related the point you make about 'general outlines' of a path toward human intelligence as well. It depends, how general your use of 'general' is I suppose. It's clear that both connectionist and symbolist approaches do simulate something like what's going on in the human brain at some very abstract level- but it isn't clear that refining such a simulation is a clear route to human-like intelligence.

Peter Ross said...

Hi Sam. I find most of what you say here reasonable, but I do disagree with much of it.

I agree on the existence of a stock market bubble.

I don’t think one needs to have a perfect way of comparing capabilities or a perfect definition of intelligence for the concepts of AGI and super-intelligence to be useful. Human intelligence possesses a generality that is missing in other animals and AI, and that is precisely why the latter are still limited to being tools. The notion of AGI is important because it emphasizes this difference and that it represents a critical threshold. The notion of super-intelligence is important because it emphasizes that human intelligence is not the absolute limit of what is achievable in nature. Whether we want to call existing AI intelligent or not is probably irrelevant. The question, as you note, is whether it truly represents a step toward general intelligence, however that is to be defined (more on this later).

I have to disagree with you on scaling laws. The nature and limitation of scaling laws is not so mysterious when we understand what LLMs are actually doing. An LLM is nothing more than a prediction machine. It builds a model of whatever slice of the world it is trained on, which can be language, or it can be vision, robotics, etc. It is passive prediction, i.e. not based on experience or interaction (that is the domain of reinforcement learning). The fact that LLMs do better as you scale them up is just a consequence of a larger/richer model of the world being more capable than a smaller/less detailed one. To the extent that something more than modeling the world is necessary for intelligence, the scaling laws fail to deliver. However, since learning to model the world is a component part of an intelligent system, any future AI will exhibit similar kinds of scaling laws.

Emergent capabilities are also maybe not so surprising, because it actually is quite natural that a larger model can do things that a smaller model can’t, and that the transition from one regime to another can be sharp, i.e. a sort of punctuated equilibrium. I don’t think this has anything to do with language in particular, but is a more general phenomenon. Do such abilities come from memorizing the training data? That seems likely, but would only indicate the limitations of the current methods and not neural networks in general.

“Hallucination” is an unavoidable property of the way LLMs are trained. One trains them to be prediction machines, and then, in the crudest way imaginable, fine tunes them with a bit of reinforcement learning. This isn’t a fundamental limitation of neural networks. Human-generated data, on the other hand, is a limitation only so long as AIs are not primarily learning from their own experiences.

(1/2)

Peter Ross said...

I don’t know, of course, how close we are to reproducing the “laws of motion” of intelligence, but I do think we can be confident that deep learning makes use of *some* of these principles. These are very simple ideas: represent everything, whether language or anything else, as matter in motion in a high-dimensional space, build a model of the world by stacking simpler models, etc. All of it is done in a simple and pretty much unavoidable way, a much more powerful way than symbol manipulation. The brain must be doing something analogous, and the great variety of biology, all the various types of neurons, etc are probably not essential.

Again, one thing to notice about these systems is how crude they are. We do the most obvious thing, yet they still end up quite capable. That is a strong indicator that much can be achieved with relatively simple architectures, and that there is a lot of room for improvement. These are early days.

What is missing from AI? Better generalization, long term/ episodic memory, the ability to continually learn, more powerful forms of recurrence, advances in reinforcement learning, etc. None of these issues seem particularly insurmountable. Given the clearly vast potential of deep learning to build arbitrary levels of complexity into a brain via a process of self-organization, i.e. “learning,” it is easy to imagine that one or two important insights applied at scale would probably give rise to a qualitative leap, and potentially very rich behavior.

So while I agree that it is vital to talk about all of the near-term implications, and to counter the techno-fascists that are steering the ship, it looks to me like there is no fundamental obstacle between here and AGI, and as little as a few major scientific insights could be all that is needed. A problem I have with some of the discourse on the left is the idea that, because we don’t really understand where this is going and everything is shrouded in uncertainty, we should downplay the possibility of AGI as hype that plays into the hands of the capitalists. I say: the total lack of understanding is very concerning! Anyone who makes a positive claim that AGI is far off must understand that the burden of proof is now fully on them.

(2/2)

Sam Tissot said...
This comment has been removed by the author.
Sam Tissot said...
This comment has been removed by the author.
Sam Tissot said...
This comment has been removed by the author.
Sam Tissot said...

To make it easier to follow I am going to respond to some of the points you make Peter by quoting them and responding underneath.

“I don’t think one needs to have a perfect way of comparing capabilities or a perfect definition of intelligence for the concepts of AGI and super-intelligence to be useful. Human intelligence possesses a generality that is missing in other animals and AI, and that is precisely why the latter are still limited to being tools. The notion of AGI is important because it emphasizes this difference and that it represents a critical threshold.”

--

I agree that ‘generality’ is an important component of the differences between human cognition and AI- however, it isn’t the only one, for example, agency and some of the other aspects of human cognition like episodic memory etc. many of which you list in your previous comment. Nonetheless, I think it is still a dangerous term because of its ambiguity and because it raises associations with sci-fi tropes like Hal from Space Odessey, Ava from Deus Ex-Machina and all the way back to Shelley’s Frankenstein’s monster.

I do think the comparison to human intelligence is crucial for grounding the discussion away from such tropes. For instance Musk describes AGI as an “AI that is smarter than the smartest human” and predicts it will arrive in 2026. But ‘smarter’ in what sense? He seems to mean that a human using Grok can outperform a professor on some (any?) test. Okay, but in that case AGI has been around for a very long time… armed with a calculator a highschooler using a calculator could outperform the world’s leading mathematician at arithmetic, a human using Google Search is massively more capable than someone going through library stacks or newspaper ads manually. So whatever the definition of AGI is, we should strive to make it consistent and clear.

I think this tension around AGI definition has been around since the beginning of AI as a field. Different researchers seem to have had different definitions of AI. Marvin Minsky, for example, described AI as “the science of making machines do things that would require intelligence if done by men.” This is subtly yet crucially different from John McCarthy’s definition of AI as “the science and engineering of making intelligent machines.” Whether or not McCarthy and Minsky thought this distinction was significant they convey a completely different vision about the goals of AI research and whether one would even seek to achieve AGI. On the first definition, machines that achieve AGI in numerous domains are completely uncontroversial, like the calculator or Google Search examples above. However, on the second definition any AI system, whether better than humans or not is ‘intelligent’ in the human sense. My sense is that Minsky’s definition is the right one and leads one to conclude that talk of AGI is unhelpful – why would we want to recreate a general human like intelligence when we can create a host of machines that perform narrow tasks at a much greater speed and scales than humans. AI development is and has always been about creating tools that reduce the amount of labour required to perform a given task, not about recreating full or partial intelligence and claims about the latter lead to speculation and scenario-building that is dislocated from the actual state of the technology. In this sense, AI isn’t so different from the general trend toward greater and greater mechanisation in economic production that has typified the history of capitalist development. 1/?

Sam Tissot said...


“The nature and limitation of scaling laws is not so mysterious when we understand what LLMs are actually doing. An LLM is nothing more than a prediction machine. It builds a model of whatever slice of the world it is trained on, which can be language, or it can be vision, robotics, etc. It is passive prediction, i.e. not based on experience or interaction (that is the domain of reinforcement learning).”

--

You may, quite understandably, respond to the above that Grok outperforming a human on any test is a lot more general than the calculator or search example, and this is certainly true. As you say, given enough data in a particular domain neural AI systems can become highly performant at tasks within that domain. Training LLMs on the huge subsections of Internet makes it very effective (often times equal or better than expert humans) at tasks found on the Internet: writing, translating, programming etc. However, I would argue that being trained on the internet is still massively narrower than the rich day-to-day experience humans (and animals) are continuously exposed to. It isn’t massively clear that this is easy to scale up, particularly across multiple modalities of human experience.

Computer Vision provides ample evidence for this. While in narrow and controlled domains it is very easy to get models to outperform humans, simple variations in light or signal interference (easily dealt with by humans) can be catastrophic. Like other neural systems applying them to test data outside of their training data leads to huge degradation in performance- the “Islands of Automation” section of this piece is particularly interesting for how this plays out in the practice of Radiology (https://www.worksinprogress.news/p/why-ai-isnt-replacing-radiologists). 2/?

Sam Tissot said...

"“Hallucination” is an unavoidable property of the way LLMs are trained. One trains them to be prediction machines, and then, in the crudest way imaginable, fine tunes them with a bit of reinforcement learning. This isn’t a fundamental limitation of neural networks. Human-generated data, on the other hand, is a limitation only so long as AIs are not primarily learning from their own experiences."

My understanding is that ‘hallucinations’ are fundamental (to neural models of any kind in isolation (I know I introduced the term in the last comment but I actually prefer the term confabulation because AI models are not deviating from some grounding world model). See: https://en.wikipedia.org/wiki/Confabulation_(neural_networks). Indeed, neural mechanisms in the human-brain presumably confabulate signals all the time but we are mostly able to correct these occurrences through our continuous interaction with the external world. You raise the centrality of this issue in your statement that “AI are not primarily learning from their own experiences.” On this point I agree.

However, how and whether it is even possible for AI to have human like experiences of their interaction of the world (or if some digitized proxy is ever capable of reaching the levels of fidelity of the input humans gather from perception) are completely open question. So far, AI is only capable of learning from data that is digitized and then recognizing patterns between the 1s and 0s we’ve assigned to images or words or sounds etc (I know this is a massive over-simplification but I am just trying to illustrate 1) that simulating the full extent of human perception and interaction with the world is much much more complex then training a gen AI in one big go on how ever many trillion words or images 2) there may be principled differences between human ‘analog’ perception and computations carried out over ‘digitized’ representations of words, images, audio). 3/?

Sam Tissot said...

"It looks to me like there is no fundamental obstacle between here and AGI, and as little as a few major scientific insights could be all that is needed. A problem I have with some of the discourse on the left is the idea that, because we don’t really understand where this is going and everything is shrouded in uncertainty, we should downplay the possibility of AGI as hype that plays into the hands of the capitalists. I say: the total lack of understanding is very concerning! Anyone who makes a positive claim that AGI is far off must understand that the burden of proof is now fully on them."

--

I disagree with the approach here. I actually think the burden of proof lies with those arguing there are no principled barriers to AGI (however that is defined) as to why things like episodic memory, some proxy of human like experience, continual learning etc. seem like they so obviously follow from next-word prediction based on neural nets trained at a massive scale. It should also be noted that models for how to divide up the various cognitive systems in the human brain and how they interact together are highly variable. I did some work on working memory and visual memory models a couple of years ago and there’s not really broad consensus on how to delimit it from long-term memory, short-term memory and iconic memory – and whether or it not it works in the same ways across various modalities of perception.

The correct position is not that these things are impossible, but they we shouldn’t be preparing for their immediate advent without pretty strong evidence backing that. What we should be preparing for is the economic impact on the huge number of people that involve repetitive tasks carried out on computer – which particularly in developed economies is a huge percent of overall employment (we should still be cautious here, studies on impacts of AI adoption aren’t as conclusive as the general narrative around AI would have you believe e.g., Becker et al.’s study of coding tools on productivity https://arxiv.org/abs/2507.09089). The economic/environmental impact of the huge investment in data centers. The fact that the technology as it exists could be massively beneficial across the whole economy but is warped to the interests and ideology of a tiny band of tech CEOs etc etc.

As for the more fundamental question of whether artificial intelligence in the strongest sense of the term is possible (where AI stops being a tool used by humans and becomes some kind of being in its own right), it seems to me that there is nothing to suggest modern methods are any closer to this now than they were 50 years ago or indeed at the onset of the industrial revolution. While it may be idealist to suggest that such an eventuality is a certain impossibility, it doesn’t mean the correct materialist analysis is to put a plus sign where idealists place a negative. The truth is we don’t really know how the human mind works or the peculiarities about how and when it evolved (theories like the ‘cognitive revolution’ happening 70,000 years ago with the overnight advent of language supported by Chomsky are pretty conjectural in my opinion). It may not be necessary to replicate the exact history of our own mind’s evolution to recreate human intelligence, but I’ve not yet seen a clear conception of any other route to that result. In the absence of such a road-map it isn’t something Marxists should be immediately concerned about in my opinion. 4/4

Sam Tissot said...

An additional comment just on your remarks abouts about warnings from experts in the field. Warnings from Hinton and Bengio are well-noted (and I actually think they raise concerns about the direction of AI research that are valid independently of the AGI question). However, it isn't clear at all that the majority of AI researchers believe AGI is imminent or the path to get there is obvious. This year's AAAI survey (report here: https://aaai.org/wp-content/uploads/2025/03/AAAI-2025-PresPanel-Report-FINAL.pdf) reported that 76% of 500 respondents (mostly academics) assert that “scaling up current AI approaches” to yield AGI is “unlikely” or
“very unlikely” to succeed." While I imagine many of them would be less skeptical than I am about the concept of AGI overall, it isn't the case there is a wide consensus among leading researchers that AGI is just around the corner.

Daniel Lazare said...

Very interesting discussion. I agree with Sam and especially appreciate the following:
"The truth is we don’t really know how the human mind works or the peculiarities about how and when it evolved (theories like the ‘cognitive revolution’ happening 70,000 years ago with the overnight advent of language supported by Chomsky are pretty conjectural in my opinion)."
Chomsky's theories about a magical development 70K years ago are absurd since it's clear that language and intelligence developed gradually over some four million years of hominid development. Some sort of threshold was crossed with Homo Erectus, which appeared some two million years and apparently possessed some kind of proto-language. But while I'm a big fan of sci-fi series like "Humans" and "Real Humans" as well as "Ex Machina," the puzzle remains how tech can possibly replicate such a rich and lengthy biological process. Indeed, technology and biology seem in many ways to be opposite.

100th Anniversary of the October Revolution

100th Anniversary of the October Revolution
Listen to special broadcast

ΟΧΙ: Greece at the Crossroads

ΟΧΙ: Greece at the Crossroads
Essays on a turning point in Greece 2014 - 2017

Order ΟΧΙ : Greece at the Crossroads

Permanent Revolution Press

Permanent Revolution Press
Print edition of Crackpot Philosophy

Order Crackpot Philosophy

Trump and the train wreck of American liberalism

Trump and the train wreck of American liberalism
Two essays by Frank Brenner

Order PDF of 'Trump and the train wreck of American liberalism'

PDF of Brenner on Trump -$1

Contact Form

Name

Email *

Message *