If you thought capitalism has been guilty of excess in its dealings with society and the environment, the bad news is that it’s getting worse, existentially worse. Michael Pascoe reports.
This is primarily a lift from the Economist magazine, specifically a piece by Silicon Valley guru Tim O’Reilly. Odds are you haven’t read it hence a summary before getting to the danger for us all from the threat he exposes.
O’Reilly’s thesis boils down to “the merchants are becoming princes”, upending the market disciplines that made it preferable to have “merchants” in charge rather than princes with untrammelled power.
The world’s first trillionaire
Uber-capitalism, personified by Elon Musk, is escaping the constraints which markets, helped by a century or so of regulation, have been supposed to impose. The motley crew of ticket clippers, carnival barkers, cult members, speculators, fellow travellers, admirers and mugs who launched Musk’s SpaceX into and beyond the stock market sold their capital into a prince’s bondage. O’Reilly summarises:
In June the company went public very much on Mr Musk’s terms. Its initial public offering sold stock to the public without giving the public any meaningful governance rights. Mr Musk kept roughly 85% of the votes, locked in through super-voting shares that can pass to his heirs through trusts. Buyers of the ordinary stock must waive their right to a jury trial and to bring class actions.
The company qualifies as a “controlled” one, which exempts it from the rule that a majority of its board be independent, and Mr Musk can be removed only by a vote of the share class he himself controls. Months earlier, shareholders of Tesla, his electric-car company, had handed him a pay package worth up to $1trn, whose real prize was voting control rather than cash.
The world’s first dollar trillionaire has assembled a set of structures designed to make him answerable to no one, for as long as he likes, and possibly past his own lifetime.
I’d add to O’Reilly’s commentary that Musk’s financial and political pressure to have SpaceX included in the major market indices will entrap the disinterested investor through ETFs and index-hugging fund managers.
Tyranny of AI
You might well think Musk is a dangerous maniac and want not a bar of him but your super fund is likely to effectively end up owning SpaceX shares and thus supporting Musk, his passions and his world view. This is someone who promotes far-right politics around the globe and claims his SpaceX will “become worth more than the rest of Earth” if it accomplishes its goals.
O’Reilly records the classical view that the self-interest of merchants would be a gentler master than the passions of princes. It was in merchants’ interests not to wreck the joint, or as John Maynard Keynes put it, better that a man should tyrannise over his bank balance than over his fellow citizens. Alas, writes O’Reilly, that’s over:
The market discipline that was meant to cage the prince has, in short, become the lever by which the prince escapes the cage. Keynes’ hope now runs backwards.
The bank balance is no longer a substitute for tyrannising over one’s fellow citizens. It is the means of doing so, and Mr Musk has already carried that power out of the boardroom and into government and party politics.
This matters well beyond one man’s ambitions, because corporations are themselves a kind of artificial intelligence, and their governance is a trial run for the governance of AI. Machines, bureaucracies and markets all belong to a family of systems that strip the world down to a few narrow inputs and act on them at a scale no individual human can match.
Apocalyptic visions
The existential bit of the problem arrives via the AI race. Not the data centre, electricity and water race* that Australian politics doesn’t seem to look beyond but the race for the next horizon: AGI, Artificial General Intelligence, the level that changes the game beyond the changes we’re already struggling to comprehend from “standard” AI. The O’Reilly article only hints at the implications:
He (Musk) is doing it at the very moment he is building a far more potent AI to power the enterprises he controls. He is also not alone. The people who promise that their frontier AI systems will be safe and aligned are, inside their own companies, stripping out the alignment mechanisms that three centuries of conflict produced. An AI shaped under the governance of Mr Musk’s xAI will not carry the same values as one shaped under a structure that still answers to somebody. The design of the company is a preview of the design of the machine.
I’ve just watched The AI Doc: Or How I Became an Apocaloptimist. Track it down. Watch it. It is frightening but watchable if you don’t mind some cutsie bits. It provides both the optimistic and apocalyptic visions of the AI overtaking us. It leaves some obviously serious questions unasked but still covers much more than any Australian politician has dared address.
There are a handful of men in the West driving corporations to win the AI race. Taking shortcuts on safety is a way to get ahead in the race. As the documentary at one point notes, there are more controls and regulations governing the sale of a sandwich than there are for the various AI models now self-creating.
And then there are authoritarian governments also in the race and an American government pledged not to let them gain an advantage.
Shortcuts anyone? Everyone?
Trust Musk?
Removed from constraints and never mind the dangers of AGI moving beyond any ability to constrain it, do you trust the motives of Musk and other princes to be seeking the best interests of society – or just themselves?
The track record of merchant princes is that they are mostly opposed to funding redundant people, surplus people not of economic use, people requiring government support to survive which in turn means taxing the rich to pay for the support.
Both the AI optimists and pessimists paint pictures of the masses not having economic purpose. The optimists see a world where we are all free to be artists, poets and musicians. The pessimists, well,
there are surplus people.
I’ve not noticed Musk and his ilk caring much about such people or being willing to spend their wealth feeding and housing them. The George Monbiot quote applies: The poor and the middle class pay taxes, the rich pay accountants, the very rich pay lawyers – and the ultra rich pay politicians.
The United States, home of the princes driving the race, demonstrates the truth of that. We will all end up suffering the consequences.
Australia far off the pace
Prime Minister Albanese’s AI speech was out of date when he delivered it. The Coalition is still waiting to be told what it is supposed to think but odds are that it will be whatever makes the quickest dollar. Victorian Premier Allan’s sideshow about AI’s role in recruitment looks positively quaint.
From rabbits and lantana to ebikes and vapes, government allows the most simple innovation to establish itself before the consequences are imagined, let alone acted upon.
Facing the complexity and ramifications of AGI, the power and wealth of those driving it, and the demonstrated incapacities of those representing us,
hopelessness is not an unreasonable reaction.
O’Reilly concluded his essay thus:
The doux commerce thinkers were not naive. They were choosing the lesser of the dangers in front of them, and their choice proved wise for centuries. We are now running the experiment the other way, betting that we can concentrate power on an enormous scale in the hands of men accountable to no one and trust the result to their vision. That bet might pay off. History suggests how seldom it does.
*A footnote from the CBA economics team: “The data centre boom is well underway in Australia with strong activity recorded across many economic indicators. We estimate data centres will add 6.0ppts to real business investment in 2026 and 5.0ppts in 2027, on a calendar year average basis. However, after imports, the net direct contribution to real GDP is much lower, closer to +0.2ppts each year.”
Hardly worth selling our future for.
Michael Pascoe is an independent journalist and commentator with five decades of experience here and abroad in print, broadcast and online journalism. His book, The Summertime of Our Dreams, is published by Ultimo Press.