Markets 
How are you going? Enjoying the winter sun? Feeling optimistic about the future? Well, for one day only, let me try to ruin your day. Or rather, let me let Dr Andrew Leigh ruin it. They say the devil makes work for idle hands, and that sums up Leigh in one go.
Leigh is a former economics professor, who for ages has been the federal government’s assistant minister for productivity, competition, charities, treasury and the kitchen sink.
If that tells you anything, it’s that he hasn’t got a real job. He’s the only minister I know of who keeps busy by writing books and thinking deep economic thoughts.
Although he’s always among the smartest in the room, Leigh hasn’t been smart enough – or maybe dumb enough – to join one of Labor’s factions. So by the time the factions have finished dividing them, all the big jobs are taken. Someone as smart as Leigh can’t be excluded, so he gets what’s left on the floor.
Unfortunately, Leigh’s deep thinking overflowed last month when gave the annual Giblin Lecture (Giblin was probably Australia’s most illustrious economist). Economists are meant to be happy, optimistic souls who advise us on how to get richer and consume more. They can deal with depressing subjects – such as recessions, wars, revolutions and pandemics – but always to find ways we can bounce back and resume economic growth.
But that wasn’t good enough for Leigh. He decided to go all the way – the full gloom and doom. Why don’t we think about the extinction of the human race? We should, because if we leave it too long, there’ll be nothing left to fix and no one left to do the fixing. So we need to begin the fixing as soon as possible.
So I hope that’s cheered you up. Leigh, naturally, has a lot more questions to ask and answer. How great is the risk that humanity could be wiped out?
His answer comes from the Australian philosopher Toby Ord, now at Oxford. Ord estimates the chances are perhaps as high as one in six – 17 per cent – over the coming century. Leigh says this fits with other expert assessments.
So what are the things most likely to cause humans’ extinction? Leigh says that whereas in the past extinctions were caused by natural events such as fluctuations in the climate or asteroid strikes, the most important risks today are “anthropogenic” – ones our species has brought upon itself.
This was made clear in the last century by the risk from nuclear weapons and also by catastrophic climate change.
But Leigh and others think the greatest risks are from emerging technologies such as advanced artificial intelligence. The second-greatest risk comes from synthetic biology.
As we seek to make greater use of AI we could unintentionally create technology that threatens our future existence. You don’t have to intend to do harmful things to cause consequences no one wants. If it takes us too long to realise what we’ve done, we’re in trouble.
Current AI developments already show forms of hacking being rewarded and behaviour which could be described as strategic. Where the new systems gain access to code, networks, financial accounts, laboratories or military infrastructure, terrible things could follow.
“The concern is not simply that a model makes mistakes. It is that a highly capable model may pursue the wrong goal in a clever, coherent, persistent and hard-to-reverse way,” Leigh says.
As for synthetic biology, he says engineered pandemics pose a danger to our species because human design can combine traits that nature often keeps apart. In principle, a pathogen can be modified to become more transmissible, more lethal, more resistant to treatment, or harder to detect.
Rabies and septicemic plague, for instance, show near total lethality in untreated cases. Biotechnology raises the possibility of bringing these attributes together – engineering a pathogen with a high infection fatality ratio and a high degree of transmissibility.
At least three pathways to catastrophe can be identified, he says. The first is an accident: a dangerous pathogen released from a laboratory. The second is state action: an offensive program justified internally as deterrence or strategic insurance. The third is non-state misuse: a terrorist group or apocalyptic cult seeking mass casualties.
Leigh worries that modern markets often reward capability more strongly than safety. “We are frequently better at inventing the rifle than a gun safe [to put them in], the pathogen than the surveillance system, the agent than the alignment protocol,” he says.
So what could we do to reduce the chances of unexpected developments leading to disastrous consequences?
In the case of rogue AI, there may be value in limiting the model’s ability to keep repeating their efforts to improve itself, in expanding research to include building “wise AI”, improving testing, strengthening monitoring, and boosting transparency internationally through global AI safety conferences.
In the case of synthetic biology, Leigh says promising solutions include screening orders involving DNA synthesis, improving the security of dangerous pathogen research, tightening laboratory practices, monitoring wastewater and strengthening collaboration on biological weapons.
Are we likely to do a lot of all that? Probably not. Why not? Because, although the risk of extinction of the species is real – with the prize for holding the winning ticket being total obliteration – the chance of holding that ticket seems fairly small, especially all the effort the above to-do list implies.
So why don’t we just get on with the exciting job of making everything bigger and better and take our chances on total extinction.
PS: please don’t mention this unpleasant matter again.
Ross Gittins is the economics editor.
From our partners
Read the full article here
