Could it mean the end of humanity?

Success in artificial intelligence “would be the biggest event in human history, and perhaps the last event in human history”, says Professor Stuart Russell in the first of his Reith Lectures, broadcast on Radio 4. The ultimate goal of researchers is to develop “general purpose AI” – robots that “have access to all the knowledge and skills of the human race”. If all goes well, AI could “herald a new age for humanity”, with robots able to design and build bridges, improve crop yields, cook dinner for 100 guests or teach a child to read. It would “raise the living standards of everyone on earth”, precipitating a 10-fold increase in GDP.

The problem is, AI could easily go wrong. By creating “entities far more powerful than humans”, we are putting our species at risk. But it’s not rebellious Terminator-style robots we should be afraid of – it’s our own choices. Machine intelligence is measured by “the extent that they can achieve their own objectives”. Humans set those objectives, yet we can’t be trusted to do so wisely. It’s like the Greek myth about King Midas, who asked the gods to turn everything he touched into gold. His family, food and drink all turned to gold, and he died of misery and starvation. The better AI gets at achieving our goals, the more we may come to regret it.

Listen to the lecture here.