Introduction
The issues related to an Intelligence Explosion implicate the question as to how close we are to achieving Artificial General Intelligence (AGI). The concept of AGI has been influenced by various thinkers, but we are still away from reaching AGI, considered the ‘Holy Grail of AI’ that scientists have been dreaming of for decades. Embodied cognition and common sense can give us an explanation regarding the process for reaching AGI. However, there is a gap between the myth and the reality of AGI. The myth is based on ideas which may not hold true. The reality is much more complex with questions this Essay is trying to address before AGI can become reality. The future of AGI and the likeliness of an Intelligence Explosion is hard to predict, but it is likely to have profound implications for humanity and society. It could be of great benefit or great harm, or both.
History
According to Wikipedia one of the earliest authors to express serious concern that highly advanced machines might pose existential risks to humanity was the novelist Samuel Butler, who in his 1863 essay ‘Darwing among the Machines’ writes that the time will come when intelligent machines hold the real supremacy over the world and its inhabitants and no individual with a philosophic mindset can question. In 1951, computer scientist Alan Turing wrote the article “Intelligent Machinery, A Heretical Theory”, in which he proposed that artificial general intelligences are likely to “take control” of the world as they become more intelligent than human beings. In 1965, I.J.Good originated the concept now known as an “intelligence explosion” and said the risks were underappreciated. Scholars such as Marvin Minsky and I. J. Good occasionally expressed concern that a superintelligence could seize control but issued no call to action. In 2000, computer scientist and Sun co-founder Bill Joy published the essay, ‘Why The Future Does not Need Us’ identifying superintelligent robots as a high-tech danger to human survival, alongside nanotechnology and engineered bio plagues. Nick Bostrom published the book ‘Superintelligence’ in 2014, which presented his arguments that superintelligence poses an existential threat. In 2015, public figures such as physicists Stephan Hawking and computer scientists Stuart J. Russell and entrepreneurs Elon Musk and Bill Gates were expressing concern about the risks of superintelligent AI and encouraged more research on how to make it robust and beneficial. In March 2023, key figures engaged in AI signed a letter written by the Future of Life Institute calling a halt to advanced AI training activities until it could be properly regulated. In May 2023, the Center for AI Safety released a statement signed by numerous experts that declares that understanding the risk of extinction caused by AI should be a global priority alongside other societal-scale risks such as pandemics and nuclear war.
From AGI to Superintelligence
For many decades, scientists have predicted that artificial intelligence will eventually enter a phase of recursive self-improvement, giving rise to systems beyond human comprehension along a period of extremely rapid technological growth. The product of an intelligence explosion would not just represent the capabilities of AGI, but a Superintelligence that far surpasses our cognitive abilities. Within the tech industry speculation is growing that an intelligence explosion may be just around the corner. Sam Altman, CEO of OpenAI, recently claimed that his organisation is confident in how to build AGI. However, we are beginning to turn our aim to superintelligence in the true sense of the word. A researcher at OpenAI referred to controlling superintelligence as a short-term research agenda while another researcher referred to the online hype surrounding recent AI breakthroughs declaring that we have not yet achieved superintelligence. One should take these insider predictions with a grain of salt, given the incentive for big companies to create hype around their products. Some researchers consider the intelligence explosion to take place within a decade. In contrast Turing Award winners and deep learning pioneers Geoffrey Hinton and Yoshua Bengo both expect superintelligence in as little as five years. What would this mean for us? Many experts worry that the development of smarter-than-human AIs could lead to human extinction if the technology is not properly controlled. Hence, research for better understanding the implications of an intelligence explosion is urgently required.
Possibility of an Intelligence Explosion
There are several counterarguments one might have to the idea that an intelligence explosion is possible. The following lists the most common:
AI machines have advantages such as being able to run 24/7, hold enormous amounts of information in memory, and share new knowledge with others instantly. It is easy to observe that the collective cognitive power of humanity significantly outweighs that of other species, which explains why we have largely been able to impose our will on the world. If Superintelligence is developed, we can expect many thousands or millions of copies available in parallel, far outnumbering the number of human experts in any particular domain.
Some also argue that an intelligence explosion will be hampered by real-world bottlenecks. For example, AI may only be able to improve itself as fast as humans are able to build data centres with the computing power required to train larger models. But new improvements in AI algorithms could significantly improve their performance and make rapid acceleration possible even while the amount of computing power remains constant. Another objection is that no AI could ever create a thing superior to itself, just as no one human ever could. Instead, superintelligent AI will be a civilisational effort, requiring the combined brainpower of many humans using their most powerful tools. However, there have also been challenges which required the collaboration of humans over many generations. Yet a new generation of intelligent machines overcame this bottleneck very quickly not by learning from human examples, but by playing against itself many millions of times. For example, the art of playing Go was passed down and developed over many years by DeepMind’s AlphaGo.
Others have argued that an intelligence explosion is impossible because machines will never truly ‘think’ in the same way as humans, since they lack certain essential properties for ‘true’ intelligence such as consciousness and intentionality. But this is a semantic distinction that breaks down when we replace the word ‘intelligence’ with ‘capability’ focusing on issues whether systems can solve problems and effectively pursue goals.
Preventing an Intelligence Explosion
Unprecedented resources are currently being poured into the project of building superintelligence. In January this year, OpenAI announced a new project named Stargate, to invest USD 500 billion into AI development over the next four years with President Donald Trump vowing to support its mission through ‘emergency declarations’ that will support data center construction. Cash is flowing in other corners of the AI ecosystem as well. Forbes has projected that US spending on Ai will exceed a quarter trillion USD this year. However, regulation to control AI development is sparse. Governance of advanced AI largely depends on voluntary commitments, and more ambitious bills for control have failed to pass. In addition President Trump has already repealed Biden’s Executive Order requiring companies to notify the US government when they conduct large training runs in order to advance AI. But none of this means an intelligence explosion is inevitable. Humanity does not have to build dangerous technology if it does not desire to do so. There are many regulatory approaches we could take to reduce risk such as pausing development of the most powerful systems, mandating safety standards and testing of new AI models as was proposed in a recently vetoed bill. There are also many ideas on the table as to how we could control smarter-than-human AI, such as creating formal verifications or quantitative guarantees to ensure that powerful systems are safe. But capabilities are advancing very quickly, and just 1 to 3 percent of new AI development is currently dedicated to safety. Researchers working on these critical problems need more time defining the barriers to control an Intelligence Explosion.
Conclusion
Whether an intelligence explosion is possible or likely remains unclear. Some experts, such as head of the US government’s AI Safety Institute Paul Christiano, believe that an uncontrolled Intelligence Explosion is possible, but that the more likely scenario is a slow ramp up of AI capabilities. This could give humanity the opportunity to intervene before systems become dangerously capable. Nonetheless, there are compelling arguments in favour of an Intelligence Explosion, and many scientists are concerned about its potential impact.