A newly released report examines the competition among major AI companies to develop self-improving AI. This report, prepared by Gladstone AI, analyzes the risks and challenges of this technology in the near future. In recent months, Anthropic and OpenAI have warned that AI may soon be capable of improving itself.
Risks and Challenges of Self-Improving AI
This milestone, known as 'Recursive Self-Improvement' or RSI, could mean the emergence of new AI capabilities at an unprecedented speed. However, the emergence of RSI could be very dangerous for humanity and may signal a stage where developers completely lose control over AI systems.
Read more: European privacy regulators introduce new rules for smart glasses
The Gladstone report emphasizes that AI companies are actively working on automating all stages of AI development. This includes generating new ideas, conducting experiments to evaluate those ideas, and interpreting the results. Most researchers Gladstone has spoken to believe that RSI may start in the near future, within 3 to 12 months.
The Importance of Regulation in AI Development
Gladstone executives have stated that the goal of this report is to enhance lawmakers' understanding of how RSI works and, consequently, their ability to regulate this technology. Jeremy Harris, CEO of Gladstone, has stated that their aim is to define the concept of recursive self-improvement in such a way that people can discuss it logically, especially in Washington. He referred to legislative efforts to ban superintelligent AI and called for a focus on processes that require regulation.
The report also provides a simple overview of how the AI research process works within companies and explains how these companies are gradually automating this process. In the first stage of training AI models, known as pre-training, the focus is on finding ways to enhance the model's intelligence using less computational power.
AI companies use a portfolio approach to test these ideas. Researchers allocate a small amount of computational power to each idea and use it to train a series of models. Models that reliably perform better are selected for another round of training, where more computational power is used.
Due to computational power limitations, companies cannot test all promising ideas on a larger scale. This means that even if AI companies can train a model capable of generating promising ideas, it is important that several bad ideas are produced simultaneously. If companies have to spend a lot of time and computational power filtering good ideas from bad ones, it would make sense to still rely on human researchers with the right intuition.
In contrast, AI companies with access to more computational power can test more ideas, which may help them reach RSI sooner. With more computational power for experiments, companies are more likely to find ideas that, according to the Gladstone report, are the rarest and most desirable: ideas that perform better with increased computational power, even if they seem unsuccessful at smaller scales.
Being able to identify when these ideas are found and especially whether AI helps in finding them is "a crucial part of this process that lawmakers currently have virtually no insight into," says Jeremy Harris. One of the features that makes regulating RSI difficult is the gradual nature of this process. Edward Harris states, "This is one of the reasons we have described it precisely to allow for pointing to a part of this process and saying that a human should be there."
