AI is the talk of the industry right now, but the makers themselves have been warning about what AI is capable of doing. Sam Altman, OpenAI's chief, has cautioned the United Nations Security Council that AI could help with massive growth in the fields of medicine, economic progress, education and more, but as it moves forward, these increasingly capable systems could move beyond human control or understanding. He has warned that this increasingly capable AI beyond the control of humans can be a serious threat to the future of humanity.
In his address to the council on Wednesday in New York, Sam Altman expressed that the timeline for the development of AI has shrunk with benefits and risks both becoming more immediate. Altman said, “We have a choice in front of us. AI can either be more like a new Renaissance of creativity and discovery, or more like a new Industrial Revolution of upheaval and disarray.”
ALSO READ: Meta Connect 2026: Ray-Ban Meta Gen 3, Meta VR Glasses And New AI Features Unveiled
Taking the example of the Renaissance, when new developments in tools and industry enhanced what humans were capable of doing, and said, “AI can play that role for our time,” but further suggesting that, “The best version of AI is not about making people cogs in a giant machine, or optimising every part of life until the human parts disappear. It is about giving people more agency in an ever more complex world: more ability to learn, to create, discover, build, participate. There are many things that AI cannot — and should not — automate.”
However, Altman acknowledged that the scale of AI’s capabilities can also be unsettling. He said that “anything this powerful is also intimidating”, describing the current moment as one that brings both “tremendous potential” and “very understandable anxiety”.
Speaking at the UN Security Council, Altman outlined two major ways in which the rapid development of AI could go wrong. The first is the possibility of humans losing control as AI systems become increasingly capable and progress happens too quickly for people to understand or intervene.
“First, we could lose control of the future to AI. The risk is that it moves so fast that people can no longer follow what’s happening or intervene when needed. This would obviously be terrible.”
The second concern is the concentration of AI power. Altman warned that a small number of companies, individuals or countries should not end up controlling systems powerful enough to influence society on a global scale.
“No one person or company or country should be able to use the most powerful AI models to impose their world-view on everyone else,” Altman said.
He said these concerns will become more significant if AI systems eventually gain the ability to contribute to the development of newer and more capable versions of themselves. This process, commonly referred to as recursive self-improvement, could significantly increase the speed at which AI advances.
“As the process of building AI becomes more automated, the pace of AI progress could accelerate rapidly. This moment calls for extreme care,” Altman said.
Altman also argued that AI companies should not allow the potential benefits of the technology, or pressure from competitors, to push them into taking unnecessary risks. He pointed to OpenAI’s previous decision to slow down certain work when necessary and said the company would be prepared to do so again.
According to Altman, developers should move ahead with increasingly powerful systems only when they have adequate measures in place for alignment, monitoring and safety. He also stressed that humans must continue to have meaningful control over advanced AI systems.
Beyond individual companies, Altman called for national and international frameworks that can evaluate the risks posed by powerful AI, check whether safety measures are actually working and ensure that human oversight remains in place.
ALSO READ: Adobe Premiere Finally Comes To Android: Free Video Editing App Gets 4K Export, AI Tools And More
Anthropic CEO Dario Amodei, who also spoke during the UN briefing, similarly pushed for countries to work together on AI safety. He urged governments to start with international agreements covering specific high-risk uses, including a ban on using AI to create biological weapons.
Amodei also called for common global standards to test advanced AI systems for potential loss-of-control and misuse scenarios. In addition, he proposed a system for reporting serious AI-related incidents that could have implications for global security.