OpenAI and other artificial intelligence companies are working to overcome unforeseen obstacles and delays in their quest for ever-larger algorithms by creating training methods that employ more human-like ways for computers to "think." According to a new analysis, these methods—which are the basis for OpenAI's recently published o1 model—may change the AI arms race and have an impact on the kinds of resources that AI companies are obsessed with.

However, some of the most well-known AI researchers are increasingly criticising the shortcomings of this "bigger is better" mentality. Results from scaling up pre-training, the stage of training an AI model that employs a large amount of unlabelled data to understand linguistic patterns and structures, have plateaued, Ilya Sutskever, co-founder of AI laboratories Safe Superintelligence (SSI) and OpenAI, told Reuters recently.

Sutskever is generally seen as an early proponent of using this strategy to achieve enormous advancements in generative AI, which ultimately led to the creation of ChatGPT. Earlier this year, Sutskever founded SSI after leaving OpenAI.

"We are once again in the age of wonder and discovery, whereas the 2010s were the age of scaling. Sutskever remarked, "Everyone is searching for the next thing. Scaling the right thing matters more now than ever."

Sutskever stated that SSI is developing a different strategy to scale up pre-training, but he would not elaborate on how his team is tackling the problem. According to those acquainted with confidential affairs, researchers at prominent AI labs have been encountering setbacks and delays behind the scenes as they compete to unveil a massive language model that surpasses OpenAI's GPT-4 model, which is almost two years old.

Recommended For You

For large models, the so-called "training runs" require hundreds of chips to operate simultaneously, costing tens of millions of dollars. Given the complexity of the system, they are more prone to experience hardware-induced failure; researchers could not be aware of the models' final performance until the run is complete, which might take months. Another issue is that AI models have used up a lot of the training data available online, and large language models consume enormous volumes of data. The training runs have also been hampered by power outages because the procedure uses a lot of energy.

In order to address these issues, scientists are investigating "test-time compute," a method that improves on current AI models while the model is in use, or during the so-called "inference" phase. For instance, a model could produce and assess several options in real time, eventually selecting the optimal course of action, rather than selecting a single response right away. This technique enables models to allocate more processing resources to difficult activities that need human-like thinking and decision-making, such as complex operations or math or coding issues.

At last month's TED AI conference in San Francisco, Noam Brown, an OpenAI researcher who worked on o1, stated, "It turned out that having a bot think for just 20 seconds in a hand of poker got the same boosting performance as scaling up the model by 100,000x and training it for 100,000 times longer."

This method was adopted by OpenAI in their recently published model, "o1," which was formerly known as Q* and Strawberry and was initially reported by Reuters in July. Like human reasoning, the O1 model is able to "think" through issues in multiple steps. Using information and comments selected from PhDs and business professionals is another aspect of it. The company claims that it intends to use this technique with more and larger base models. The secret sauce of the O1 series is another set of training done on top of "base" models like GPT-4.

According to those familiar with the initiatives, researchers at other prestigious AI laboratories, including Google DeepMind, xAI, and Anthropic, have also been striving to create their own variations of the method.

"We see a lot of low-hanging fruit that we can go pluck to make these models better very quickly," OpenAI Chief Product Officer Kevin Weil stated during an October tech conference. "By the time people do catch up, we're going to try and be three more steps ahead."

The implications might change the AI hardware market, which is now driven by the unquenchable demand for Nvidia's AI chips. Well-known venture capitalists, ranging from Sequoia to Andreessen Horowitz, are paying attention to the shift and assessing how it will affect their costly investments. These investors have contributed billions of dollars to the costly development of AI models at various AI laboratories, such as OpenAI and xAI.

According to Sonya Huang, a partner at Sequoia Capital, "this shift will move us from a world of massive pre-training clusters towards inference clouds, which are distributed, cloud-based servers for inference," she told Reuters.

Demand for Nvidia's most advanced AI chips has propelled the company's ascent to the top of the global value rankings, overtaking Apple in October. In contrast to training chips, where Nvidia is the market leader, the inference industry may be more competitive.

Jensen Huang, the company's CEO, has discussed the growing demand for its chips' inference capabilities. The scaling law at a time of inference is the second scaling law that we have now found. In reference to the company's most recent AI chip, Huang stated this month at a conference in India, "All of these factors have led to the demand for Blackwell being incredibly high."


Also In News