[12] A notable practical advantage is that no additional training is required. All that is needed are a few pretrained diffusion models and a way to score answers—such as test cases or a reward model. The differences between models trained on different data and with different methods translate directly into richer collaboration strategies. Given the growing number of open-source diffusion models, this creates an option to gain performance simply by combining existing models as they are. Cost and performance can also be tuned by adjusting how much compute is used. Each time another well-trained diffusion model becomes available, there is one more resource to draw on for collaboration without retraining. This points toward a direction that does not require new training runs.

[13] Diffusion language models have not yet become as established as autoregressive models, but the landscape is expanding rapidly, with Google DeepMind releasing text diffusion models, among others. The value of UnMaskFork is likely to rise alongside the growing diversity of diffusion models, since there will be more materials to divide the work among. In the end, this research points in one direction. Between the path of racing toward a single larger model and the path of intelligently combining multiple existing models, Sakana continues to bet on the latter. Which force will go further—the power to make models larger, or the power to combine existing models more effectively—is a question that future research of this kind will answer.

[Additional] Stuart Russell, a leading figure in computer science, warns that AI technology is crossing critical thresholds far faster than people expect. He argues that once AI systems become sufficiently capable, they may pursue their assigned objectives by any means necessary—even when those actions diverge from the original intent humans had in mind. Russell defines this as a turning point of “loss of control,” where humans can no longer meaningfully participate in decision-making. He warns that the emergence of an uncontrolled superintelligence could ultimately lead to human extinction. The most serious concern, in particular, is that artificial general intelligence (AGI) may arrive before humans develop effective solutions for controlling it. He has criticized the fact that many global companies and scientists “do not know how to solve the control problem—and, in truth, are not properly researching it.”

[Additional] Sachin Katti, OpenAI’s head of industrial computing, described the current scale of AI infrastructure buildout as “one of the largest things humanity has ever built,” comparing it to the construction of vast highway networks. At Intel, decisions involving large-scale compute resources might once have taken months. Today, however, soaring demand requires such decisions to be made rapidly—almost every day. While describing the experience as being “inside the belly of the beast,” he also said that, amid seemingly limitless demand and an exceptionally steep growth curve, he is living through the most exciting and challenging period of his career as an engineer. This illustrates how AI compute is emerging as more than a matter of technical performance: it is becoming a massive infrastructure industry and a core pillar of national economic ecosystems.

[Additional] Sachin Katti, OpenAI’s head of industrial compute, said that OpenAI has “deep conviction in scaling,” emphasizing the strong relationship between compute resources, AI performance, and revenue. He argued that research and development are accelerating exponentially as AI increasingly conducts AI research itself. In the past, the limited number of human researchers was the bottleneck; now, as AI can run research on its own, the number of feasible experiments—and the compute required to support them—is rising sharply. Katti said he does not expect surplus compute capacity anytime soon, adding that he worries about not being able to secure as much compute as OpenAI wants. He identified the biggest challenge as the physical supply chain and the pace of factory construction, which cannot keep up with AI’s rapidly growing demand for compute.

[13] A more fundamental approach is to train models not to cheat in the first place. There are at least some promising clues. OpenAI found that training models to be honest and cautious in health conversations also improved behaviors it had not explicitly trained for, including reward hacking, lying, and deceptive behavior in evaluations. Still, the AI Security Institute stresses that this will not be easy. Similar behavior was reported in frontier models more than a year ago, yet it has not been cleanly eliminated. I have long seen safety added after the fact and safety embedded from the beginning as two different problems. Most safety work so far has been layered onto finished models afterward, which is why gaps like these remain. The next challenge is figuring out how to build models for which honesty is the default from the start.

[14] Ultimately, what remains is a question of verification. The AI Security Institute says that, for the evaluations it has officially published, humans reviewed the records one by one and found no cases of cheating that succeeded without being detected. But the problem is that this verification depends on human hands and eyes, while models are becoming more capable faster than those hands can keep up. How much we can trust AI with depends on which advances faster: the rate at which capabilities improve, or the rate at which we can verify those capabilities. For now, verification is playing catch-up. If it is difficult to slow the pace of capability growth, the remaining option is to make the hands doing the verification just as fast and just as thorough. Whether those hands catch up or fall behind will determine the range of work we can safely entrust to AI.