John Schulman argues that fast takeoff is no magic trick

By Imani Sutton · Reporting from Atlanta ·

The path to superintelligence depends on verifiable environments and human rubrics rather than a mathematical trigger.

# The Resource Cost of Superintelligence

Baron Militch bets on a specific mathematical trigger in the latest episode of the Dwarkesh Patel podcast. He argues that if an AI agent becomes just one-thousandth more capable than the top human researchers, running millions of those agents in parallel at chip speeds triggers a "fast takeoff." This surge would bypass every technical bottleneck on the road to superintelligence.

This vision suggests a singularity where software outruns its engineers. Yet the conversation between Patel, Militch, and John Schulman reveals that this spark is no magic trick. It is a shortage of electricity and clean data.

How do we build the ladder?

The debate centers on Recursive Self-Improvement (RSI), where AI rewrites its own code to become more efficient. Militch said this is a game of accumulation, like chess or mathematics, where one discovery permanently raises the floor. He suggests that once AI crosses a certain ELO threshold, the takeoff becomes inevitable unless the technology simply hits a ceiling.

John Schulman, however, anchors the conversation in the distance between a digital map and the actual terrain. He said models often seem incompetent after a month because they lack judgment and cannot self-check. To climb toward superintelligence, researchers must create verifiable environments—high-fidelity simulators where a robot can fail to pick up a cup a thousand times without breaking the hardware.

The signal for higher intelligence does not exist in the scrapings of Reddit or Wikipedia. As the guests note, no one will find the proof to a millennium prize problem hiding in Common Crawl. This means the takeoff depends on who can build the most effective rungs on the ladder. Schulman said the final human role is telling the machine what a "good" answer looks like. Human taste and human-designed rubrics still steer the intelligence.

Harvesting the Hive

The discussion turns to the "hive mind," where labs use deployment data to iterate models. Cursor, for example, updates its models in five-hour cycles based on user feedback. The labs use "distillation," a process where a lean model copies the patterns of a massive frontier model to gain efficiency.

To the labs, this is a matter of tuning a knob. In reality, it is unpaid labor. This system harvests millions of coders and writers who unconsciously fix the model's mistakes every time they hit "backspace" or "regenerate." When the speakers discuss Chinese labs using proxy servers to siphon data from US models for distillation, they are not describing a technical breakthrough. They are describing the theft of a digital blueprint. The intelligence does not emerge from a vacuum; the labs distill it from the traces of human work.

Software Cannot Outrun the Grid

The strongest case for a fast takeoff is that AI can perform a hundred years of physics research in a data center before running a single physical experiment. This is the peak of marketing hype: the idea that we can simulate our way out of copper and concrete.

But a century of theory still requires a century of electrons. The podcast treats compute as a background utility, like oxygen. It ignores that scaling laws are actually energy bills. According to the guests, RSI is easier than teaching a bot to file a legal brief because it is a cumulative task, but that ease is relative to the amount of power the labs are willing to burn.

The historical pattern of "inevitable" leaps—from the promise of nuclear power to the "frictionless" economy—always ignores the point where software hits the hardware. The jumps Militch hopes for are usually just shifts in who pays for the infrastructure.

The industry tells us that the path to superintelligence is a straight line of recursive loops and chip speeds. But the record shows that every intelligence explosion is a mining operation. The hive mind is just a new name for the old practice of building wealth on the uncompensated data of the many to benefit the few in the lab.

Sources

  1. AI researchers debate how close we are to recursive self-improvement