AI Agents Must Learn from Time, Not Just Weights
By Klaus Berger ·
The next frontier of intelligence isn't better models, but systems that demonstrably improve and compound with real-world use over time.
The core argument emerging from the recent "Sequoia Capital" podcast, featuring Arjun Karanam of Trajectory, is not about building smarter models—it is about solving the problem of experience. The panelists argue that current large language models suffer acutely from an "experience gap," meaning raw intelligence alone, no matter how impressive, is insufficient; real-world time in a job remains critical. This realization shifts the focus entirely: the goal is no longer merely better weights, but systems where agents get demonstrably better and better over time, achieving what Trajectory calls models that "compound with use."
The mechanics of this continual learning are intricate. The process moves from capturing real interactions (Traceability) to defining goals (Model Spec), which then feeds into improving the model via RL/SDPO on long traces, alongside improving the supporting infrastructure—the harnesses. A key technical distinction noted was that knowledge updates, such as a company being delisted, should often update the harness or context rather than requiring retraining of the core model weights.
The Infrastructure of Intelligence
The discussion detailed four crucial pillars for this emerging agent ecosystem. First is Traceability, which must capture not just successful actions but the entire "tree of actions," including tool calls and critically, corrective behavior like edits, undos, or retries—going far beyond simple thumbs up/down feedback. Second are Evals (Evaluations), which must be robustly designed to mirror production environments for reliable testing. Third, the Harnesses themselves must evolve from rigid flow enforcers into orchestrators of core primitives, requiring that the agent interface mirrors the user interface (UI) to simplify training. Finally, Models must facilitate easy ownership and deployment of open-weight architectures, necessitating tools like "model routers" to route intelligence exactly where it is needed.
From Global Training to Private Context
Perhaps the most financially resonant insight concerns how feedback signals are processed. Arjun Karanam drew a clear distinction between two types of feedback: negative signals ("Something went wrong") and positive/corrective signals (a successful retry or explicit user correction). Crucially, he categorized these based on relevance scope. While globally accurate information—like a tool call repeatedly failing—is universally relevant and should be trained into the model, context-specific data remains highly valuable for continual learning at the per-organization or per-customer level. This focus on localized expertise is immensely important; it suggests that while the underlying AI might be global, its most powerful application layer will be hyper-private.
This echoes a fundamental principle of finance: institutional knowledge and proprietary context are where true value resides. The vision articulated by Trajectory—that "owning your own intelligence" should become an accessible expertise rather than a service requiring constant consultation—is essentially the industrialization of intellectual property in the AI age. Furthermore, addressing data privacy was addressed through synthetic generation; instead of training directly on customer data, one can sample distributions and synthetically generate new points, allowing for system improvement while maintaining strict confidentiality.
The Frontier of Failure
The central thesis remains that the most impactful applications lie at the frontier. Continual learning is not just about optimizing existing processes; it is about users attempting things that current models can barely manage. By absorbing these failures and limitations into their training data, the system learns to handle its own shortcomings, thus pushing the boundaries of what is possible.
This entire mechanism—the compounding improvement driven by capturing failure and specialized context—is a massive infrastructural shift. It requires integrating complex feedback loops (Traceability $\rightarrow$ Evals $\rightarrow$ Harnesses) into the core business process. The implication for capital markets and corporate finance is clear: the value capture will flow not to those who build the base models, but to the entities that can effectively structure, govern, and monetize their own unique, proprietary streams of failure and success data.