Arjun and Ronak, co-founders of Trajectory AI, presented their vision for "continual learning," addressing what they call the "experience gap" in modern AI models. They argue that while models are rapidly advancing in "IQ" (intelligence), they often lack the "experience" that allows them to perform tasks with human-like proficiency, making them feel perpetually new to the job.
Trajectory's core worldview is that vast amounts of valuable "experience" are generated daily through user interactions—billions of tokens and real work that agents perform—which are then discarded. They believe this data is the crucial signal for models to learn and improve, mirroring how humans gain expertise. Their mission is to transform static, expensive, and non-improving AI agents into "learning agents" that continually get better with use, leading to faster, cheaper, and more effective AI systems that "compound with use."
Their approach to achieving this involves a multi-faceted platform and cutting-edge research:
1. **Traceability:** The first step is to capture all interactions, including the full "tree" of actions, sub-agents, and tool calls, which are often overlooked.
2. **Model Specification & Reward Extraction:** They focus on extracting precise feedback from user interactions. Instead of just basic thumbs-up/down, they prioritize "corrective behaviors" like edits, undoes, and retries as high-fidelity signals for defining what the agent should learn.
3. **Continual Improvement:** This feedback is then used to improve *both* the models (e.g., through RL algorithms like SDPO on long traces) and the "harnesses" that orchestrate them. They differentiate between what should update the model's behavior versus what should update the harness's contextual information (e.g., a factual correction might go to the harness, while a procedural improvement goes to the model).
4. **Deployment:** The platform is designed to seamlessly deploy these continually improving models and harnesses.
Arjun outlined "four wishes" for the agent ecosystem to accelerate continual learning:
1. **Traceability:**
* Trace the entire action tree, including all sub-agent and tool calls.
* Design products to not only capture interaction data but also actively elicit and log high-signal feedback like user edits, undoes, and retries.
2. **Evals:**
* Evaluations should be drawn directly from real product traffic and "frontier" use cases.
* Tasks should be "roll-outable," meaning user actions can be replayed for evaluation.
* Grading must occur using the actual production harness, ensuring fidelity to real-world performance.
3. **Harnesses:**
* Shift from strict, enforced flows to allowing agents to "cook" by orchestrating core product primitives (tools, private data).
* Make the agent interface as close as possible to the user interface, ideally allowing every UI action to be a tool call.
* Ensure tool responses are highly informative (e.g., detailing what was written or read), not just vague confirmations like "done."
4. **Models:**
* Companies should become comfortable running open-weight models to gain ownership over their AI stack and enable continuous fine-tuning.
* Experiment with model routers to intelligently direct different parts of a task to the most suitable model.
Trajectory's product, currently in beta, aims to democratize this capability. They emphasize abstracting away the complexities of post-training and fine-tuning, allowing users to import, train, evaluate, and deploy models (like Harvey or Lab benchmarks) with minimal effort (e.g., 15 minutes of user interaction). Their goal is to empower any company to "own its own experience layer" and leverage continual learning.
During the Q&A, Arjun addressed key concerns:
* **Trainable Objects:** He clarified that continual learning is a system-level optimization, where the platform intelligently decides whether to update model weights, harness logic, or context based on the learned signal, abstracting this complexity from the user.
* **Customer Data & Privacy:** Trajectory avoids direct training on customer data. Instead, they use techniques like sampling distributions, synthetic data generation, and cryptographic comparisons to learn from customer interactions without compromising privacy.
* **Scope of Continual Learning:** While broadly applicable, it's particularly exciting for "frontier" tasks. Continual learning allows agents to learn from user attempts at the edge of current capabilities, enabling the product to expand its functionality rather than forcing users to retreat to simpler requests after initial failures.