Gabe, co-founder and president of Harvey, shared insights on how their company built a robust research lab, effectively demonstrating how application-layer companies can compete with frontier AI labs on a budget by leveraging the broader AI ecosystem. His talk, aptly titled "Building a Research Lab on a Budget," highlighted key strategies for developing frontier intelligence without the vast resources of major research institutions.
Gabe began by outlining the inherent challenges faced by application-layer companies: competing with "rich teams" possessing superior money, talent, compute, infrastructure, and data. Harvey's approach, born out of a landscape where many foundational AI companies were nascent, was to focus on product and go-to-market. Today, however, he argued that the mature frontier ecosystem enables application companies to build competitive intelligence.
The first critical step in Harvey's playbook is **building benchmarks**. Without robust benchmarks, training and serving models effectively is impossible. Harvey has open-sourced three significant legal datasets:
1. **Legal Agent Bench:** A taxonomy of complex tasks for associates at large law firms, covering diverse practice areas like drafting fund formation documents and case law research.
2. **Contracting Data Set:** Designed to teach agents negotiation skills for in-house legal departments.
3. **Large Diligence Data Set:** The most exciting, featuring data rooms up to 80 million tokens, allowing research into long-context, highly complex tasks.
A major challenge for Harvey, working with highly sensitive and privileged legal data from top law firms, is the inability to train on customer data. Their innovative solution involves **domain experts guiding synthetic data generation**. Gabe's lawyer brother, for instance, learned to "vibe code" using AI models to create incredibly realistic legal datasets. This process starts with defining a rubric of issues and a scenario, then generating the data room around these planted issues. Tools like Mercore and Snorkel then help scale this process and make the generated contracts look realistic. These synthetic datasets solve the "chicken and egg" problem of needing data to prove model capability and needing proof to get data.
These datasets are then turned into efficient RL environments, often integrated with tools like LangChain, to manage the high cost of evaluation, especially with large models and thousands of unit tests. Open-sourcing these datasets was a controversial but strategic move, inviting community input, identifying issues, and encouraging labs to benchmark on Harvey's data, which Gabe credits for receiving suggestions, pull requests, and even a retweet from Elon Musk.
Next, Gabe discussed **post-training models**. With open-source models like Kim E3, GLM 5.2, Nemotron, and Inkling reaching competitive levels, post-training has become a viable strategy. Harvey collaborates with "neo labs" (e.g., Fireworks, Base10, Graham, Applied Compute, Trajectory) which offer expertise and infrastructure. This multi-vendor approach allows Harvey to explore diverse research bets, train various open-source models, and gain valuable insights, complementing their increasing internal post-training efforts using APIs like Tinker. The goal is to package this work into Harvey's proprietary "Composer One" model.
**Serving models in production** is a non-trivial but crucial step. Harvey operates in 60 countries with multiple product areas and varying customer model preferences, necessitating a complex model serving infrastructure. This matrix includes handling fallback mechanisms across providers to meet SLAs, now extended to include open-source models.
Before any model (open-source, closed-source, or post-trained) goes into production, it undergoes rigorous **pre-production evaluation**:
* **Automated:** Lab benchmarks assess frontier status and legal domain strength.
* **Human Testing:** Side-by-side comparisons and critical user journey tests specific to product surfaces.
* **Heuristics:** Cost, latency, region, and availability.
Once in production, continuous **monitoring** is vital, using A/B testing, engagement tracking, uptime, token efficiency, and direct product feedback ("angry customer emails"). Gabe stressed that this infrastructure is essential *before* even considering post-training, as it provides the critical feedback loop.
He also recommended starting with "simple open-source switches" by identifying parts of the product where open-source models can be naively swapped for cost or performance benefits (e.g., citation generation), or by implementing model routing for specific queries. This builds the muscle for managing open-source models alongside proprietary ones, ultimately feeding into the post-training flywheel.
Gabe concluded by reiterating that every company will become an AI company, and Harvey's playbook offers a path to "change the game." He drew an analogy to Moneyball, where winning on a budget with a unique team ultimately transforms the sport.
During the Q&A, Gabe expanded on several points:
* **Synthetic Data Generation:** His brother's method involved planting issues into a rubric and then generating the data room, allowing for effective validation of whether a model catches those planted problems.
* **Hiring:** While initially difficult to compete with top labs, the maturation of the ecosystem and availability of specialized APIs reduce the need for deep infrastructure talent, broadening the pool of suitable researchers.
* **Benchmark Philosophy:** Open-sourcing creates a balance between community engagement and strategic differentiation. The true long-term value lies in helping law firms train on *their* private, sensitive data.
* **Remaining Open Questions:** Key challenges include bridging the distribution gap between synthetic and production data without using customer data, improving model performance on long-context complex tasks (like 80M token data rooms), and developing robust continual learning mechanisms for enterprise clients while preserving data privacy.
* **Competing with General Products:** Harvey's strategy shifts from individual productivity (like Codex Co-work) to **organizational productivity** for large law firms, orchestrating humans and AI agents across thousands of client projects, effectively "hyper-verticalizing" into the legal domain.
Gabe's talk provided a pragmatic and inspiring roadmap for application-layer companies aiming to build sophisticated AI capabilities, emphasizing strategic leveraging of the frontier ecosystem, innovative data generation, and robust operational infrastructure.