Parag Agrawal, former CEO of Twitter and now founder of Parallel Web Systems, shared his vision for a future where search is optimized for artificial intelligence agents rather than humans. Parallel aims to reinvent web search technology and its underlying business models, believing that human click data is a "bug" and agents should rely on agent feedback.
Agrawal explained that Parallel is building infrastructure to allow AI agents to search and utilize the web, anticipating that agents will use the web a thousand times more than humans ever have. He highlighted a key difference from his Twitter experience: Twitter was a scaled, post-product-market-fit business with established feedback loops, whereas Parallel is a pre-product-market-fit company building for a "not yet here" customer, requiring daily learning and adaptation.
The fundamental problem of web search, for both humans and agents, involves crawling the vast web, indexing information, and then retrieving and ranking relevant results for a query – a "billion-to-billion matching problem." Historically, this has been an enormously expensive endeavor, limiting full web-scale indexes to giants like Google and Bing. However, Agrawal argues that the advent of AI agents makes this problem tractable for a new company. Agent feedback for evaluating search results is significantly cheaper than human ratings, and large language models (LLMs) are adept at compressing information. Parallel's strategy involves an incremental approach, initially offering a "search agent product" that performs deep research by crawling the web *after* a query is made, compensating for a nascent index. This allowed them to compete with outsourced human work for data curation, rather than Google's existing search engine.
Early use cases for Parallel's search agents included insurance underwriting, sales data enrichment, and financial data collection, tasks previously reliant on human labor. Agrawal clarifies that Parallel isn't a "neolab" in the sense of primarily producing large models; instead, its output complements existing models, giving agents "superpowers" by providing better, faster, and cheaper access to information. Parallel focuses on optimizing the retrieval and ranking process to deliver highly relevant "thousand tokens from a trillion web pages" in milliseconds, a significant reduction from the seconds typically allotted for such tasks.
A core distinction between human and agentic search lies in the query interface and content interpretation. Humans tend to be "lazy" with short, typo-ridden queries, and web content often caters to this by presenting fast-loading, easily digestible (though not always authoritative) information. Agents, conversely, provide more precise and often longer queries, and critically, Parallel can directly extract and deliver authoritative excerpts to an agent's context window, bypassing the need for an agent to "click around" or deal with SEO-optimized "slop" pages designed for human attention.
The economics of the internet are also shifting dramatically. The traditional ad-based model, which monetizes human attention, faces an existential threat as agents increasingly interact with the web. If content owners cannot monetize agent visits, they may block agents, disrupting the free flow of information. Parallel proposes a new, scalable business model based on "Shapley values," a game-theoretic concept for fair value distribution. This system would attribute incremental value to content based on how much it contributes to an agent's output quality, paying more for unique, differentiated data or if the data is used in high-value work (e.g., by a financial analyst). Agrawal believes that if a small percentage (2-10%) of LLM inference spend were allocated to web data, it could create a significantly larger and more sustainable ecosystem for content owners. This approach aims for incentive alignment, where content owners are compensated fairly, fostering participation rather than exclusion.
Parallel's strategy has gained traction, notably through a partnership with Google Cloud, where Parallel provides search and grounding services for Google's enterprise agent APIs, demonstrating that even large model companies are willing to "buy" rather than solely "build" web access for their agents.
Agrawal envisions a multi-stage evolution for agentic search. Initially, agents act as tools, performing simple web searches. Next, multi-agent systems will emerge, orchestrating complex, long-running tasks. Ultimately, he sees the web transforming from a "pull" model (where agents actively search for information) to a "push" model, a "parallel web" where the system continuously monitors for relevant changes and "calls" agents when actionable information appears. This future state would involve constant compute allocation across the entire web, driving new agentic work based on real-time events and insights, marking a profound shift in how we interact with and extract value from the internet.