首页  >>  来自播客: Sequoia Capital 更新   反馈  

Sequoia Capital - Parallel’s Parag Agrawal: Building a New Web for AI Agents

发布时间:   原节目
推特前首席执行官、现任Parallel Web Systems创始人帕拉格·阿格拉瓦尔(Parag Agrawal)阐述了他对未来搜索的愿景,即搜索将为人工智能代理(AI agents)而非人类优化。Parallel旨在重塑网络搜索技术及其底层商业模式,他们认为人类点击数据是一个“缺陷”,代理应依赖代理自身的反馈。 阿格拉瓦尔解释道,Parallel正在构建基础设施,以允许AI代理搜索和利用网络,预计代理使用网络的频率将比人类高出千倍。他强调了这与他在推特任职期间的一个关键区别:推特是一个已规模化、产品与市场契合(post-product-market-fit)并拥有成熟反馈机制的业务,而Parallel则是一个产品与市场尚未契合(pre-product-market-fit)的公司,正在为“尚未出现”的客户构建产品,这需要每天的学习和适应。 网络搜索的根本问题,无论是对人类还是代理而言,都包括抓取浩瀚的网络、索引信息,然后为查询检索和排序相关结果——这是一个“万亿对万亿的匹配问题”。历史上,这是一项极其昂贵的任务,将全网规模索引的能力限制在谷歌和必应等巨头手中。然而,阿格拉瓦尔认为,AI代理的出现使得新公司也能有效处理这个问题。用于评估搜索结果的代理反馈比人类评分便宜得多,并且大型语言模型(LLMs)擅长压缩信息。Parallel的策略是采取渐进式方法,最初提供一种“搜索代理产品”,该产品在查询发出*后*才通过爬取网络进行深度研究,以弥补其尚处于萌芽状态的索引。这使得他们能够与外部承包的人工数据整理工作竞争,而不是直接与谷歌现有的搜索引擎竞争。 Parallel搜索代理的早期用例包括保险核保、销售数据丰富化和金融数据收集,这些任务此前都依赖人工完成。阿格拉瓦尔澄清说,Parallel并非传统意义上主要生产大型模型的“新实验室”(neolab);相反,它的产出补充了现有模型,通过提供更好、更快、更便宜的信息获取方式,赋予代理“超能力”。Parallel专注于优化检索和排序过程,以在毫秒级别内从“万亿网页中提取出高度相关的数千个token”,这比通常为这类任务分配的几秒钟时间大大缩短。 人类搜索和代理搜索之间的一个核心区别在于查询界面和内容解读方式。人类在查询时往往“懒惰”,使用简短且常有错别字的查询词,并且网络内容通常会迎合这一点,通过呈现加载快速、易于消化(尽管不总是权威的)信息。相反,代理提供更精确且通常更长的查询,关键在于,Parallel可以直接提取权威摘要并将其发送到代理的上下文窗口,从而避免了代理需要“四处点击”或处理那些为吸引人类注意力而进行SEO优化的“垃圾”(slop)页面。 互联网的经济模式也在发生巨大转变。随着代理与网络的交互日益增多,传统的、通过人类注意力变现的广告模式正面临生存威胁。如果内容所有者无法从代理访问中获利,他们可能会阻止代理,从而扰乱信息的自由流动。Parallel提出了一种新的、可扩展的商业模式,该模式基于“沙普利值”(Shapley values)——一种用于公平价值分配的博弈论概念。该系统将根据内容对代理输出质量的贡献程度来分配增量价值,对独特、差异化的数据支付更多费用,或者当数据被用于高价值工作(例如由金融分析师使用)时支付更多。阿格拉瓦尔认为,如果将LLM推理支出的一小部分(2-10%)分配给网络数据,就能为内容所有者创造一个显著更大、更可持续的生态系统。这种方法旨在实现激励对齐,即内容所有者获得公平补偿,从而促进参与而非排斥。 Parallel的策略已获得关注,特别是通过与谷歌云(Google Cloud)的合作,Parallel为谷歌的企业代理API提供搜索和基础服务,这表明即使是大型模型公司也愿意“购买”而非仅仅“构建”其代理的网络访问能力。 阿格拉瓦尔设想了代理搜索的多阶段演进。最初,代理充当工具,执行简单的网络搜索。接下来,多代理系统将出现,协调复杂的、长时间运行的任务。最终,他认为网络将从“拉取”模式(即代理主动搜索信息)转变为“推送”模式,形成一个“平行网络”(parallel web),在该网络中,系统会持续监控相关变化,并在可操作信息出现时“呼叫”代理。这种未来状态将涉及整个网络的持续计算资源分配,基于实时事件和洞察力驱动新的代理工作,这标志着我们与互联网互动和从中提取价值方式的深刻转变。

Parag Agrawal, former CEO of Twitter and now founder of Parallel Web Systems, shared his vision for a future where search is optimized for artificial intelligence agents rather than humans. Parallel aims to reinvent web search technology and its underlying business models, believing that human click data is a "bug" and agents should rely on agent feedback. Agrawal explained that Parallel is building infrastructure to allow AI agents to search and utilize the web, anticipating that agents will use the web a thousand times more than humans ever have. He highlighted a key difference from his Twitter experience: Twitter was a scaled, post-product-market-fit business with established feedback loops, whereas Parallel is a pre-product-market-fit company building for a "not yet here" customer, requiring daily learning and adaptation. The fundamental problem of web search, for both humans and agents, involves crawling the vast web, indexing information, and then retrieving and ranking relevant results for a query – a "billion-to-billion matching problem." Historically, this has been an enormously expensive endeavor, limiting full web-scale indexes to giants like Google and Bing. However, Agrawal argues that the advent of AI agents makes this problem tractable for a new company. Agent feedback for evaluating search results is significantly cheaper than human ratings, and large language models (LLMs) are adept at compressing information. Parallel's strategy involves an incremental approach, initially offering a "search agent product" that performs deep research by crawling the web *after* a query is made, compensating for a nascent index. This allowed them to compete with outsourced human work for data curation, rather than Google's existing search engine. Early use cases for Parallel's search agents included insurance underwriting, sales data enrichment, and financial data collection, tasks previously reliant on human labor. Agrawal clarifies that Parallel isn't a "neolab" in the sense of primarily producing large models; instead, its output complements existing models, giving agents "superpowers" by providing better, faster, and cheaper access to information. Parallel focuses on optimizing the retrieval and ranking process to deliver highly relevant "thousand tokens from a trillion web pages" in milliseconds, a significant reduction from the seconds typically allotted for such tasks. A core distinction between human and agentic search lies in the query interface and content interpretation. Humans tend to be "lazy" with short, typo-ridden queries, and web content often caters to this by presenting fast-loading, easily digestible (though not always authoritative) information. Agents, conversely, provide more precise and often longer queries, and critically, Parallel can directly extract and deliver authoritative excerpts to an agent's context window, bypassing the need for an agent to "click around" or deal with SEO-optimized "slop" pages designed for human attention. The economics of the internet are also shifting dramatically. The traditional ad-based model, which monetizes human attention, faces an existential threat as agents increasingly interact with the web. If content owners cannot monetize agent visits, they may block agents, disrupting the free flow of information. Parallel proposes a new, scalable business model based on "Shapley values," a game-theoretic concept for fair value distribution. This system would attribute incremental value to content based on how much it contributes to an agent's output quality, paying more for unique, differentiated data or if the data is used in high-value work (e.g., by a financial analyst). Agrawal believes that if a small percentage (2-10%) of LLM inference spend were allocated to web data, it could create a significantly larger and more sustainable ecosystem for content owners. This approach aims for incentive alignment, where content owners are compensated fairly, fostering participation rather than exclusion. Parallel's strategy has gained traction, notably through a partnership with Google Cloud, where Parallel provides search and grounding services for Google's enterprise agent APIs, demonstrating that even large model companies are willing to "buy" rather than solely "build" web access for their agents. Agrawal envisions a multi-stage evolution for agentic search. Initially, agents act as tools, performing simple web searches. Next, multi-agent systems will emerge, orchestrating complex, long-running tasks. Ultimately, he sees the web transforming from a "pull" model (where agents actively search for information) to a "push" model, a "parallel web" where the system continuously monitors for relevant changes and "calls" agents when actionable information appears. This future state would involve constant compute allocation across the entire web, driving new agentic work based on real-time events and insights, marking a profound shift in how we interact with and extract value from the internet.