Andrej Karpathy, a leading figure in AI, shared his recent, startling realization: he's "never felt more behind as a programmer." This sentiment, arising around December 2023, signifies a fundamental shift in AI capabilities. He observed LLM tools evolving from requiring frequent correction to reliably producing fine-tuned code chunks, enabling "vibe coding" – a fluid, intuitive development process that has led him to pursue an "infinity side project" list. Karpathy emphasizes that AI's evolution is not merely incremental; it's a fundamental change demanding a new perspective.
Karpathy posits that LLMs are ushering in "Software 3.0," a radical departure from previous paradigms. Software 1.0 involved explicit rule-based coding, and Software 2.0 leveraged data to train neural networks (programming by creating datasets). Software 3.0, however, redefines programming as "prompting," treating the LLM as an interpreter where the "context window is your lever." He provides striking examples: installing "Open Claw" now involves giving text instructions to an agent, rather than running a complex shell script. More dramatically, his personal "MenuGen" app, designed to overlay menu item pictures, is rendered "spurious" because the same functionality can be achieved by simply prompting Gemini with an image input and getting an image output. This paradigm shift means AI doesn't just make existing programming faster; it enables entirely new forms of information processing, like generating knowledge bases from unstructured documents, which were previously impossible.
Looking towards 2026, Karpathy envisions a future of "neural computers" where raw sensory data (video, audio) directly feeds into neural networks that dynamically render UIs. He suggests a reversal of current computing architecture, with neural nets becoming the "host process" and traditional CPUs serving as "co-processors," leading to an "extremely foreign" computing landscape.
A core concept Karpathy explores is "verifiability." LLMs excel at automating tasks where outputs can be objectively verified, a trait stemming from their reinforcement learning (RL) training using verification rewards. This explains the "jaggedness" of LLM intelligence: models can flawlessly refactor vast codebases or find zero-day vulnerabilities, yet struggle with simple common-sense questions like whether to walk to a car wash 50 meters away. This jaggedness necessitates human oversight, as LLMs, while powerful, remain fallible tools. Lab decisions on training data (e.g., extensive chess data for GPT-4) significantly influence these capabilities, implying users are "at the mercy" of what data is included. For founders, this means opportunities exist in verifiable domains not yet fully explored by major labs, where custom RL environments and fine-tuning could yield significant results.
Karpathy differentiates "vibe coding," which raises the accessibility floor for all programmers, from "agentic engineering." The latter focuses on maintaining the quality bar of professional software while dramatically accelerating development. He believes the traditional "10x engineer" concept is now understated, as effective agentic engineers achieve far greater speed-ups by coordinating "spiky, fallible, stochastic" agents. Consequently, hiring processes must adapt, moving from puzzle-solving to evaluating candidates based on their ability to implement large-scale projects using agentic tools, with other agents tasked to break their creations.
In this agent-driven world, human skills like aesthetics, judgment, taste, and high-level oversight become invaluable. While agents can handle API specifics and rote details (e.g., `keepdims` vs `axis`), humans must retain an understanding of underlying fundamentals (e.g., memory management in tensors) and provide strategic design. Karpathy notes that current agent-generated code can be "bloaty" or "gross," underscoring the ongoing need for human discretion in maintaining quality and elegance.
Ultimately, Karpathy foresees an "agent-native" world where infrastructure is designed for agents, not just humans. He expresses frustration with current documentation that provides human instructions ("go to this URL") rather than agent-ready "copy-paste" commands. The ideal scenario would be to simply prompt an LLM to "build MenuGen" and have it fully deploy without any manual intervention. This agent-first approach will extend to inter-agent communication, with "my agent talk[ing] to your agent" for tasks like scheduling.
Regarding education, Karpathy highlights the enduring value of understanding, quoting the idea: "You can outsource your thinking but you can't outsource your understanding." He stresses that humans remain the bottleneck for direction, purpose, and true comprehension. While tools like LLM-powered knowledge bases can enhance understanding by re-processing information, the human role in discerning "what to build, why it's worth doing, and how to direct" these powerful agents remains irreplaceable.