Stanford CS329A (Ep 7): Open-Ended Evolution: ADAS, The AI Scientist, and AlphaEvolve — When Code Becomes the Genome
In 2019, computing pioneer Richard Sutton articulated what is now recognized as one of the most prescient observations in artificial intelligence history — The Bitter Lesson:
"The biggest lesson that can be read from 70 years of AI research is that general methods that leverage computation are ultimately the most effective, and by a large margin... Human-crafted algorithms are invariably surpassed by systems that learn, search, and evolve autonomously."
Computer vision spent decades hand-crafting SIFT and HOG feature extractors, only for deep convolutional networks to surpass them overnight. Natural language processing spent years hand-engineering parse trees and grammar formalisms, only for attention-based transformers to dominate.
In this seventh lecture of Stanford CS329A: Self-Improving AI Agents, we confront the most provocative question facing modern agentic engineering:
"Why are AI researchers still hand-writing every line of ReAct loops, PRM verifiers, and MCTS tree search heuristics? What happens when we treat Turing-complete source code as biological DNA, allowing autonomous meta-agents to design, benchmark, and evolve superior AI agents without human intervention?"
This post investigates the frontier of Open-Ended Evolution LLM systems, exploring three landmark breakthroughs: Automated Design of Agentic Systems ADAS, Sakana AI's The AI Scientist Sakana for Autonomous Machine Learning Discovery, and Google DeepMind's AlphaEvolve powered by Self-Referential Code Evolution.