E2B

Exo: Harnesses should see their own code and logs — Alex Krentsel, UC Berekeley / Google Research

Aug 15, 2026 · 47:11

Alex Krentsel presents Exo, a fully recursive self-improving agent harness he built with Martín Casado and Anker Goyal, arguing agents can now safely edit their own code and runtime policy. Exo's architecture splits an agent into a stateless executor, a stateful harness with history and secrets, and an isolated sandbox, letting it change every component, roll back automatically, and teleport state—unlike OpenClaw/Pi, which only allow memory, skills, and tool extensions. The episode details Exo cutting its Discord adapter costs by 96% at runtime, plus secret stores and evals to prevent reward hacking. Krentsel argues RSI is newly possible because the harness is code in the same medium as LLM output; Exo runs in production at Brain Trust, with contributors directed to github.com/exoharness/exo.

The new Claude 3.5 Sonnet, Computer Use, and Building SOTA Agents — with Erik Schluntz, Anthropic

Nov 28, 2024 · 1:11:08

Anthropic's Erik Schluntz explains how Claude 3.5 Sonnet achieved state-of-the-art 49% on SWE-Bench Verified by using a minimal agent framework that gives the model full control via tools like bash and file editing, letting it self-correct without hard-coded workflows. He details the importance of tool design over prompt engineering, why XML tags work well, and how computer use reduces integration friction by letting models interact with any browser-based interface. Schluntz also shares his robotics experience, noting that while LLMs and diffusion models are promising for general-purpose robots, reliability at 99.9% and hardware variability remain major hurdles, and he expresses skepticism about self-driving as a business due to high vehicle costs versus driver-equivalent revenue.

In the Arena: How LMSys changed LLM Benchmarking Forever

Nov 1, 2024 · 41:02

Anastasios and Weilin from LM-Sys explain how ChatBot Arena became the de facto standard for LLM benchmarking by replacing static benchmarks with dynamic human preference evaluations. They trace the origin from Vicuna fine-tuning to the viral launch in April 2023, where anonymous side-by-side battles let the community decide which model is better. To address biases like length preference, they developed Style Control, a logistic regression method that adjusts for confounders such as response length and markdown formatting. They address the controversy around labs testing multiple private models, arguing that selection bias is empirically small and the live benchmark self-corrects over time. They also discuss RouteLLM for cost-performance routing, the graduation of ChatBot Arena from LMSys to support new projects, and call for community help with red teaming, vision modalities, and implementing a REPL for coding evaluations.