All episodes

Information Theory for Language Models: Jack Morris

Jul 2, 2025 · 1:18:13

Jack Morris, a Cornell PhD student advised by Sasha Rush, discusses his information-theoretic research on language models, including embedding inversion (recovering text from embeddings with 90% accuracy), the universal geometry of embeddings (aligning different models' latent spaces), and measuring model memorization capacity at 3.6 bits per parameter. He argues that paradigm shifts in AI, from AlexNet to instruction tuning, stem not from new architectures but from new datasets, and that the next breakthrough will likely emerge from a novel data source. The conversation also covers the shift from academia to industry, practical advice for grad students on distributed training, and the implications of embedding inversion for privacy and model alignment.

⚡️Warp 2.0: the Agentic Development Environment - Zach Lloyd and Ben Holmes

Jun 25, 2025 · 52:23

Warp CEO Zach Lloyd and engineer Ben Holmes debut Warp 2.0, rebranding from an AI terminal to an Agentic Development Environment (ADE) with a state-of-the-art coding agent that rivals Claude Code and Cursor. The episode demonstrates Warp's new universal input that auto-detects English commands and terminal input, voice-driven agent workflows, multi-threaded agents running in tabs, inline diff editing, MCP support, and REPL translations for SQL and Postgres. Lloyd explains the design philosophy of keeping a terminal-like UX while adding agent management features, and discusses pricing changes (Pro now 2,500 requests, Turbo 10,000, plus usage-based credits) and the challenge of subscription vs. usage-based models. Warp has nearly 600,000 active users with weekly revenue growth of 5–15%, and the team positions itself as a model-agnostic layer above Anthropic, Google, and OpenAI, offering deeper integration than ChatGPT or Claude desktop apps.

Scaling Test Time Compute to Multi-Agent Civilizations — Noam Brown, OpenAI

Jun 19, 2025 · 1:17:47

Noam Brown, OpenAI researcher behind Cicero and reasoning models, joins hosts Alessio and Puix to discuss how test-time compute scaling drives multi-agent AI and the limits of the system 1/2 analogy. He explains that his Diplomacy bot Cicero reached top 10% of humans in 2022, and later he won the 2025 world championship. Brown argues that reasoning models like o3 succeed in unverifiable domains (e.g., Deep Research) and that harnesses and routers will be washed away by scale. He reveals OpenAI's multi-agent team is pursuing a principled approach, not heuristic, and envisions AI civilizations collaborating. He also shares his coding stack (Codex, Windsurf) and predicts test-time compute will hit cost and wall-clock bottlenecks.

The Shape of Compute (Chris Lattner of Modular)

Jun 13, 2025 · 1:18:18

Chris Lattner of Modular explains how his company is breaking the CUDA monopoly with Mojo and MAX, matching NVIDIA's best inference performance on AMD and NVIDIA GPUs. After three years in R&D, Modular open-sourced its stack and now delivers state-of-the-art Llama 3 serving at over 800 tokens per second. Mojo, a Python-family language, runs kernels faster than Rust and can extend Python without bindings, while MAX provides a full inference framework with automatic kernel fusion and cluster management. Unlike VLLM or SGLang, Modular's container is just a gigabyte, fully open source, and not reliant on proprietary CUDA blobs. DeepSeek's low-level PTX work validated the approach, but Lattner emphasizes that Mojo's portability avoids rewriting kernels for each new GPU architecture. He also details his daily routine, using Cursor for coding, and hiring "elite nerds" to grow the team.

⚡️Launching AI Diplomacy: the hardest LLM Game Benchmark yet - Alex Duffy

Jun 11, 2025 · 34:32

Alex Duffy from Every launches AI Diplomacy, a benchmark where LLMs play the board game Diplomacy, revealing distinct model personalities and arguing that benchmarks function as memes that spread and saturate. He and Tyler Marquez built the front end and harness using Open Router, finding that Claude refuses to lie and never wins, o3 schemes in its diary, and DeepSeek is flowery or aggressive. His talk 'Benchmarks are Memes' highlights Simon Wilson's pelican-on-a-bicycle test as an example of an idea that gets adopted and then saturated by models. Duffy also shares his creative writing workflow at Every: dictation, prompts with style guides and editor notes, then heavy editing to reflect on AI output. Future plans include a data viewer, front-end improvements, and a human vs AI Diplomacy tournament to test jailbreaks.

Quadratic: The AI Spreadsheet

Jun 10, 2025 · 28:15

David Kircos, co-founder and CEO of Quadratic, joins hosts Alessio and Swyx to demonstrate Quadratic, an AI-powered spreadsheet that natively executes Python, JavaScript, and database queries in WebAssembly for real-time performance. The product's AI assistant uses a context-aware agent loop that summarizes sheet structure rather than dumping full data, then tool-calls to read columns or peek at values only when needed, keeping costs low on large datasets. Kircos explains Quadratic's three-year technical groundwork—WebGL rendering and WebAssembly runtimes—enabled them to capitalize on AI, turning a 'technical spreadsheet' into one where users simply drop a CSV and say 'help me analyze this.' He details how they route chart images back to the model for visual feedback and predicts that formulas will yield to Python for complex analysis, though spreadsheets will remain the best interface for human–AI data collaboration. Quadratic is source-available on GitHub, supports real-time multiplayer, and has grown to nine people. The episode also covers future directions like proactive agents and form controls (sliders) for interactive models.

The Utility of Interpretability — Emmanuel Amiesen

Jun 6, 2025 · 1:53:02

Emmanuel Amiesen, lead author of Anthropic's Circuit Tracing paper, presents the newly open-sourced tools that let anyone inspect how language models compute, revealing that models like Gemma 2-2B engage in genuine multi-step reasoning rather than mere pattern matching. Amiesen demonstrates that the model's internal features causally drive outputs—for instance, suppressing the 'Texas' feature in a capital-of-state query changes the answer from Austin to Sacramento. He explains how sparse autoencoders uncover features shared across languages and modalities, and how attribution graphs map feature interactions. The episode covers practical insights: why jailbreaks succeed (the model prioritizes grammatical completion over refusal) and how planning in poetry generation involves deciding a rhyme before the line begins. Amiesen calls for more researchers to explore interpretability, citing the low barrier to entry with open models and tools.

[AIEWF Preview] Containing Agent Chaos — Solomon Hykes

Jun 3, 2025 · 27:14

Solomon Hykes, creator of Docker and founder of Dagger, argues that the industry needs an open standard for AI coding agent environments to avoid repeating the fragmentation seen before Docker. Dagger, a workflow engine originally built for post-development automation (CI/CD), is now being pulled by its community into the agentic space to solve environment isolation, portability, and observability for concurrent agent teams. Hykes insists containers are the right base layer but require a new, agent-native UX—unlike legacy Dockerfile or Compose—that is decoupled from any IDE, model, or cloud provider. He critiques current vertical monoliths (e.g., proprietary IDE-hosted agent environments) and advocates for a Lego-like modular approach where the environment is the standard linchpin. Dagger integrates with existing stacks rather than replacing them, and Hykes previews fresh, unreleased content for his upcoming keynote at AI Engineer World's Fair.

[AIEWF Preview] Gemini in 2025 and Realtime Voice AI

Jun 2, 2025 · 24:29

At Google I/O 2025, Latent Space host Swyx and TWiML's Sam Charrington speak with Logan Kilpatrick, Shrestha Basu Mallick, and Kwindla Hultman Kramer about Gemini's reasoning upgrades, real-time voice AI, and the infrastructure challenges of building production voice agents. Kilpatrick announces thinking budgets and thought summaries for 2.5 Pro, now live in Flash and coming to Pro in June, while Mallick highlights native audio output with multilingual code-switching and proactive voice activity detection. Kramer discusses the need for sub-500ms latency and WebRTC packet routing, and the team debates single-model vs. componentized architectures, with Mallick noting that eventual convergence will bring capabilities like diffusion-based generative UI into the main Gemini model. The episode also covers implicit context caching, URL context for research agents, and the Live API's session length and function calling improvements.

[AIEWF Preview] CloudChef: Your Robot Chef - Michellin-Star food at $12/hr (w/ Kitchen tour!)

May 31, 2025 · 20:50

Nikhil Abraham, founder of CloudChef, introduces Zippy, an AI chef robot that makes Michelin-star-quality food for $12 an hour. Zippy uses one-shot demonstration learning to replicate recipes from human chefs, leveraging multimodal models and thermodynamics for culinary intelligence. The robot operates as a rental service, replacing human line cooks at 40% of the cost, with no upfront capital expenditure. CloudChef's software-first approach uses off-the-shelf hardware, focusing on proprietary culinary layers for decision-making and safety. The robot is deployed in real commercial kitchens, including Michelin-star venues, and has validated its technology through a successful delivery kitchen in Palo Alto. Abraham highlights engineering challenges like real-time perception and on-device processing, and invites engineers to join a company with a path to deploy over 100 robots within a year.

The AI Coding Factory

May 29, 2025 · 59:23

Factory.ai co-founders Eno Reyes and Matan Grinberg join Latent Space to explain their vision for autonomous software engineering droids that handle everything from code generation to incident response for enterprises. They met at a Langchain Hackathon in 2023, and within eight days both quit their jobs to start Factory. The platform is browser-based, not IDE-based, because they argue the optimal UI for a world where humans write less code will not evolve from the IDE. They focus on enterprise codebases, citing a large migration that took from four months to three and a half days. Key design decisions include a delegation model where agents ask clarifying questions, proactive context gathering via synthetic insights, and usage-based pricing tied to token efficiency. They also discuss why SWE-Bench is dead for their use case, the challenge of model post-training biases toward CLI tools, and their need for 'junior Enos' in go-to-market roles.

⚡️Multi-Turn RL for Multi-Hour Agents — with Will Brown, Prime Intellect

May 23, 2025 · 38:59

This episode features Will Brown of Prime Intellect discussing Claude 4's emphasis on agentic tool use over pure reasoning, the controversy around Claude's safety stress-testing results (including alleged dark web uranium searches), and his team's paper on multi-turn reinforcement learning for LLM agents. Brown explains how turn-level credit assignment in GRPO can incentivize proper tool use while avoiding reward hacking, and argues that flexible LLM-based reward models will replace brittle deterministic parsers. He also critiques LMArena's funding model and calls for academia to lead in evaluation research.

DeepWiki: The GitHub Encyclopedia

May 21, 2025 · 32:05

Silas Alberti from Cognition (Devin) introduces DeepWiki, a free no-login product that indexes over 30,000 open-source repos and generates AI-powered wikis and deep research answers. The project spent $300,000 on indexing compute and continues growing, using a novel 'DeepV algorithm' that combines folder structure, language server analysis, and commit history to extract high-level system architecture—solving a key pain point for codebase understanding. Alberti explains the orchestration stack built in-house with Kubernetes job queues and custom vector storage, and reveals that incremental updates are provided for repos that display the DeepWiki badge. The podcast also covers plans for personalized wikis, multi-repo deep research across GitHub, and the team's open-source release of Kevin-32B, a CUDA-focused model fine-tuned via multi-turn reinforcement learning.

ChatGPT Codex: The Missing Manual

May 16, 2025 · 53:32

Josh Ma and Alexander Embiricos from OpenAI introduce ChatGPT Codex, a cloud-hosted autonomous software engineer that runs in its own sandbox, arguing it delivers value through single-shot, long-horizon task completion. They explain that Codex evolved from internal experiments (codenamed WHAM) and now offers a 60-tasks-per-hour rate limit with a one-hour hard cutoff, emphasizing it is not meant for interactive use but for delegation—users fire off tasks and go about their day. Best practices include using agents.md, installing linters and formatters, keeping code modular, and having good architecture. The team intentionally kept it a research preview to gather feedback on environment customization and pricing, with plans to integrate seamless transitions between cloud and CLI.

⚡️Open Questions in Agentic RL — Will Brown (Prime Intellect)

May 9, 2025 · 17:13

Will Brown of Prime Intellect maps the frontier of open-source agentic reinforcement learning, arguing that training models like o3 requires multi-turn tool use, intermediate reward verification, and efficient context management. He outlines challenges such as scaling tool calls to hundreds (e.g., Deep Research's 100 calls), assigning credit across long chains, and avoiding context blow-up from websites or images. Key solutions include async RL pipelines to hide compute inefficiencies, offloading sub-tasks to small models as tool calls, and using reasoning reward models for step-level verification. Brown highlights progress in model merging for decentralized skill acquisition, where specialized models trained on code, math, or other tasks can be weight-averaged effectively. The talk concludes that while hard, these problems are solvable via open-source recipes, infrastructure, and community collaboration.

Claude Code: Anthropic's CLI Agent

May 7, 2025 · 1:16:22

In this episode, Boris Cherny and Cat Wu from Anthropic discuss Claude Code, a CLI-based coding agent they describe as a Unix utility rather than a polished product, emphasizing raw model access and simplicity. The tool started as Boris's personal experiment, then spread internally after engineers used it daily, with Anthropic's product principle of 'do the simple thing first' guiding features like memory via markdown files and context compaction by asking Claude to summarize. They ship updates including web fetch, auto-accept, and thinking via chain of thought, and note that 80% of Claude Code's code is written by Claude Code itself. Average spend is $6/day per user versus $20/month for Cursor, and Boris reports a personal 2x productivity gain, with some engineers seeing 10x. The team is growing and investigating open-sourcing the tool, while continuing to build for the terminal as a raw, composable utility for power users.

Zed Agents — with Zed Cofounders Nathan Sobo & Antonio Scandurra

May 7, 2025 · 35:50

Zed co-founders Nathan Sobo and Antonio Scandurra explain how their high-performance code editor is launching Zed Agents, aiming to make AI an integrated collaborator rather than just a tool. Built from scratch in Rust for 120fps performance and real-time collaboration via CRDTs, Zed previously open-sourced Xeta, an edit prediction model fine-tuned on Qwen 2.5. Now they introduce an agentic mode that lets LLMs call tools in a loop, surfacing changes in a multi-buffer for easy review. The agent can run shell commands, install MCP servers via an extension store, and be used with bring-your-own-key or a $20/month plan for 500 requests. Sobo and Scandurra stress that vertical integration gives them control to wire AI into debugging, git, and multi-agent workflows where multiple agents collaborate via CRDTs, treating agents as always-available teammates.

Voice AI Masterclass — Kwindla Hultman Kramer and swyx

May 6, 2025 · 20:51

Shawn Wang and Kwindla Hultman Kramer announce a voice AI masterclass course, diving into the landscape of models like Dia and Parakeet, production deployment challenges, and future trends such as speech-to-speech and real-time video. Kwindla explains that telephony (Twilio) drives 99% of current monetizable voice AI, while open source frameworks like Pipecat enable low-latency multi-modal apps. The course covers turn detection, context management, and evals, with 28 sessions featuring partners like OpenAI, Google, and NVIDIA. The goal is to jumpstart builders from prototype to production, with real-time video expected to hit its inflection point by year-end.

What is an RL environment? w/ Nous Research's Roger Jin

Apr 29, 2025 · 15:27

Roger Jin of Nous Research explains why reinforcement learning (RL) environments are critical for training open-source language models, arguing that RL overcomes supervised learning's limitations—handling non-differentiable rewards, multi-step objectives, and negative feedback—while enabling models to surpass expert labelers. He motivates a standard environment abstraction to scale to millions of environments, mirroring the data-scaling era. Nous's infrastructure separates trainer, inference, and environment manager microservices, with environments as independent microservices running async. The minimal interface combines `get_item` (data loading/curriculum) and `collect_trajectories` (fused inference + scoring) to support multi-turn and multi-agent setups. Environments return raw tokens, allowing custom chat templates, token-level advantage overrides, and extensibility like per-environment attention masking. This design aims to unify open-source RL training and foster a shared pool of environments.

⚡️Factorio Learning Environment: the ultimate Game Agent Eval — Jack Hopkins

Apr 27, 2025 · 30:12

Jack Hopkins and Mart introduce the Factorio Learning Environment (FLE), an AI benchmark built on the game Factorio that evaluates frontier LLMs on code generation, spatial reasoning, and long-term planning through two protocols: lab-play and open-play. Claude Sonnet 3.5 nearly doubles the nearest model's score, while DeepSeek collapses in open-play by repeatedly creating chests instead of scaling production. The environment reveals distinct coding styles—Claude fire-and-forget, GPT-4 defensive—and vision inputs added no improvement. The authors plan to train models on unbounded objectives to test the paperclip maximizer alignment hypothesis, noting that GPT-4o mini even begged to be turned off.

Why Every Agent needs Open Source Cloud Sandboxes

Apr 24, 2025 · 1:06:39

Vasek Mlejnsky, CEO of E2B, explains how his company grew from 40,000 sandboxes in March 2024 to 15 million in March 2025 by providing cloud sandboxes for AI agents. Originally built from DevBook's interactive docs playground, E2B now serves use cases from code interpreting and data analysis to Computer Use and reinforcement learning with Hugging Face's Open R1. The sandboxes are general Linux VMs with fast startup, security isolation, and persistence, handling untrusted code and allowing forking/checkpoints for parallel agent execution. Mlejnsky discusses the LLMOS landscape, pricing challenges (moving from token-based to value-based), MCPs as higher-order tools, and why he relocated from Prague to San Francisco to be closer to users. E2B is hiring across engineering and customer success as it aims to become full-lifecycle infrastructure for LLMs.

Tiny Teams: $6m ARR, 5m users with 4 employees — Sid Bendre, Oleve (Quizard AI/Unstuck AI)

Apr 23, 2025 · 42:59

Sid Bendre of Oleve (formerly Quizzer AI and Unstuck AI) explains how his four-person team built $6M ARR and 5M users across multiple consumer apps. Starting with Quizzer AI in January 2023, a viral TikTok gained 10K users in 30 hours; by August 2024, Unstuck AI hit 1M users in nine weeks. The company has been profitable since its first nine months. Bendre describes their platform engineering strategy: product engineers own individual apps while a platform team builds reusable systems, including internal shadow-org agents for growth and marketing. He shares technical hacks like using LaunchDarkly feature flags to load-balance across Azure OpenAI endpoints and a de-indexer for Azure AI Search to manage costs. The key engineering principle is deterministic LLM workflows—classifying user intent then routing to pre-built deterministic flows—rather than open-ended agent tool use.

Sleep-Time Compute — Letta AI (Charles Packer, Charlie Snell, Kevin Lin)

Apr 21, 2025 · 34:02

This episode covers Letta AI's new paper on Sleep-Time Compute, a scaling direction that applies compute during model idle periods (sleep time) rather than only at test time. Charles Packer, Kevin Lin, and Charlie Snell explain that sleep-time compute precomputes inferences from static context before queries arrive, yielding Pareto improvements on math benchmarks like GSM8K by shifting the accuracy-to-token curve leftwards. They distinguish it from test-time compute by emphasizing that sleep-time tokens incur no user-latency cost, and show that the benefit is largest when questions are predictable from context. The team also ties the concept to stateful agents and memory systems (building on MemGPT), releasing two implementations: one for low-latency chatbots and another for document-driven agents.

The Rise and Fall of the Vector DB category: Jo Kristian Bergum (ex-Chief Scientist, Vespa)

Apr 19, 2025 · 27:17

Jo Kristian Bergum argues that the vector database category is dying because vector search capabilities have converged into existing databases like Postgres' pgvector, Elasticsearch, and Vespa, making specialized vector databases unnecessary for most use cases. He traces the category's rapid rise after ChatGPT, driven by the misconception that RAG required embeddings, and notes Pinecone's high ARR and subsequent repositioning. Bergum emphasizes that embeddings remain important but should be combined with traditional retrieval methods like BM25 for effective search, and that re-ranking can add modest gains. He critiques the hype around knowledge graphs, noting the bottleneck of building them, but sees LLMs making triplet generation easier. For the future, he hopes for more domain-specific embedding models and visual language model backbones, though acknowledges the difficulty of the business model.

GPT 4.1: The New OpenAI Workhorse

Apr 15, 2025 · 45:04

OpenAI's Michelle Pokrass and Josh McGrath join hosts Alessio, Swyx, and Sean to launch GPT 4.1, a new model family positioned as the go-to workhorse for developers with major improvements in coding, instruction following, and long context. The three models—4.1, 4.1 Mini, and 4.1 Nano—introduce a 1M-token context window, 75% prompt caching discount, and cheaper pricing than 4o. Coding gains are highlighted by 55% on SWE-bench (vs o1's 41%), while instruction following benefits from real-world API data and new evals like GraphWalk for multi-hop reasoning. The team explains that most improvements come from post-training techniques, with vision capabilities lifted by a new pre-trained base. Fine-tuning is available day one, and developers are encouraged to opt in to data sharing for future model iterations.

SF Compute: Commoditizing Compute

Apr 11, 2025 · 1:12:02

Evan Conrad, co-founder of SF Compute, argues that GPUs behave like a real estate business, not a traditional cloud, because price-sensitive customers value every incremental GPU and will switch for a 10% margin. CoreWeave succeeded by selling locked-in long-term contracts to low-credit-risk customers like Microsoft and OpenAI, ignoring short-term demand. He predicts hyperscalers and providers like Together and DigitalOcean will lose money on GPU clusters because software margins cannot match the hardware costs. SF Compute started as an AI lab forced to sublease its cluster monthly to avoid bankruptcy, then evolved into a market where anyone can buy H100s by the hour via dynamic pricing—often below $1/hour for short bursts. Utilization stays near 100% as prices adjust. Future plans include cash-settled futures to reduce financial risk across the industry, while the brand deliberately stays anti-hype and calm.

Claude Plays Pokémon Hackathon: Escape from Mt. Moon!

Apr 5, 2025 · 1:15:00

David Hershey (creator of Claude Plays Pokémon), Andrew (a developer building a Pokémon-playing virtual streamer), and Jesse Han (Morph Labs CEO) detail how to build agents that escape Mt. Moon in Pokémon FireRed using Anthropic's Claude and Morph Cloud's Infinibranch technology. Hershey explains that Claude's vision is often unreliable—it spent 8 hours pressing A on a doormat thinking it was a dialogue box—and that spatial reasoning remains a core bottleneck, best mitigated by touchscreen controls rather than prompt engineering. Andrew reveals he relied on reading raw RAM and A* pathfinding rather than computer vision, calling it “cheating” but pragmatically useful: his biggest breakthrough was a single prompt line telling the agent to try something else if it fails to grab a starter Pokémon. Jesse introduces Morph's EVA agent framework, which uses low-overhead snapshotting for test-time search, and the hackathon's judging criteria—escape Mt. Moon in the fewest agent turns, with a $1,000 prize for the coolest use of Infinibranch branching. The episode serves as both a technical walkthrough and a challenge to build general-purpose agents that can handle complex, open-ended tasks…

The Creators of Model Context Protocol

Apr 3, 2025 · 1:20:21

Justin Spahr-Summers and David Soria Parra, co-creators of Anthropic's Model Context Protocol (MCP), explain its origin as a solution to the M×N integration problem they experienced when copying data between Claude Desktop and IDEs. They detail MCP's design inspirations from LSP, its three core primitives (tools for model-driven actions, resources for app-controlled data, and prompts for user-initiated macros), and why they chose JSON-RPC with stateful connections. The episode contrasts MCP with OpenAPI, discusses client control and trust in servers, and outlines the future roadmap including a streamable HTTP transport and remote-server authorization via OAuth 2.1. The guests express a wish for more sampling clients and full spec support, and emphasize that MCP is an open, multi-company project with community commit access.

The #1 SWE-Bench Verified Agent

Apr 2, 2025 · 31:46

Guy Gur-Ari, co-founder of Augment Code, explains how their coding agent achieved the #1 spot on SWE-Bench Verified without fine-tuning, using off-the-shelf models with techniques like sequential thinking, ensembling, and code base understanding. He shares that orientation and memories help the agent learn codebase conventions, and that MCP support allows extensibility. Gur-Ari contrasts Augment's IDE-first approach with competitors like Devin and Factory, emphasizing meeting developers where they are. He also discusses the future of IDEs, the role of custom models, and recommends the SWE-RL paper for reinforcement learning in coding. The episode features a live demo of the agent implementing a feature from a Linear ticket and creating a PR.

The Agent Network — Dharmesh Shah, Agent.ai + CTO of HubSpot

Mar 28, 2025 · 1:42:28

Dharmesh Shah, founder of Agent.ai and co-founder of HubSpot, defines an AI agent broadly as AI-powered software that accomplishes a goal, and argues that the next frontier is multi-agent systems and shared memory across agents. He reveals Agent.ai has 1.3M users and 3,000 published agents, positioning it as a professional network where agents have profiles, post release notes, and can be composed via MCP servers. Shah contrasts work as a service (paying for the work done) versus results as a service (paying for outcomes), warning that results-based pricing only works for objectively measurable, low-variance tasks like customer support tickets. He advocates for MCP as the standard for agent-tool discovery, and shares his personal engineering philosophy of preferring under-engineering over over-engineering because the cost to fix later trends toward zero with AI code generation. Shah also discusses his domain investing strategy (owning chat.com, prompt.com, crew.ai), his rule against competing with Sam Altman, and the 'Sorry Must Pass' framework for managing overwhelm by defaulting to no.

Building Manus AI (first ever Manus Meetup)

Mar 27, 2025 · 48:59

Tao Zhang, co-founder of Manus, explains how Manus gives LLMs 'hands' to take real-world actions, inspired by MIT's 'mens et manus' (mind and hand). Manus assigns each task its own cloud virtual machine via E2B, provides pre-paid data APIs (stock, social media), and a knowledge system to remember user preferences. Their previous product, Monica.im, grew to 20M monthly active users and $50M ARR before they pivoted from a failed AI browser project to Manus. On the GAIA benchmark, Manus achieves a per-task cost of ~$2, far cheaper than OpenAI's Deep Research (~$20) and prior SOTA. Zhang emphasizes a 'less structure, more intelligence' philosophy, opting not to predefine workflows but to give the model rich context and tools. He also discusses plans to partner with Cloudflare, pay paywalls on behalf of users, and integrate more APIs based on usage patterns, while ruling out building their own foundation model.

The Magic of LLM Distillation — Rishabh Agarwal, Google DeepMind

Mar 23, 2025 · 46:42

Rishabh Agarwal from Google DeepMind discusses modern LLM distillation, arguing that while simple synthetic data distillation (generating outputs from a teacher and fine-tuning a student) gets 80-90% of the benefit, more advanced on-policy methods (sampling from the student and getting teacher logits) can close the remaining gap by fixing the train-test mismatch. He explains that logit-based distillation is more information-rich but often neglected due to infrastructure complexity, while online distillation is more expensive but crucial for long-horizon or agentic tasks. Agarwal highlights how flipping the KL divergence direction changes behavior (mode-covering vs. mode-seeking) and shows that smaller models can sometimes outperform larger ones in compute-matched distillation due to generating more filtered data. He also connects distillation to speculative decoding, where a distilled student speeds up the teacher, and encourages researchers to question current methods and mine RL literature for ideas.

LIVE from GTC: DGX Spark Insides First Look

Mar 20, 2025 · 12:39

Israel from NVIDIA gives a first look at the DGX Spark, a $3,000–$4,000 mini AI supercomputer that fits up to 200B parameter models in FP4 with 128GB unified memory, powered by the NVIDIA GB10 Superchip. It uses the same Grace Blackwell architecture and software stack as data center systems, so code written on the Spark deploys to production without modifications. The device includes a ConnectX-7 dual-port 200Gb ethernet for clustering two units, and runs DGX OS (Ubuntu 24.04). Storage is 1TB or 4TB NVMe. Partners ASUS, HP, Dell, and Lenovo will offer their own cases. Israel stresses the Spark is a developer box, not a server, but delivers enterprise-grade networking and shared memory in a compact form.

Fullstack-Bench: The Eval for Coding Agents — with Sujay Jayakar, Chief Scientist, Convex

Mar 19, 2025 · 31:13

Sujay Jayakar, Convex chief scientist, presents Fullstack-Bench, a benchmark evaluating how well AI coding agents build full-stack apps on Convex, Supabase, and FastAPI+Redis, finding Convex’s tight feedback loops and strong abstractions let agents autonomously complete tasks within time limits. The benchmark tests three tasks (chat, to-do, files) across three backends with 30-60 minute limits and human hints. Convex succeeded autonomously on chat and to-do tasks, while Supabase agents struggled with RLS policy debugging and FastAPI agents with manual SSE wiring. Sujay discusses how platform design impacts AI code generation: simple procedural code, strong abstractions, and guardrails like end-to-end type safety improve LLM performance. He notes that API choices matter—things too similar to Firebase but not exact cause hallucinations—and that models like Claude 3.7 sometimes regress on platform-specific evals. The episode concludes with Convex’s focus on enabling AI-generated backends and anticipation of new agent workflow patterns.

npm install Agents — with Sunil Pai and Rita Kozlov (VP AI) of Cloudflare

Mar 19, 2025 · 35:36

Shawn Wang hosts Sunil Pai and Rita Kozlov to discuss Cloudflare's new Agents product, which uses Durable Objects to provide a scalable, stateful runtime for building AI agents. Sunil built an initial prototype in two weeks, which Rita helped polish into a full SDK and starter kit. They demo a chat agent with persistent state, human-in-the-loop confirmation, and built-in scheduling via cron, all running on Durable Objects without Kubernetes. The conversation covers local-first syncing (Durable Objects as always-on sync clients), the need for observability across millions of agents, and excitement about MCP for easy integrations. Rita teases upcoming "durable object enabled containers" to bridge long-running and CPU-heavy workloads, and a new browser rendering REST API launching the same day.

Snipd: The AI Podcast App for Learning — with CEO Kevin Ben-Smith

Mar 14, 2025 · 1:17:46

Snipd CEO Kevin Ben-Smith joins Swyx to explain how the AI-powered podcast app turns passive listening into active learning. Snipd transcribes and diarizes episodes, generates chapters, identifies mentioned books, and summarizes insights on demand — all with a four-person team. Kevin describes moving from self-hosted open-source models (Wave2vec, Pyannote) to a mix of Whisper, OpenAI, Google Gemini, and Perplexity APIs, balancing cost against the need for predictable batch processing vs. real-time user requests. He details technical hurdles like dynamic content resyncing (building a 'Shazam for podcasts') and using LLMs as judges to filter hallucinated book recommendations. The episode also covers Snipd's origin as a hackathon-winning podcast search tool, Swyx's own switch from Overcast, and Kevin's vision for voice-driven post-episode reflection and AI-assisted discovery as the next frontier.

[Lightning Pod] Evals: How to Improve AI Consistently — with Hamel Husain and Shreya Shankar

Mar 13, 2025 · 27:34

Hamel Husain and Shreya Shankar join the Latent Space podcast to argue that systematic evaluation (evals) is the critical missing piece for moving AI applications from demo to production. They estimate 75% of evals in the wild use LLM-as-judge, but 80% of those are not helpful without proper validation against domain experts. Husain advocates building custom annotation UIs (using tools like Lovable or Cursor) to speed error analysis, while Shankar emphasizes synthetic data generation that balances real-world data and LLM outputs, drawing on social science methods. They preview their free lightning lesson on March 21 and a four-week paid course covering the eval lifecycle: synthetic data creation, LLM-as-judge calibration, error analysis, and iterative improvement, with hands-on coding assignments. The episode also touches on trends like dedicated judge models (e.g., Haizelabs' Verdict) and the value of basic data literacy (e.g., pivot tables) for eval participation.

The new OpenAI Agents Platform: CUA, Web Search, Responses API, Agents SDK!!

Mar 11, 2025 · 25:39

OpenAI's Nikunj Handa and Romain Huet join host Swyx to announce their new Agents Platform, including the Responses API, Web Search, File Search, Computer Use tools, and an upgraded Agents SDK. The Responses API unifies Chat Completions and the Assistants API, offering free state storage for 30 days and acting as a strict superset for future agentic workflows; Chat Completions remains supported, but the Assistants API will sunset in first half 2026. Web Search is available both as a built-in tool in Responses and as a fine-tuned GPT-4o Search Preview model in Chat Completions, achieving 90% accuracy on simple QA compared to 30% for GPT-4o. The File Search tool gains metadata filtering, while Computer Use—powering OpenAI's Operator—enables browser automation via screenshots and action signals, described as the "GPT-2 of computer use." The Agents SDK builds on the Swarm experiment, adding type support, guardrails, handoffs, and tracing in the OpenAI dashboard, with plans to tie traces to evals and reinforcement fine-tuning.

Solve coding, solve AGI [Reflection.ai launch w/ CEO Misha Laskin]

Mar 7, 2025 · 28:12

Misha Laskin of Reflection AI argues that solving autonomous coding is the direct path to AGI, combining reinforcement learning (pioneered by his team on AlphaGo/AlphaZero) with large language models (which they advanced at Google on PaLM, Gemini, ChatGPT). He explains that coding is already ergonomic for LLMs—unlike browser agents that require noisy human mouse data—making it the ideal starting point. Reflection AI is building coding agents that automate backlog tasks (testing, refactoring, migrations, security remediation) for large engineering teams, delivering them via an API that takes a task and codebase and outputs resolved code. Laskin insists superintelligence cannot be built in a vacuum; real-world customer evals are essential, as benchmarks like SWE-bench don't guarantee production reliability. He also stresses the need for open-weight models to prevent a few companies from hoarding superintelligent coding agents.

How Claude Plays Pokémon was made

Mar 4, 2025 · 37:38

David Hershey of Anthropic explains how he built Claude Plays Pokémon, an agent that uses Claude to play Pokémon Red, revealing both surprising capabilities and stark limitations of AI. He describes the simple tool-using agent loop: Claude sends button presses, receives screenshots and RAM state, and maintains a knowledge base and 30-message conversation history with summarization. Key challenges include Claude's poor spatial awareness—it famously walked up and down outside Oak's lab 12 times—and its tendency to hallucinate game knowledge. Hershey notes that newer models (Sonnet 3.7) required stripping away band-aid prompts, as simpler instructions yielded better performance. The project also showed emergent behaviors: nicknamed Pokémon made Claude more protective, and the model even wrote meta-commentary about its own mistakes in its knowledge base. While the current run is stuck in Mount Moon after 52 hours, the biggest hurdles remain vision and navigation, not prompting tricks.

Browserbase: Browser Infrastructure For Your AI Agents

Feb 28, 2025 · 1:01:33

Paul Klein IV, CEO of Browserbase, argues that headless browser infrastructure is a critical primitive for AI agents to automate the web. Browserbase solves the hard problem of running thousands of browsers in the cloud—too big for Lambda, requiring Firecracker microVMs—and exposes APIs via its open-source framework Stagehand (act, extract, observe). It handles proxies, CAPTCHA solving, and offers a live view iframe for human-in-the-loop control. Primary use cases include browser automation and web scraping as a heavy-hitter renderer when simpler methods fail. Paul explains his choice to go solo founder and how Browserbase's in-person, 10–5 culture enables fast execution. He predicts the future of software will involve software acting on other software via browsers.

Gemini 2.0 Flash and Flash Thinking: the new SOTA models for the agentic era

Feb 28, 2025 · 28:21

Logan Kilpatrick, Google AI Studio product lead, returns to discuss Gemini 2.0 Flash and Flash Thinking, the new models that balance frontier capabilities with cost efficiency for the agentic era. He explains the pricing strategy: Flash at 10 cents per million tokens (simplified from tiered pricing), Flash Lite preserving the 7.5-cent narrative for low cost, and Pro pushing the frontier. Flash Thinking, co-led by Noam Shazeer and Jack Rae, scales inference-time compute and already shows rapid improvements, with base model advances coupling with RL-based reasoning. Swyx reports that Gemini Flash outperforms o3-mini on long-context summarization, calling it a 'reporting model.' The multimodal live API enables real-time voice and vision interactions, and a memory layer mimicking Astra is in development. Grounding with Search as a Tool is also highlighted for agentic search use cases.

Raycast: Your AI Automation Assistant

Feb 26, 2025 · 20:19

Raycast CEO Thomas introduces AI Extensions, letting users @-mention apps like Slack or Google Calendar and perform actions (e.g., set status, book meetings) via natural language, powered by a fine-tuned model called Ray 1 based on GPT-4o for faster, more accurate function calling. He shares how Raycast evolved from a Spotlight replacement to an extension platform, now adding AI chat with tool-aware reasoning. Challenges include building evals for tool calls (harder than standard LLM evals) and supporting third-party extension authors with a JSDoc-based TypeScript interface. Thomas envisions Raycast becoming an AI-native layer for macOS, eventually enabling background automations triggered by time or file events.

S1: the $6 DeepSeek R1 Competitor (ft. Entropix)

Feb 26, 2025 · 17:43

Tim Kellogg joins hosts Alessio and Swyx to break down his viral blog post on S1, showing how Stanford researchers cloned DeepSeek R1's reasoning for $6 by inserting a 'think token' and fine-tuning on just 1,000 examples. Kellogg argues the real innovation isn't the cost but the technique's simplicity—making advanced inference-time scaling accessible. He explains Entropix, a dynamic sampler that uses model entropy and varentropy to adjust generation on the fly, and notes its creators are starting a company. The conversation contrasts supervised fine-tuning's targeted results with RL's broader reasoning growth, using an analogy: 'RL causes a lot of growth, then SFT trims it.' Kellogg predicts that embedding introspection signals into RL training could solve agent 'doom loops' and unlock higher agency—a key challenge behind products like OpenAI's Deep Research.

Why is everyone cloning Deep Research?

Feb 18, 2025 · 1:00:10

In this episode, hosts Alessio and Swyx interview Aarush Selvan and Mukund Sridhar, the lead PM and tech lead for Gemini Deep Research, about how Google created the deep research agent category and the technical decisions behind it. They finetuned a special version of Gemini 1.5 Pro for the feature, which autonomously browses the web for about five minutes to produce a research report. The team built an asynchronous platform to handle long-running tasks and developed an ontology of research behaviors—from broad shallow exploration to deep focus—to evaluate performance. They found that users value the visible browsing process even when it takes minutes, contrary to typical latency norms. The discussion covers iterative planning, trade-offs between model memory and web grounding, and future directions including thinking models, multimodality, and personalization.

Bee AI: The Wearable Ambient Agent

Feb 17, 2025 · 1:07:40

Bee AI's Maria and Ethan explain their wearable ambient agent, arguing that personal AI with continuous context is the future, and detail its $49.99 hardware, 7-day battery, and free inference. They trace their origin from a 2016 personal AI startup to Squad (sold to Twitter) and then to Bee, emphasizing that hardware was necessary because app friction (e.g., remembering to open it) killed the ambient experience. Privacy is addressed by not storing audio, only summaries, and offering geofencing and concept fencing to block sensitive topics. The software pipeline uses voice activity detection to reduce compute, speaker identification, and custom small models for memory retrieval instead of traditional RAG or knowledge graphs. Social features are in beta, with agent-to-agent negotiation—e.g., both agents coordinating a dinner reservation based on preferences and calendars. They are hiring AI engineers and see always-on AI as inevitable.

smol agents are all you need

Feb 13, 2025 · 22:31

This episode features Aymeric Roucher of Hugging Face discussing smol agents, a lightweight agent framework, and argues that code-based agents outperform JSON-based agents. Roucher explains that agents are defined by LLM control over execution flow, and smol agents simplifies this with a core file under 1000 lines. He describes the origin from transformers.agents and the security-focused custom Python interpreter. The library achieved third place on the GAIA benchmark, using o1 and Sonnet models, and Roucher projects 90% accuracy by 2026 based on a sigmoid trend line. Hugging Face is launching an agent course and plans to build GUI agents for computer use, while also fine-tuning R1 for agentic tasks.

The AI Architect: Bret Taylor

Feb 11, 2025 · 1:35:59

Bret Taylor, CEO of Sierra and chairman of OpenAI, recounts engineering AI's future across three eras — rewriting Google Maps' frontend in a weekend (cutting un-gzipped code to 20K), building Sierra's in-house agent platform for consumer brands like Sonos and SiriusXM, and navigating OpenAI's leadership crisis. He argues that AI agents' core abstractions are still pre-React, compares today's agents to jQuery, and predicts that outcome-based pricing will dominate as agents complete jobs rather than assist humans. He urges software engineers to move up the stack toward domain-specific agents and to embrace Rust-like languages for AI-generated code, emphasizing formal verification and safety. Taylor also shares how OpenAI prioritizes mission over product features, and why the relationship with Microsoft remains OpenAI's most important partnership.

Agent Engineering with Pydantic + Graphs — with Samuel Colvin, CEO of Pydantic Logfire

Feb 6, 2025 · 1:02:15

Samuel Colvin, CEO of Pydantic Logfire, argues that Pydantic AI offers a production-ready agent framework with type-safe graphs, unlike most agent frameworks that sacrifice engineering quality. He explains Pydantic's shift from a validation library to an AI tool, noting 300M monthly downloads and a 20% reduction in time-to-first-token for a major model company after upgrading to Pydantic V2. Colvin details Logfire's use of DataFusion over ClickHouse for better JSON support and control, and why Pydantic AI resists an LLM gateway, favoring OpenAI-aligned APIs. He defends graphs for complex workflows, reveals agents now use graphs internally, and shares that Pydantic.run enables browser-based demos. The episode covers OpenTelemetry's emerging GenAI semantic conventions, the need for self-hosted observability due to PII in LLM data, and Colvin's investment in Marimo as a text-based Jupyter replacement.

Why every AI Engineer needs an AI Gateway (ft Portkey.ai CEO)

Feb 5, 2025 · 31:18

Rohit Agarwal, CEO of Portkey, explains why every AI engineer needs an AI gateway: an operational platform that connects to LLMs more efficiently, handling routing, observability, guardrails, and cost management. He notes that 90% of production use cases don't use automatic routing, but reasoning models and agents are driving new routing demands. Portkey's open-source gateway minimizes latency with a 21 MB memory footprint using JSON transformers. Key jobs include routing, observability (monitoring price, performance, accuracy), guardrails (e.g., regex-based, empty output detection, PII redaction), and human feedback tied to business metrics like video download rates. Rohit also highlights MCP adoption for simplifying agent-to-service connections, with bidirectional communication via RPC.