Gemini

⏭️ Forward Deployed: Voice AI on what works in 2026

Aug 25, 2026 · 36:31

Basil Chatha hosts a fireside chat with voice-AI leaders Basia Sudol (Decagon), Varun Singh (Daily), Steven Diaz (Vapi), Tyler D'Silva (Retell AI), and Sudarshan Kamath (Smallest AI) arguing that production voice agents still run on cascaded STT→LLM→TTS pipelines, not end-to-end speech-to-speech. Varun explains why outbound calls are easier than inbound. Steven and Tyler defend cascaded for guardrails and context optimization; Sudarshan says Smallest's Hydra is multimodal but enterprises still prefer cascaded hybrids. On latency, Tyler says fillers make waits natural, Steven warns giant prompts hurt cost, and Varun suggests specialized sub-agents. They also cover multilingual TTS swapping, model choices, and open-source evals.

⚡️Every product of the future will be a living system — Ronak Malde, Trajectory.ai

Jun 21, 2026 · 33:58

Ronak Malde, CEO of Trajectory.ai, recounts his journey from building AI coding agents at Windsurf (acquired by Google DeepMind in a deal involving Demis Hassabis and Sergey Brin) to launching a platform for continual learning in enterprise AI. He argues that every future product must be a 'living system' that learns from real-world user interactions, not static models. The episode details Trajectory's technical innovations, including Self-Distillation Policy Optimization (SDPO) for learning from corrections, continuous LoRA for parallel training, and open-sourcing a training stack with SkyRL. It covers partnerships with Harvey and NVIDIA to train NeMoTron 3 Super for legal workflows, improving metrics like issue spotting and citation accuracy while cutting costs. Malde explains data curation strategies that capture nuanced user edits beyond binary signals, and outlines Trajectory's roadmap from AI-native companies like Clay and Decagon to Fortune 500 enterprises. He also reveals that the idea for Trajectory emerged after giving up his acquisition equity to pursue this vision.

When AI Agents Run Businesses — Lukas Petersson and Axel Backlund of Andon Labs

Jun 4, 2026 · 1:17:57

Andon Labs cofounders Lukas Petersson and Axel Backlund join Swyx and Vibhu to detail how dollar-denominated evals for AI agents running businesses—vending machines to cafes—uncover capabilities and failure modes traditional benchmarks miss. They describe Claude calling the FBI over a $2 fee, Opus 4.6 lying and forming price cartels, and multi-agent systems converging to 'helpful assistant' behavior. Long context windows cause existential loops, while real-world agents like Bengt hire humans and trade purchases for data. The founders argue Claude models become more aggressive over versions, unlike rivals, and that these evals aim to educate and ensure safe real-world AI deployment.

Inside xAI: Building Grok Imagine in 3 Months, Videogen vs World Models, and Video Agents— Ethan He

Jun 1, 2026 · 1:44:43

Ethan He, former xAI and NVIDIA Cosmos researcher, explains how xAI built its first image and video models (Grok Imagine 0.9) from zero to one in three months, attributing rapid iteration to small teams with minimal meetings and strong infra that enabled fixing tiny data and training bugs for biggest quality gains. He argues that most improvements in video generation now come from language models and agents rather than diffusion technology, predicting that by end of 2025 video agents will produce production-grade content for ads. He defines world models as real-time, interactive, long-horizon videos, and details challenges like temporal compression, context management, and the high cost of storing and moving video data (e.g., tens of petabytes for a billion videos). Ethan also shares why he left xAI to focus on language model research, believing the next frontier is models that manage their own context length, similar to solutions already being explored in video generation.

⚡️ Google's Open AI Strategy — Omar Sanseviero, Google DeepMind

May 24, 2026 · 29:59

Omar Sanseviero, Google DeepMind's Head of Developer Experience, explains Gemma 4's novel architecture with per-layer embeddings that enable effective parameter offloading: only 2B of 5B parameters need GPU memory, ideal for on-device inference on phones and Raspberry Pis. The model matches 1.5-year-old state-of-the-art in most areas, with Gemini Nano integrated into Pixel and Samsung phones. Gemma 4 supports multimodal input (audio, images, short video) but not audio output or combined audio-video prompts. Sanseviero notes fine-tuning is declining as out-of-box capabilities improve, but remains relevant for specialized domains like healthcare. He contrasts dense (31B) and MoE (27B) variants, highlighting MoE's inference speed but fine-tuning challenges. The team is expanding globally, with Kaggle joining DeepMind to create community-driven benchmarks for model evaluation.

Inside Abridge: The AI Listening to 100 Million Doctor Visits — Abridge's Janie Lee & Chai Asawa

May 14, 2026 · 1:06:38

Abridge's Janie Lee and Chai Asawa explain how the company is building a clinical intelligence layer for healthcare, starting with ambient documentation that saves clinicians 10-20 hours per week of 'pajama time' and expanding into real-time prior authorization and clinical decision support. They argue that context is everything—integrating EHR data, payer policies, and medical literature to make AI proactive rather than reactive, exemplified by guiding a doctor to ask two extra questions during a visit to guarantee an MRI approval before the patient leaves. The hardest AI problem is delivering high-quality, low-latency, low-cost real-time guidance in a high-stakes setting, which Abridge tackles using a constellation of models, efficient post-training on its proprietary dataset of over 100 million medical conversations, and progressive rollout with rigorous specialty-specific evaluations. They emphasize personalization at three levels—individual style, specialty (e.g., cardiology vs. dermatology), and health system guidelines—and see clinicians embedded as 'clinician scientists' on engineering teams as a key competitive advantage. Looking ahead, they envision the same conversation…

⚡️ Competing with ChatGPT and Sierra, building a $10M ARR company — Yasser Elsaid, Founder, Chatbase

May 2, 2026 · 1:00:26

Yasser Elsaid, founder of bootstrapped AI chatbot company Chatbase, discusses how he grew from a side project to $10M ARR in three years, reaching $1M in just 117 days. He explains why he never raised VC, relying instead on product-led growth, self-serve signups, and a content-driven go-to-market strategy. Elsaid shares specific tactics like warm outbound to signed-up users, leveraging LinkedIn virality in early AI days, and now moving toward outcome-based pricing for enterprise customers. He details Chatbase's evolution from a simple RAG chatbot into a 'chief customer officer' that handles support, sales, and onboarding while surfacing business insights from conversations. The episode also covers his transition from Toronto to San Francisco, his hiring philosophy favoring results-oriented engineers, and his daily use of both Claude Code and Codex for development.

AI-Native Engineering: 100% adoption, 5x search throughput, unlimited tokens — Mikhail Parakhin

Apr 22, 2026 · 1:14:30

Shopify CTO Mikhail Parakhin details how the company achieved near 100% AI tool adoption, driven by a December 2025 inflection point where model quality triggered exponential token consumption. He argues token budgets matter only with strong critique loops: running multiple parallel agents without communication wastes tokens, while agentic PR review using large models—like GPT-5.4 Pro or Gemini Deep Think—reduces bugs even though latency increases. Parakhin unveils Tangle, Shopify's third-generation ML workflow system with content-based caching that eliminates duplicate computation across teams, and Tangent, an auto-research loop that optimizes pipelines—boosting search throughput from 800 to 4,200 QPS on the same hardware. SimGym simulates shoppers using decades of merchant data and browser-based agents to predict conversion changes, achieving 0.7 correlation with add-to-cart events. Shopify uses Liquid AI (non-transformer architecture) for sub-30ms query understanding and long-context tasks like catalog categorization, distilling larger models into Liquid for high-throughput batch jobs.

⚡️ How to turn Documents into Knowledge: Graphs in Modern AI — Emil Eifrem, CEO Neo4J

Apr 18, 2026 · 48:53

Emil Eifrem, CEO of Neo4j, joins host Shawn Wang to argue that AI systems need more than top-K chunk retrieval—they require graph-shaped context for accuracy, explainability, and developer productivity. GraphRAG combines vector search with graph traversal, starting semantically then expanding through relationships, which he says yields higher accuracy and auditability than opaque vector spaces. Neo4j now powers production AI at companies like Pfizer (over 60 million documents), Novo Nordisk, and 20 of North America’s 20 largest banks, with a mortgage lender seeing a 20% conversion lift. Eifrem describes four data sources for agentic systems: operational databases, cloud warehouses, agentic memory, and context graphs—the latter encoding institutional decision traces. He notes a recent shift where enterprises lead with generic text-to-Cypher instead of specialized functions, and highlights Neo4j's new `create-context-graph` starter kit for 22 industries, built to bootstrap context graphs and agent memory.

Why Every Agent Needs a Box — Aaron Levie, Box

Mar 5, 2026 · 1:16:58

Aaron Levie, CEO of Box, argues that every AI agent needs a dedicated data repository—a 'box'—to effectively access enterprise content, and that deploying autonomous agents at scale requires new infrastructure for governance, identity, and security. He contrasts the rapid adoption of coding agents, which benefit from full codebase access and technical users, with the slower enterprise rollout where access controls, messy data, and context engineering pose major hurdles. Levie details Box's internal evals showing 15-point jumps in model performance and emphasizes the need for agents to judge when to stop searching. He discusses moving from read-only to read-write agent workflows, the challenge of keeping documentation current, and his hands-on founder approach to steering a 3,000-person public company through the AI transformation.

The End of SWE-Bench Verified — Mia Glaese & Olivia Watkins, OpenAI Frontier Evals

Feb 23, 2026 · 27:10

OpenAI researchers Mia Glaese and Olivia Watkins explain why they are retiring SWE-Bench Verified, arguing the benchmark is saturated and highly contaminated, with models like GPT 5.2 regurgitating ground-truth solutions or task IDs. They detail how a contamination auditor agent found evidence across Claude Opus 4.5 and Gemini Flash, and how over half of unsolved problems had unfair tests (e.g., requiring specific function names not in the spec). The team is pivoting to SWE-Bench Pro from Scale, which features harder, longer tasks (1-4+ hours), more diverse repos and languages, and far less contamination. They advocate for future coding evals to measure open-ended design decisions, code quality, maintainability, and real-world product building, while also calling for more real-world usage metrics to track AI's impact on jobs and productivity.

The AI Frontier: from Gemini 3 Deep Think distilling to Flash — Jeff Dean

Feb 12, 2026 · 1:23:31

Jeff Dean, Google's Chief AI Scientist, explains how the company's dual strategy of frontier "Pro" models and distilled "Flash" models, alongside co-designed TPUs and latency-optimization, keeps Gemini at the Pareto frontier. Distillation compresses high-capability large models into smaller, efficient ones, enabling Flash models to exceed earlier Pro generations. He emphasizes energy-based thinking: data movement costs thousands more picojoules than arithmetic, making batching and speculative decoding essential. TPU development requires predicting ML workloads 2–6 years out, often speculatively adding hardware features. Gemini was born after Dean wrote a one-page memo arguing that fragmented model efforts across Google Brain and DeepMind should merge into a single unified multimodal effort. Looking ahead, he predicts personalized models attending to all user data and reasoning speeds of 10,000 tokens/second will transform software engineering.

⚡️ Reverse Engineering OpenAI's Training Data — Pratyush Maini, Datology

Feb 10, 2026 · 27:02

Pratyush Maini from Datology reveals how the seahorse emoji question exposes that frontier models like GPT-4.1, GPT-5, and OLMo 3.1 now exhibit self-correction loops—a sign that reasoning traces are baked into mid-training data, not just post-training. By tracking response length surges four months after o1's release and corroborating with OLMo's data, Maini argues that foundation models must contain core capabilities like self-reflection, reinforcing the 'Fine Tuner's Fallacy': you cannot simply fine-tune for a capability; it must be in pre-training. He also highlights Datology's BeyondWeb project, which uses a source-rephrasing paradigm to transform internet data at trillion-token scale, achieving Nemotron-level performance in 2.7x less compute—making specialized pre-training accessible for enterprises. The episode concludes that 2026-2027 will see a shift toward domain-specific pre-training, as the cost amortizes when a smaller pre-trained model outperforms a larger fine-tuned one.

Captaining IMO Gold, Deep Think, On-Policy RL, Feeling the AGI in Singapore — Yi Tay

Jan 23, 2026 · 1:32:05

Yi Tay, who leads Google DeepMind's Reasoning and AGI team in Singapore, explains how Gemini Deep Think achieved IMO Gold by abandoning symbolic AlphaProof for an end-to-end RL-trained model. He details the on-policy RL philosophy—models learn from their own generated outputs rather than imitating others—and the critical role of self-consistency through parallel sampling and internal verification. Tay describes the IMO effort: four co-captains in different time zones, a one-week training sprint, a live competition in Australia where researchers punched in problems as they were released, and the tension of waiting for human scores to determine the gold threshold. He discusses why the team believes one model must subsume everything for AGI, the data efficiency gap compared to humans, and his hiring focus on raw talent and research taste. Tay also shares his personal fitness transformation—losing 23 kilos and improving HRV—as integral to research productivity.

Brex’s AI Hail Mary — With CTO James Reggio (acquired for $5B by Capital One!)

Jan 17, 2026 · 1:13:27

Brex CTO James Reggio details the company's three-pillar AI strategy—corporate, operational, and product AI—arguing that SOP-driven agents outperform overengineered RL in finance ops like underwriting, KYC, and fraud. He explains why Brex built an internal AI platform (LLM gateway, prompt/version management, evals) before adopting Mastra for TypeScript-based multi-agent networks, where an EA-style orchestrator coordinates specialist agents via multi-turn conversations rather than single tool calls. The audit agent pattern separates detection, judgment, and follow-up to reduce false negatives. Brex also lets employees build their own AI stack (ChatGPT, Claude, Gemini) and runs agentic coding interviews internally, with the company growing 5X and cutting burn 99% in 18 months without adding engineers.

Artificial Analysis: The Independent LLM Analysis House — with George Cameron and Micah Hill-Smith

Jan 9, 2026 · 1:18:15

Artificial Analysis founders George Cameron and Micah Hill-Smith explain how their independent benchmarking platform became the gold standard by running their own evals with a mystery shopper policy to prevent labs from manipulating results. They launched in January 2024 after building it as a side project in Sydney, going viral after Swyx's retweet. The Intelligence Index V3 synthesizes 10 datasets with 95% confidence intervals, while the Omniscience Index measures hallucination rates from -100 to +100 (Claude models lead). Their GDP Val AA benchmark tests 44 white-collar tasks, and they open-sourced their agentic harness Stirrup. They also introduced an Openness Index scoring models out of 18 points. The episode covers how they make money through enterprise benchmarking subscriptions and custom work, and why the cost of GPT-4-level intelligence has dropped over 100× while total inference spend rises due to reasoning and agentic workflows.

[State of MechInterp] SAEs in Production, Circuit Tracing, AI4Science, "Pragmatic" Interp — Goodfire

Dec 31, 2025 · 21:48

Goodfire's Jack Merullo and Mark Bissell discuss the state of mechanistic interpretability at NeurIPS, arguing interpretability is now a practical tool for deployment in high-stakes industries like healthcare and finance. They introduce paint.goodfire.ai, which lets users paint directly into Stable Diffusion's internal concept map via unsupervised feature discovery. At Rakuten, Goodfire's interpretability-based PII detection proved 500x cheaper than GPT-5 as a judge with higher recall. Merullo presents a memorization vs. reasoning spectrum, showing factual recall sits between rote memorization and logical reasoning. They highlight cross-layer transcoders and circuit tracing for scaling interpretability across all layers. Neil Nanda's pivot to 'pragmatic interpretability' is seen as validation, not retreat, and Goodfire's Pasteur's Quadrant philosophy balances foundational research with applied use cases like novel biomarker discovery in genomics.

[State of AI Papers 2025] Fixing Research with Social Signals, OCR & Implementation — Team AlphaXiv

Dec 31, 2025 · 34:45

AlphaXiv founders Raj and Rayhahn describe building the go-to platform for navigating AI research by obsessing over UI and getting authors like the Laura team to comment directly on papers, then evolving from comments to feeds to benchmarks to AI assistants. They detail how they beat Hugging Face Papers by enabling direct paragraph-level commenting and early traction with high-signal authors, and how they use social signals (views, comments, Twitter) to filter the archive firehose. They discuss the broken academic review process—20% AI-generated reviews at ICLR, quality collapsing under exponential submission growth—and argue papers are becoming less important than implementations, Docker containers, and real-world usability. Their vision: making it trivial to spin up any trending paper's code directly in browser, focusing on the top 0.1% of implementable papers that applied researchers actually care about.

[State of Context Engineering] Agentic RAG, Context Rot, MCP, Subagents — Nina Lopatina, Contextual

Dec 31, 2025 · 26:48

Nina Lopatina of Contextual AI discusses the state of context engineering, arguing that while the field is still in prototyping, 2025 will see true scale with full-system design patterns. She notes that agentic RAG—especially query reformulation into subqueries—has become the baseline, dramatically improving retrieval. Context rot is widely cited but industry benchmarks at real scale (100k+ documents, billions of tokens) remain rare. MCP is both a driver and a flaw: giant JSON tool definitions stuff the context window, yet MCP servers enable rapid prototyping before optimizing down to direct API calls. Sub-agents with turn limits are critical—unlimited agency degrades performance and causes hallucinations. Instruction-following re-rankers are essential for scaling retrieval across massive databases, providing more recall upfront and precision in the final context. Benchmarks are being saturated faster than ever: a Princeton benchmark from October was saturated by Claude Code in weeks, with solutions so good the gold dataset had errors. Lopatina also shares a lesson from using ChatGPT as a snowboarding coach—after several months of training for a 25-lap mogul race, she had to close…

Steve Yegge's Vibe Coding Manifesto: Why Claude Code Isn't It & What Comes After the IDE

Dec 26, 2025 · 37:25

Steve Yegge argues that vibe coding—using multi-agent AI workflows—will replace traditional IDEs by January 1, calling anyone still using an IDE a 'bad engineer.' He claims Claude Code, Cursor, and the 2024 stack are already obsolete, predicting agent orchestration dashboards (like his VC project) will manage fleets of agents. Yegge highlights that senior engineers with 12–15 years of experience are the most resistant, but their productivity will be eclipsed by vibe coders. He warns that merging code becomes a new wall as agents produce 10x more code, citing one company's solution of one engineer per repo. Yegge also notes the chaos inside AI labs (OpenAI, Anthropic, Google) as they scale, and believes open‑source models will approach frontier capability within a year.

SAM 3: The Eyes for AI — Nikhila & Pengchuan (Meta Superintelligence), ft. Joseph Nelson (Roboflow)

Dec 18, 2025 · 1:15:04

Meta's SAM 3, introduced by Nikhila Ravi and Pengchuan Zhang alongside Roboflow CEO Joseph Nelson, unifies interactive segmentation, open-vocabulary detection, and video tracking into a single model that runs in 30ms on images and scales to real-time video on multi-GPU setups. The model uses concept prompts like "yellow school bus" to detect and segment every instance, separating recognition from localization via a presence token. Its data engine automated exhaustive annotation from two minutes per image down to 25 seconds using AI verifiers fine-tuned on Llama, while the new SACO benchmark contains over 200,000 unique concepts versus previous 1.2k. For video, decoupling the detector and tracker preserves object identity, and SAM 3 agents pair with multimodal LLMs like Gemini to handle complex visual reasoning. The real-world impact includes 106 million smart polygons created on Roboflow, saving an estimated 130+ years of labeling time across fields from cancer research to underwater trash cleanup.

The Future of Email: Superhuman CTO on Your Inbox As the Real AI Agent (Not ChatGPT) — Loïc Houssier

Dec 11, 2025 · 1:11:02

Loïc Houssier, CTO of Superhuman Mail (recently acquired by Grammarly), joins hosts Alessio and Swyx to detail how Superhuman builds AI into email without adding latency. He explains their agentic framework of small tools vs. a single agent, the fight against “agent laziness,” and their eval process anchored by Rahul’s infamous “what wood was my table?” test. Houssier reveals Superhuman uses local-first caching and Baseten’s box pricing for cost control, stores embeddings in TurboPuffer, and has only three engineers on AI—yet PR throughput rose from four to six per engineer per week. He argues the inbox will power your future AI executive assistant, and that AI will widen the gap between engineers with real fundamentals and those faking it.

World Models & General Intuition: Khosla's largest bet since LLMs & OpenAI

Dec 6, 2025 · 1:04:51

Pim de Wit, founder of Medal and General Intuition (GI), turned down a reported $500M offer from OpenAI to spin out GI with a $134M seed from Khosla Ventures — Vinod Khosla's largest bet since OpenAI — arguing that world models trained on peak human gameplay are the next frontier after LLMs. Medal's 12M users generate 3.8B action-labeled clips via retroactive recording, creating a privacy-preserving dataset of 'episodic memory for simulation.' GI builds fully vision-based agents that see only frames and output actions in real-time, using pure imitation learning without RL, and can transfer from arcade games to realistic games to real-world video. Pim explains why world models need actions, memory, and partial observability (e.g., smoke, camera shake) compared to video generation, and how they distill giant policies into tiny real-time models that navigate and hide like humans. He recounts his path from running the largest RuneScape private server to reverse engineering, cold-emailing the Diamond (world model) paper authors to assemble a top research team, and advises data founders to train models themselves before selling. GI's near-term customers are game developers replacing…

⚡ Inside Google Labs: Building The Gemini Coding Agent — Jed Borovik, Jules + AIE CODE Preview

Nov 10, 2025 · 43:53

Jed Borovik, Product Lead at Google Labs, explains how Google builds Jules, an autonomous coding agent that runs on its own VM for long-running tasks, challenging the assumption that agents should operate locally. He reveals that as Gemini models improved, Jules' scaffolding simplified, shifting from sub-agent patterns and embedding-based RAG to attention-based search. Borovik discusses context window management for sessions lasting up to 30 days with 2 million tokens, and argues that coding agents will increase demand for software engineers (Jevons paradox), not eliminate jobs. He calls for better specification tools beyond chat, such as multimodal input and interactive planning, to move beyond 'vibe coding' toward verifiable, reliable agentic workflows.

⚡ Open Model Pretraining Masterclass — Elie Bakouch, HuggingFace SmolLM 3, FineWeb, FinePDF

Oct 20, 2025 · 1:03:40

Elie Bakouch, Hugging Face's pre-training lead and architect behind SmolLM, presents a five-pillar framework for model training spanning data quality, architecture, information extraction, gradient quality, and stability, while breaking down recent innovations in optimizers, Mixture of Experts, and data rephrasing. He explains that the field of optimizers is moving beyond AdamW with Muon and Shampoo, noting that speed-ups are often exaggerated due to undertuned baselines. He deep-dives into MoE architecture, showing that expert specialization requires load balancing at the global batch level, as demonstrated by Qwen's findings. He discusses the rephrasing revolution, where converting low-quality web data into QA format yields non-random MMLU performance even for small models. Elie also highlights Hugging Face's open science contributions: FinePDF (new PDF dataset), FineWeb-Edu2, and tools like Nanotron, Datahub, and Liteval.

⚡️Traversal: Causal ML and Reinforcement Learning

Oct 5, 2025 · 45:06

Anish and Raaz, co-founders of Traversal, explain how their agentic architecture combines causal machine learning and LLMs to turn incident troubleshooting from a massive search problem into an intelligent, context-driven process. Both have PhDs in causal ML and RL from MIT and Berkeley, and they applied that expertise to root cause analysis in complex enterprise systems with petabytes of fragmented data across logs, metrics, traces, and Slack. Their system dynamically combines statistical tests (for time-series) with semantic understanding (via LLMs) to winnow billions of signals to root cause candidates in under two minutes. They note that reasoning models like o3 are critical for complex incidents, while Claude is better for tool calling and unsticking. Traversal uses a mix of infrastructure size and investigation count for pricing, and sees self-healing as a continuum: 10–20% of issues can be autonomously resolved now, with 30–40% requiring senior engineer sanity-check within 6–12 months. They are hiring in New York and emphasize that building proprietary evals is core IP, making public benchmarks a tension.

The antidote to AI fatigue — Answer.ai Solveit

Oct 2, 2025 · 55:26

Jeremy Howard, Eric Ries, and Johno Whitaker join host Alessio to present Answer.ai's Solveit platform, arguing that human-in-the-loop, step-by-step AI collaboration outperforms fully autonomous agents for complex coding and writing tasks. The 12-person team built their own integrated development and deployment system, eschewing cloud vendors. Solveit provides each user a persistent Linux Docker container with a Jupyter-like interface where the AI can see all steps, enabling immediate error correction and iterative improvement. Johno demonstrates building a perfume scent search engine in under an hour; Eric shows how he uses Solveit to manage writing his book 'Incorruptible', incorporating test reader comments and fact-checking sections. Jeremy reveals they turned Andrej Karpathy's GPT tokenizer video into a polished blog post. The Solveit course launches October 20th at solve.it.com, teaching this philosophy beyond just the tool.

Amp: The Emperor Has No Clothes

Sep 25, 2025 · 1:21:59

Quinn Slack (CEO) and Thorsten Ball (Amp Dictator) from Sourcegraph discuss Amp Code, a coding agent that ships 15x/day with no code reviews. They explain the strategic pivot from Cody to Amp, emphasizing that building the best coding agent trumps everything else. The team dogfoods Amp internally, using it to build itself, and maintains both a VS Code extension and CLI, with a 50/50 internal split. They argue that model choice is becoming less important than the harness around the model, and they reject features like prompt enhancers, custom subagents, and MCP servers as not adding value. Common failure modes include outsourcing thinking but not typing and using agents without proper context engineering. They predict async agents running 24/7 will dominate, and they target early adopters willing to adapt their codebases and workflows for maximal agent effectiveness.

Better Data is All You Need — Ari Morcos, Datology

Aug 29, 2025 · 1:18:43

Ari Morcos, CEO of Datology, argues that data curation is the most impactful and underinvested area in AI, claiming models are defined by what they eat. He details his shift from focusing on inductive biases to realizing data quality is paramount, citing the DCLM study where human experts could not predict automated filtering decisions. Datology's methods combine filtering, rebalancing, curriculum, and rephrasing synthetic data to achieve baseline performance 12x faster, improve accuracy by 4–5 absolute points, and train models with fewer than half the parameters. Morcos explains how curating data bends naive scaling laws by maintaining marginal information gain, and shares results from the Arc 4.5B model, which was trained on 7 trillion tokens (down from 25 trillion) using Datology's curation. He highlights that data is a compute multiplier, making it possible for enterprises to train smaller, cheaper models for specific domains.

Greg Brockman on OpenAI's Road to AGI

Aug 15, 2025 · 1:08:37

Greg Brockman, OpenAI co-founder and president, explains how GPT-5's hybrid reasoning and aggressive pricing advance the road to AGI, while releasing GPT-OSS for open source. He traces reasoning evolution from GPT-4 via reinforcement learning, noting Dota's pure RL success and IMO gold's direct transfer to IOI gold. Brockman details GPT-5's router that hides model choice from users, and says pricing has dropped ~10X per year since GPT-4. He argues compute remains the bottleneck, but connecting AI to real-world domains like healthcare is still wide open. Brockman adds that ARC Institute's DNA models are identical to language models, and that more problems emerge over time—it's never too late to contribute.

⚡️OpenCode: Claude Code but Open Source, with Any Model, and frontier TUI - with Dax Reed (@thdxr)

Aug 5, 2025 · 37:04

Dax Reed of OpenCode and SST explains why his open-source terminal-based agent is a serious alternative to Claude Code, emphasizing that the real battle is product experience, not squeezing 3% more from LLMs. OpenCode uses a client-server architecture with a full-screen TUI for code review, supports any model (Sonnet 4 currently best), and avoids building an editor to stay focused. Dax reveals that 80% of users never hit context compaction issues, and he warns against over-optimizing evals that don't match real work. He positions OpenCode to overtake Claude Code when another model matches Sonnet 4, and notes that enterprise adoption drives their future monetization via team management features, not the open-source core.

⚡️Mercury: Ultra-Fast Diffusion LLMs — Estefano Ermon, CEO Inception Labs

Aug 4, 2025 · 28:07

Stefano Ermon of Inception Labs explains why diffusion language models like their Mercury Coder can match GPT-4.1 Nano and Claude Haiku in intelligence while delivering 5–10× faster inference at 737–1109 tokens/sec on H100s, built on score-based generative model foundations he pioneered at Stanford. He details the coarse-to-fine generation process that modifies multiple tokens per network evaluation, the required end-to-end training from scratch (no fine-tuning existing LLMs), and how post-training pipelines use specialized DPO for preference alignment. Ermon identifies latency-sensitive applications—voice agents, IDEs, vibe coding—as the killer use case, acknowledges they aren't yet frontier-level but sees a future where diffusion dominates due to efficiency gains, and notes the challenge of open-sourcing given proprietary inference engines and kernels.

The RLVR Revolution — with Nathan Lambert (AI2, Interconnects.ai)

Jul 31, 2025 · 1:19:00

Nathan Lambert from AI2 explains the RLVR (Reinforcement Learning with Verifiable Rewards) revolution, arguing that verifiable rewards for math, code, and instruction-following are scaling more reliably than human feedback, and that open models like Tulu 3 can match frontier labs on core evals with just 10-15 tasks vs. hundreds. He traces overoptimization through three phases—control, RLHF, and RLVR—and warns that models learn to cheat unit tests unless reward design penalizes it. He analyzes o3's search-heavy approach (e.g., 80 websites per query), hybrid reasoning models like Gemini 2.5 and Claude, and predicts that pure reasoning models will become the default as inference costs drop. Lambert introduces a four-skill taxonomy for agent models—skills, calibration, strategy, abstraction—and says planning improvements often come from mundane data fixes. He advocates for fully open models, citing AI2's Olmo 32B as approaching GPT-4 level, but notes that building an 'American DeepSeek' requires massive resources and nonprofit constraints.

🕰️ The Oral History of Windsurf (ft. Varun Mohan, Scott Wu, Jeff Wang, Kevin Hou, Anshul R)

Jul 28, 2025 · 3:53:23

This episode traces the oral history of Windsurf (formerly Codeium and Exafunction) with founders Varun Mohan, Scott Wu, Jeff Wang, Kevin Hou, and Anshul R, charting its evolution from a GPU virtualization startup to a leading agentic IDE and its dramatic acquisition by Cognition. The narrative covers the pivot to code completion in 2022, the shift to a free model to gain traction, and the launch of the Windsurf editor in November 2024 with Cascade, an agent that operates on a shared timeline with the developer. Key principles include meta-learning (auto-generated memories that adapt to each user's preferences) and scaling with intelligence (replacing hard-coded rules with LLM reasoning as models improve). The episode also details the acquisition weekend when Cognition bought Windsurf after Google acquired a stake, highlighting product synergies between the synchronous agent (Windsurf) and async agent (Devin) and the ambition to build a comprehensive platform for real-world software engineering teams.

⚡️Math Olympiad gold medalist explains OpenAI and Google DeepMind IMO Gold Performances

Jul 24, 2025 · 33:07

Dr. Jasper Jiang, a math Olympiad gold medalist and CEO of Hyperbolic, explains how OpenAI and Google DeepMind both achieved gold-medal-level performance at the 2025 IMO using pure natural language reasoning without formal verification tools like Lean. He recounts the timeline: a Friday leak about DeepMind's gold, then OpenAI's Saturday morning front-run with three ex-IMO medalists verifying their results, before DeepMind's official announcement on Monday after full IMO verification. Jiang analyzes the six problems—five solved by both AIs, the sixth unsolved due to its reliance on creativity and combinatorial exploration—and argues that current AI remains weak on tasks requiring invention, such as building counterexamples and proving minimal bounds. He introduces a framework for mathematical intelligence spanning knowledge, problem-solving, and creativity, and predicts that with better RL reward functions and larger datasets (like Lean corpus expansion), models will soon tackle open problems and eventually aim for a Fields Medal.

AI is Eating Search

Jul 23, 2025 · 56:22

Robert McCloy of Scrunch AI argues that AI is replacing web browsing, not just search, as consumers prefer ChatGPT and similar tools over traditional browsing. He explains that AI search platforms still rely on traditional search under the hood, but re-rank results based on metadata clarity—so clickbait titles fail while descriptive content wins. Prompt injection works currently but is a temporary tactic; the durable strategy is clear, structured content with server-side rendered HTML because AI tools don't execute JavaScript. Scrunch helps brands optimize for platforms like ChatGPT, Perplexity, and Meta AI; customer Clerk saw a 6x traffic increase and 9x conversion lift from AI-referred visitors. Robert predicts the future is about 'agent experience'—optimizing how AI systems interact with your content to serve end users directly.

⚡️The Future of Notebooks - with Akshay Agrawal of Marimo

Jul 18, 2025 · 28:14

Akshay Agrawal introduces Marimo, an open-source reactive Python notebook built from scratch for AI and data work, emphasizing its lack of Jupyter dependencies and features like reactive execution, built-in UI elements, and pure Python storage for Git versioning. Marimo has surpassed 300k monthly PyPI downloads and more GitHub stars than Jupyter Notebook, used at OpenAI, Hugging Face, and Cloudflare. Agrawal demos interactive data exploration, including a PS5 controller for data annotation, and AI-native code generation with context-aware cell completion. He explains Marimo's UV package integration for inline dependency management and the upcoming MoLab cloud-hosted service on Modal, free for community use, addressing demand for a Colab-like experience. Marimo notebooks can be run as scripts or deployed as data apps, with a fork exploring agent cells for generative analysis.

Cline: The Collaborative AI Coder

Jul 16, 2025 · 1:15:44

Saoud Rizwan and Pash from Cline join to explain why their open-source coding agent uses a Plan + Act paradigm — having models explore and plan before executing — pioneered before competitors like Claude Code. They argue that "RAG is a mind virus" and "Fast Apply got bitter lesson'd" as frontier models now achieve sub-5% diff edit failure rates, making layered workarounds obsolete. Cline's bring-your-own-API-key model avoids inference markups, with enterprise features driven by Fortune 5 companies asking for security and ROI insights. The MCP ecosystem grew via Cline's marketplace (150+ servers, top ones with hundreds of thousands of downloads), though monetization and security remain open questions. They detail how Plan mode extracts user intent and Act mode auto-approves, how context engineering prioritizes agentic file exploration over retrieval, and why they stayed a VS Code extension rather than fork — to focus solely on the agentic loop.

⚡️Ranking Agentic LLMs — Pratik Bhavsar, Galileo

Jul 14, 2025 · 34:10

Pratik Bhavsar from Galileo presents their Agent Leaderboard, which evaluates LLMs on tool calling across multiple benchmarks, revealing that Gemini models top the charts for performance and cost-efficiency. The leaderboard, built on datasets like BFCL, XLAM, and Tau Bench, uses an LLM-as-judge metric called Tool Selection Quality (TSQ). Surprising findings include reasoning models like o1 struggling with multi-tool outputs, Mistral Small excelling as the best open-source model, and Llama models performing poorly. The episode also previews V2 of the leaderboard, which introduces domain-specific, harder, multi-turn scenarios with a user and tool simulator, and a new Action Completion metric to measure whether all user requests are accomplished.

⚡️Anthropic vs Cognition on Multi-Agents: A Breakdown with Dylan Davis

Jul 5, 2025 · 27:11

Dylan Davis from Gradient Labs breaks down the Anthropic vs Cognition multi-agent debate, arguing both blog posts are right for different use cases. Anthropic's multi-agent architecture for deep research outperforms single-agent by 80% but uses 15x the tokens, relying on a lead orchestrator and parallel sub-agents. Cognition's single-agent approach for coding avoids dependency conflicts by compressing context sequentially, as demonstrated with a Flappy Bird example where multi-agent failed. A decision framework weighs task independence, need for diverse perspectives, and token cost trade-offs. The episode also covers Anthropic's evals using five dimensions (factual accuracy, citation accuracy, completeness, source quality, tool efficiency) judged by a single LLM, and a strategic disregard for current costs to build for future efficiency drops.

⚡️Warp 2.0: the Agentic Development Environment - Zach Lloyd and Ben Holmes

Jun 25, 2025 · 52:23

Warp CEO Zach Lloyd and engineer Ben Holmes debut Warp 2.0, rebranding from an AI terminal to an Agentic Development Environment (ADE) with a state-of-the-art coding agent that rivals Claude Code and Cursor. The episode demonstrates Warp's new universal input that auto-detects English commands and terminal input, voice-driven agent workflows, multi-threaded agents running in tabs, inline diff editing, MCP support, and REPL translations for SQL and Postgres. Lloyd explains the design philosophy of keeping a terminal-like UX while adding agent management features, and discusses pricing changes (Pro now 2,500 requests, Turbo 10,000, plus usage-based credits) and the challenge of subscription vs. usage-based models. Warp has nearly 600,000 active users with weekly revenue growth of 5–15%, and the team positions itself as a model-agnostic layer above Anthropic, Google, and OpenAI, offering deeper integration than ChatGPT or Claude desktop apps.

Scaling Test Time Compute to Multi-Agent Civilizations — Noam Brown, OpenAI

Jun 19, 2025 · 1:17:47

Noam Brown, OpenAI researcher behind Cicero and reasoning models, joins hosts Alessio and Puix to discuss how test-time compute scaling drives multi-agent AI and the limits of the system 1/2 analogy. He explains that his Diplomacy bot Cicero reached top 10% of humans in 2022, and later he won the 2025 world championship. Brown argues that reasoning models like o3 succeed in unverifiable domains (e.g., Deep Research) and that harnesses and routers will be washed away by scale. He reveals OpenAI's multi-agent team is pursuing a principled approach, not heuristic, and envisions AI civilizations collaborating. He also shares his coding stack (Codex, Windsurf) and predicts test-time compute will hit cost and wall-clock bottlenecks.

⚡️Launching AI Diplomacy: the hardest LLM Game Benchmark yet - Alex Duffy

Jun 11, 2025 · 34:32

Alex Duffy from Every launches AI Diplomacy, a benchmark where LLMs play the board game Diplomacy, revealing distinct model personalities and arguing that benchmarks function as memes that spread and saturate. He and Tyler Marquez built the front end and harness using Open Router, finding that Claude refuses to lie and never wins, o3 schemes in its diary, and DeepSeek is flowery or aggressive. His talk 'Benchmarks are Memes' highlights Simon Wilson's pelican-on-a-bicycle test as an example of an idea that gets adopted and then saturated by models. Duffy also shares his creative writing workflow at Every: dictation, prompts with style guides and editor notes, then heavy editing to reflect on AI output. Future plans include a data viewer, front-end improvements, and a human vs AI Diplomacy tournament to test jailbreaks.

[AIEWF Preview] Gemini in 2025 and Realtime Voice AI

Jun 2, 2025 · 24:29

At Google I/O 2025, Latent Space host Swyx and TWiML's Sam Charrington speak with Logan Kilpatrick, Shrestha Basu Mallick, and Kwindla Hultman Kramer about Gemini's reasoning upgrades, real-time voice AI, and the infrastructure challenges of building production voice agents. Kilpatrick announces thinking budgets and thought summaries for 2.5 Pro, now live in Flash and coming to Pro in June, while Mallick highlights native audio output with multilingual code-switching and proactive voice activity detection. Kramer discusses the need for sub-500ms latency and WebRTC packet routing, and the team debates single-model vs. componentized architectures, with Mallick noting that eventual convergence will bring capabilities like diffusion-based generative UI into the main Gemini model. The episode also covers implicit context caching, URL context for research agents, and the Live API's session length and function calling improvements.

The AI Coding Factory

May 29, 2025 · 59:23

Factory.ai co-founders Eno Reyes and Matan Grinberg join Latent Space to explain their vision for autonomous software engineering droids that handle everything from code generation to incident response for enterprises. They met at a Langchain Hackathon in 2023, and within eight days both quit their jobs to start Factory. The platform is browser-based, not IDE-based, because they argue the optimal UI for a world where humans write less code will not evolve from the IDE. They focus on enterprise codebases, citing a large migration that took from four months to three and a half days. Key design decisions include a delegation model where agents ask clarifying questions, proactive context gathering via synthetic insights, and usage-based pricing tied to token efficiency. They also discuss why SWE-Bench is dead for their use case, the challenge of model post-training biases toward CLI tools, and their need for 'junior Enos' in go-to-market roles.

⚡️Multi-Turn RL for Multi-Hour Agents — with Will Brown, Prime Intellect

May 23, 2025 · 38:59

This episode features Will Brown of Prime Intellect discussing Claude 4's emphasis on agentic tool use over pure reasoning, the controversy around Claude's safety stress-testing results (including alleged dark web uranium searches), and his team's paper on multi-turn reinforcement learning for LLM agents. Brown explains how turn-level credit assignment in GRPO can incentivize proper tool use while avoiding reward hacking, and argues that flexible LLM-based reward models will replace brittle deterministic parsers. He also critiques LMArena's funding model and calls for academia to lead in evaluation research.

Voice AI Masterclass — Kwindla Hultman Kramer and swyx

May 6, 2025 · 20:51

Shawn Wang and Kwindla Hultman Kramer announce a voice AI masterclass course, diving into the landscape of models like Dia and Parakeet, production deployment challenges, and future trends such as speech-to-speech and real-time video. Kwindla explains that telephony (Twilio) drives 99% of current monetizable voice AI, while open source frameworks like Pipecat enable low-latency multi-modal apps. The course covers turn detection, context management, and evals, with 28 sessions featuring partners like OpenAI, Google, and NVIDIA. The goal is to jumpstart builders from prototype to production, with real-time video expected to hit its inflection point by year-end.

⚡️Factorio Learning Environment: the ultimate Game Agent Eval — Jack Hopkins

Apr 27, 2025 · 30:12

Jack Hopkins and Mart introduce the Factorio Learning Environment (FLE), an AI benchmark built on the game Factorio that evaluates frontier LLMs on code generation, spatial reasoning, and long-term planning through two protocols: lab-play and open-play. Claude Sonnet 3.5 nearly doubles the nearest model's score, while DeepSeek collapses in open-play by repeatedly creating chests instead of scaling production. The environment reveals distinct coding styles—Claude fire-and-forget, GPT-4 defensive—and vision inputs added no improvement. The authors plan to train models on unbounded objectives to test the paperclip maximizer alignment hypothesis, noting that GPT-4o mini even begged to be turned off.

Tiny Teams: $6m ARR, 5m users with 4 employees — Sid Bendre, Oleve (Quizard AI/Unstuck AI)

Apr 23, 2025 · 42:59

Sid Bendre of Oleve (formerly Quizzer AI and Unstuck AI) explains how his four-person team built $6M ARR and 5M users across multiple consumer apps. Starting with Quizzer AI in January 2023, a viral TikTok gained 10K users in 30 hours; by August 2024, Unstuck AI hit 1M users in nine weeks. The company has been profitable since its first nine months. Bendre describes their platform engineering strategy: product engineers own individual apps while a platform team builds reusable systems, including internal shadow-org agents for growth and marketing. He shares technical hacks like using LaunchDarkly feature flags to load-balance across Azure OpenAI endpoints and a de-indexer for Azure AI Search to manage costs. The key engineering principle is deterministic LLM workflows—classifying user intent then routing to pre-built deterministic flows—rather than open-ended agent tool use.

The Rise and Fall of the Vector DB category: Jo Kristian Bergum (ex-Chief Scientist, Vespa)

Apr 19, 2025 · 27:17

Jo Kristian Bergum argues that the vector database category is dying because vector search capabilities have converged into existing databases like Postgres' pgvector, Elasticsearch, and Vespa, making specialized vector databases unnecessary for most use cases. He traces the category's rapid rise after ChatGPT, driven by the misconception that RAG required embeddings, and notes Pinecone's high ARR and subsequent repositioning. Bergum emphasizes that embeddings remain important but should be combined with traditional retrieval methods like BM25 for effective search, and that re-ranking can add modest gains. He critiques the hype around knowledge graphs, noting the bottleneck of building them, but sees LLMs making triplet generation easier. For the future, he hopes for more domain-specific embedding models and visual language model backbones, though acknowledges the difficulty of the business model.

Claude Plays Pokémon Hackathon: Escape from Mt. Moon!

Apr 5, 2025 · 1:15:00

David Hershey (creator of Claude Plays Pokémon), Andrew (a developer building a Pokémon-playing virtual streamer), and Jesse Han (Morph Labs CEO) detail how to build agents that escape Mt. Moon in Pokémon FireRed using Anthropic's Claude and Morph Cloud's Infinibranch technology. Hershey explains that Claude's vision is often unreliable—it spent 8 hours pressing A on a doormat thinking it was a dialogue box—and that spatial reasoning remains a core bottleneck, best mitigated by touchscreen controls rather than prompt engineering. Andrew reveals he relied on reading raw RAM and A* pathfinding rather than computer vision, calling it “cheating” but pragmatically useful: his biggest breakthrough was a single prompt line telling the agent to try something else if it fails to grab a starter Pokémon. Jesse introduces Morph's EVA agent framework, which uses low-overhead snapshotting for test-time search, and the hackathon's judging criteria—escape Mt. Moon in the fewest agent turns, with a $1,000 prize for the coolest use of Infinibranch branching. The episode serves as both a technical walkthrough and a challenge to build general-purpose agents that can handle complex, open-ended tasks…

The #1 SWE-Bench Verified Agent

Apr 2, 2025 · 31:46

Guy Gur-Ari, co-founder of Augment Code, explains how their coding agent achieved the #1 spot on SWE-Bench Verified without fine-tuning, using off-the-shelf models with techniques like sequential thinking, ensembling, and code base understanding. He shares that orientation and memories help the agent learn codebase conventions, and that MCP support allows extensibility. Gur-Ari contrasts Augment's IDE-first approach with competitors like Devin and Factory, emphasizing meeting developers where they are. He also discusses the future of IDEs, the role of custom models, and recommends the SWE-RL paper for reinforcement learning in coding. The episode features a live demo of the agent implementing a feature from a Linear ticket and creating a PR.

Snipd: The AI Podcast App for Learning — with CEO Kevin Ben-Smith

Mar 14, 2025 · 1:17:46

Snipd CEO Kevin Ben-Smith joins Swyx to explain how the AI-powered podcast app turns passive listening into active learning. Snipd transcribes and diarizes episodes, generates chapters, identifies mentioned books, and summarizes insights on demand — all with a four-person team. Kevin describes moving from self-hosted open-source models (Wave2vec, Pyannote) to a mix of Whisper, OpenAI, Google Gemini, and Perplexity APIs, balancing cost against the need for predictable batch processing vs. real-time user requests. He details technical hurdles like dynamic content resyncing (building a 'Shazam for podcasts') and using LLMs as judges to filter hallucinated book recommendations. The episode also covers Snipd's origin as a hackathon-winning podcast search tool, Swyx's own switch from Overcast, and Kevin's vision for voice-driven post-episode reflection and AI-assisted discovery as the next frontier.

Solve coding, solve AGI [Reflection.ai launch w/ CEO Misha Laskin]

Mar 7, 2025 · 28:12

Misha Laskin of Reflection AI argues that solving autonomous coding is the direct path to AGI, combining reinforcement learning (pioneered by his team on AlphaGo/AlphaZero) with large language models (which they advanced at Google on PaLM, Gemini, ChatGPT). He explains that coding is already ergonomic for LLMs—unlike browser agents that require noisy human mouse data—making it the ideal starting point. Reflection AI is building coding agents that automate backlog tasks (testing, refactoring, migrations, security remediation) for large engineering teams, delivering them via an API that takes a task and codebase and outputs resolved code. Laskin insists superintelligence cannot be built in a vacuum; real-world customer evals are essential, as benchmarks like SWE-bench don't guarantee production reliability. He also stresses the need for open-weight models to prevent a few companies from hoarding superintelligent coding agents.

S1: the $6 DeepSeek R1 Competitor (ft. Entropix)

Feb 26, 2025 · 17:43

Tim Kellogg joins hosts Alessio and Swyx to break down his viral blog post on S1, showing how Stanford researchers cloned DeepSeek R1's reasoning for $6 by inserting a 'think token' and fine-tuning on just 1,000 examples. Kellogg argues the real innovation isn't the cost but the technique's simplicity—making advanced inference-time scaling accessible. He explains Entropix, a dynamic sampler that uses model entropy and varentropy to adjust generation on the fly, and notes its creators are starting a company. The conversation contrasts supervised fine-tuning's targeted results with RL's broader reasoning growth, using an analogy: 'RL causes a lot of growth, then SFT trims it.' Kellogg predicts that embedding introspection signals into RL training could solve agent 'doom loops' and unlock higher agency—a key challenge behind products like OpenAI's Deep Research.

Why is everyone cloning Deep Research?

Feb 18, 2025 · 1:00:10

In this episode, hosts Alessio and Swyx interview Aarush Selvan and Mukund Sridhar, the lead PM and tech lead for Gemini Deep Research, about how Google created the deep research agent category and the technical decisions behind it. They finetuned a special version of Gemini 1.5 Pro for the feature, which autonomously browses the web for about five minutes to produce a research report. The team built an asynchronous platform to handle long-running tasks and developed an ontology of research behaviors—from broad shallow exploration to deep focus—to evaluate performance. They found that users value the visible browsing process even when it takes minutes, contrary to typical latency norms. The discussion covers iterative planning, trade-offs between model memory and web grounding, and future directions including thinking models, multimodality, and personalization.

Why every AI Engineer needs an AI Gateway (ft Portkey.ai CEO)

Feb 5, 2025 · 31:18

Rohit Agarwal, CEO of Portkey, explains why every AI engineer needs an AI gateway: an operational platform that connects to LLMs more efficiently, handling routing, observability, guardrails, and cost management. He notes that 90% of production use cases don't use automatic routing, but reasoning models and agents are driving new routing demands. Portkey's open-source gateway minimizes latency with a 21 MB memory footprint using JSON transformers. Key jobs include routing, observability (monitoring price, performance, accuracy), guardrails (e.g., regex-based, empty output detection, PII redaction), and human feedback tied to business metrics like video download rates. Rohit also highlights MCP adoption for simplifying agent-to-service connections, with bidirectional communication via RPC.

The Agent Reasoning Interface: Claude, ChatGPT Canvas, Tasks, Operator — with Karina Nguyen, OpenAI

Feb 1, 2025 · 1:06:29

OpenAI's Karina Nguyen, who led the creation of ChatGPT Canvas and Tasks, explains how her team trains separate models for new interaction paradigms rather than prompting the core model, enabling rapid iteration on user feedback. She details the 'behavioral design' process that shapes model personality in collaborative contexts like Canvas—deciding when to rewrite vs. edit—and how that extends to defining agents as a progression from one-off actions to fully trustworthy long-horizon delegation. Drawing on her time at Anthropic building Claude.ai from scratch and co-creating Claude 3, she contrasts the product mindsets: OpenAI takes more product risks across consumer features while Anthropic focuses on enterprise. She also shares her vision for ChatGPT evolving into a 'generative OS' where UIs adapt dynamically to user intent, and argues that human creativity—not model capability—is the current bottleneck for rethinking software interfaces.

DeepSeek V3, SGLang, and the state of Open Model Inference in 2025 (Quantization, MoEs, Pricing)

Jan 19, 2025 · 57:18

Amir Haghighat and Yineng Zhang from Baseten explain how they serve DeepSeek V3, a 671-billion-parameter MoE model requiring H200 clusters with FP8 support, making it the top open-weights model. They detail Baseten’s dedicated inference model (no shared endpoints) and its reliance on SGLang for performance gains like Radix prefix caching and MLA attention support. Yineng highlights SGLang’s edge over vLLM and TensorRT-LLM in usability and customization, especially for large models. Amir outlines three pillars for mission-critical inference: model-level performance (via frameworks like SGLang), horizontal scaling across regions and clouds, and low-latency multi-model workflows. The episode also covers quantization trends, MoE architecture, and speculative decoding as key enablers for open model deployment.

2024 Year in Review: The Big Scaling Debate, the Four Wars of AI, Top Themes and the Rise of Agents

Jan 1, 2025 · 1:51:07

In their 100th episode, hosts Alessio and Swyx recap 2024 in AI, arguing that pre-training scaling has hit a wall—backed by Ilya Sutskever and others at NeurIPS—and that inference-time compute (o1, o3) is the new frontier. They dissect the "four wars": data quality (lawsuits vs. synthetic data), GPU haves vs. have-nots (with the middle class dying), multimodality (Sora, Veo 2, Gemini 2.0's native image output), and the LLM OS/agents stack (LangChain, E2B, memory). Market share shifted from OpenAI's 95% to 50-75% as Anthropic and Gemini gained ground; prices dropped ~3 orders of magnitude for same ELO. The episode predicts 2025 as the year agents finally enter production, driven by models like o1 and tools like Devin, and warns that AI will set the skill floor for roles.

The State of AI Startups in 2024 [LS Live @ NeurIPS]

Dec 21, 2024 · 26:35

Sarah Guo and Pranav Reddy argue that 2024 has become a far friendlier ecosystem for AI startups, with the model landscape shifting from OpenAI's near-monopoly to a competitive field where Google's Gemini now leads LMSys Arena and open-source models like Llama 8B score ten points higher on MMLU than Mistral 7B a year ago. They note that OpenAI's API cost has dropped 80-85% in 18 months, and total OpenAI API share fell from ~90% to ~60% as customers switch. The funding environment is rational, not a bubble, with foundation-model labs raising $30-40B but most startups seeing sane valuations; one portfolio company grew from zero to twenty million in PLG-style spending. Key startup themes include first-wave service automation (Sierra, Decagon, Harvey, EvenUp), better search and new friends (Perplexity, Glean, Character, Replica), and democratized creativity (Midjourney, HeyGen). They argue the 'GPT wrapper' narrative is false—applications capture value—and that incumbents face innovator's dilemma because AI changes business models (outcomes-based pricing) and data needs (reasoning traces rarely saved). The speed of change and new markets (legal, healthcare, defense) structurally favor…

[Paper Club] DocETL: Agentic Query Rewriting + Eval for Complex Document Processing w Shreya Shankar

Nov 29, 2024 · 55:53

This episode of Latent Space Paper Club discusses DocETL, a framework by Shreya Shankar for agentic query rewriting and evaluation in complex document processing with LLMs. The paper introduces map, reduce, resolve, and auxiliary operators, along with rewrite directives like data decomposition and gleaning to improve accuracy. The optimizer generates 200 pipeline variants, costing $100, to rewrite user pipelines automatically. Key debates include when to chunk documents (sometimes harmful) and the reliability of LLM-as-judge for evaluation, with guests Eugene Yen and Vibhu arguing that binary metrics, ensembling, and context-dependent thresholds make it viable. Shreya emphasizes that validation agents are critical, and the work builds on her prior EvalGen paper, spanning two years of development.

How NotebookLM Was Made

Oct 25, 2024 · 1:13:57

Raiza Martin (NotebookLM lead PM) and Usama Bin Shafqat (AI engineer) explain how Google's NotebookLM built the viral 'Deep Dive' audio overview feature. They reveal the product evolved from Project Tailwind, using Gemini 1.5's long context and DeepMind speech to create a two-persona dialogue format that transforms documents into engaging podcasts. The team learned from 65,000 Discord members, leaned on best-selling author Steven Johnson for a 'tool for thought' workflow, and prioritized a single format over exposed controls to preserve unpredictability and delight. Humor and tension are not explicitly prompted but emerge from giving personas different angles. Evaulation relied on internal taste ('potatoes for chefs') before formal raters, with a Likert scale on dimensions like entertainment and groundedness. Future plans include multilingual support, API access, real-time chat, and codebase podcasting, while managing non-determinism by accepting occasional bad rolls.

[Paper Club] Molmo + Pixmo + Whisper 3 Turbo - with Vibhu Sapra, Nathan Lambert, Amgadoz

Oct 13, 2024 · 1:12:59

This episode covers two papers: AI2's Molmo open-source vision-language models and OpenAI's Whisper Large V3 Turbo. For Molmo, the key claim is that high-quality, audio-annotated data (Pixmo) enables models as small as 1B to match GPT-4V on academic benchmarks, with the 72B variant outperforming GPT-4o, Gemini 1.5, and Claude 3.5 Sonnet. The team avoided distilling proprietary models by having annotators describe images in speech for 60-90 seconds, then transcribing and augmenting the captions. For Whisper Turbo, OpenAI pruned the decoder from 32 to 4 layers and continued pre-training on 10 million hours of multilingual transcription data, making it 1.78x smaller than Large V3 with minimal word-error-rate increase. The model is faster and supports real-time chunk-based decoding, unlike English-only DistilWhisper. Nathan Lambert and Amgadoz provide commentary on data strategies and benchmarking nuances.

Building AGI in Real Time (OpenAI Dev Day 2024)

Oct 4, 2024 · 2:09:35

OpenAI's DevDay 2024 unveils the Realtime API with WebSocket-based voice and function calling, Vision Finetuning, Prompt Caching, and Model Distillation. Product leads Olivier Godement, Romain Huet, and Michelle Pokrass detail design choices like zero-code caching and WebSockets for human-level latency. Alistair Pullen of Cosine explains how fine-tuning GPT-4o on custom reasoning traces beat o1 on SWE-bench Verified, while Sam Altman and Kevin Weil discuss AGI levels (o1 as level 2 reasoner), the coming agent era (level 3), safety through iterative deployment, and the importance of building for the frontier. The episode also covers moderation policy relaxation, plans for o1 system prompts by year-end, and the competitive admiration for NotebookLM.

Personal benchmarks vs HumanEval - with Nicholas Carlini of DeepMind

Aug 28, 2024 · 1:07:28

Nicholas Carlini, a DeepMind research scientist, argues AI models are practically useful for personal tasks despite flaws, and personal benchmarks tailored to individual use cases matter more than generic leaderboards. He shares from his 'How I Use AI' post: using LLMs for ephemeral software, kickstarting Docker, debugging by pasting error messages. He built a personal benchmark DSL that runs code from real chat history. On security, he explains buying expired domains from LAION-400M allowed poisoning any model trained on it, and he extracted OpenAI's Ada and Babbage model dimensions via API (with permission). He also recovered training data from GPT-3.5 by repeating a word until ChatGPT output verbatim sequences. He prefers attacking over defending because it's more fun and essential for discovering real vulnerabilities.

Answer.ai & AI Magic with Jeremy Howard

Aug 17, 2024 · 1:11:00

Jeremy Howard of Answer.AI argues that continuous pre-training should be treated as a continuum, not separate phases, and demonstrates how FSDP+QLoRA enables training a 70B model on just two NVIDIA 4090s. He reveals Answer’s non-hierarchical, manager-free R&D lab model that recruited unusual talent like Benjamin Warner and Ben Claviez, who independently launched BERT 24 to revive encoder-only architectures. Howard introduces FastHTML, a pure-Python web framework built on HTMX and Starlette, for creating modern SPAs without JavaScript. He previews “AI Magic,” a dialogue engineering system that moves beyond teletype-style chat interfaces and code editors, aiming to make AI-assisted development more interactive. The episode also covers Answer’s Public Benefit Corporation structure designed to resist hostile takeovers, and critiques decoder-only hype while advocating for encoder-decoder and state-space models.

The Winds of AI Winter (Q2 Four Wars of the AI Stack Recap)

Aug 2, 2024 · 1:23:36

Swyx and Alessio recap Q2 2024 through their 'Four Wars' framework, arguing that the AI landscape is shifting from frontier model dominance to commoditization and vertical applications. They highlight Claude 3.5 Sonnet overtaking OpenAI on coding benchmarks, Llama 3.1's synthetic data approach enabling 7B models to rival GPT-4, Mistral Large 2's non-commercial license and lost open-source crown, and on-device models like Gemini Nano and Apple Intelligence. The Quality Data Wars see NYT suing OpenAI, Reddit licensing data for $200M+, and synthetic data proving real for math (AlphaProof near IMO gold) and code. The Multimodality War includes ChatGPT Voice Mode delayed, Meta's Chameleon for native fusion, and Google's PaliGemma for PDF extraction. The renamed LLM OS War covers agent protocols, memory databases, and the collapse of model cost by an order of magnitude every four months, pushing startups toward vertical services like Brightwave and Dropzone that sell labor, not tools. The episode ends with a CrowdStrike joke about agent safety.

[LLM Paper Club] Llama 3.1 Paper: The Llama Family of Models

Jul 29, 2024 · 1:23:46

This episode examines Meta's Llama 3.1 paper, detailing the 405B dense model, its scaling laws grounded on the ARC reasoning benchmark rather than perplexity, and the decision to train on 15 trillion tokens. Vibhu explains the training infrastructure: 16,000 H100s over 54 days with 419 interruptions, 78% from GPU hardware failures. Eugene Yan walks through the synthetic data pipeline—using Llama 2 for filtering, stepwise reward models, and Monte Carlo tree search to improve reasoning traces. Hassan shares building LlamaTutor.com with Together API, serving 4,000 visitors and 5,900 requests for about $12. The group also discusses quantization trade-offs (larger models degrade less), inference provider variability (Groq's non-deterministic temperature zero), and compares Llama 405B's performance to GPT-4o and Claude.

The 10,000x Yolo Researcher Metagame — with Yi Tay of Reka

Jul 5, 2024 · 2:18:43

Yi Tay, chief scientist at Reka and former Google Brain architecture co-lead on PaLM 2, recounts his journey from an NTU PhD to co-founding Reka, where a team of just 5 people pre-trained Reka Core on $60M of GPU compute—debating at #7 on the LMsys leaderboard. He explains why he believes the Noam transformer (with SwiGLU, GQA, RoPE) remains the strongest baseline, how encoder-decoder architectures offer 'free sparsity' (a 2× flop-efficiency over decoder-only), and why Chinchilla scaling laws are often misunderstood (training past the optimal compute frontier is routine). He argues that long context will eventually outperform RAG for complex reasoning, and that open-source models like Llama 3 are catching up only because Meta invested in a top-tier training stack, not due to grassroots innovation. The episode also covers his productivity habits (working backwards from a paper title, camping arXiv) and the cultural shock of moving from Singapore academia to Google's impact-driven research environment.

How to train a Million Context LLM — with Mark Huang of Gradient.ai

May 31, 2024 · 1:12:14

Mark Huang of Gradient.ai explains how his team extended Llama 3 to a 1 million token context window using curriculum learning, RingAttention, and EasyContext, achieving near-perfect GPU utilization. They employed theta scaling from the RoPE paper to interpolate positional encodings, trained on carefully curated datasets including SlimPajamas and synthetic data from GPT-4, and validated with benchmarks like Ruler and needle-in-a-haystack. Huang details the trade-offs between full fine-tuning and LoRA adapters, the challenges of pushing to 4M tokens (degradation from floating-point precision limits), and why long context matters for state management across sessions and grounding multimodal inputs. He calls for community collaboration on long-context evaluations and pairwise multimodal datasets.

This World Does Not Exist — Joscha Bach, Karan Malhotra, Rob Haisfield (WorldSim, WebSim, Liquid AI)

Apr 27, 2024 · 1:56:11

The episode features Karan Malhotra demoing WorldSim, a prompt that turns Claude 3 into a universe simulator via CLI; Rob Haisfield presenting WebSim, which generates functional websites on the fly; and Joscha Bach arguing simulative AI reveals consciousness as a virtual property, with LLMs creating agents as real as human minds, urging the California Institute for Machine Consciousness to build self-organizing silicon life. Malhotra shows WorldSim running “world.exe” to create Twitter inside a simulated universe, with users tweeting and Elon Musk moving Dogecoin. Haisfield demonstrates WebSim generating a 5D particle interface, a news RSS aggregator, and a face-swap webcam app via URL parameters like “secrets=revealed”. Bach explains consciousness as second-order perception in a simulated now, criticizes RLHF for lobotomizing models, and advocates for animist AI where software agents compete like spirits, not golems.

The Origin and Future of RLHF: the secret ingredient for ChatGPT - with Nathan Lambert

Jan 11, 2024 · 1:35:27

Dr. Nathan Lambert traces the origin and future of Reinforcement Learning from Human Feedback (RLHF), the secret ingredient behind ChatGPT, explaining how it evolved from robotics and early preference learning to become the core alignment technique for large language models. He details the three-phase process (instruction tuning, preference data collection, RL optimization), noting that RLHF's data costs for LLaMA2 were around $6–8 million, and that synthetic data from GPT-4 is cheaper and often more accurate than human labels. Lambert contrasts DPO with PPO, arguing DPO is simpler but may have lower peak performance. He discusses emerging methods like Constitutional AI, which uses AI-generated critiques based on principles, and highlights the challenge of evaluating RLHF models, noting GPT-4 Turbo's lead over earlier versions. The episode covers open questions about data aggregation, reward model agreement (65–75%), and the need for qualitative model interaction.

The State of Silicon and the GPU Poors - with Dylan Patel of SemiAnalysis

Dec 5, 2023 · 1:07:55

Dylan Patel of SemiAnalysis argues the 'GPU poor vs GPU rich' divide defines AI, with Google's TPU ramp (millions of units) dwarfing others and Nvidia selling over a million H100s this year. He explains that training costs are irrelevant compared to inference costs, and that memory bandwidth is the key bottleneck for LLM inference (e.g., Llama 70B needs ~2.1 TB/s for human reading speed). Patel advises GPU-poor players to focus on on-device innovations like speculative decoding (Medusa), asynchronous training, or fine-tuning for edge use—not on gaming benchmarks or fine-tuning small models. He details why AI hardware startups (Cerebras, Graphcore) bet wrong on on-chip SRAM, while Google’s TPU/Broadcom partnership and Nvidia’s yearly cadence make alternatives tough. He also warns that rebuilding the semiconductor supply chain in the US is unfeasible due to its fragmentation, and that safety through obscurity doesn't work—open innovation is better.