Hugging Face

Next 100x in AI: Inference, Networking, & Self-Optimizing Models — Philip Kiely & Ali Taha, Baseten

Aug 3, 2026 · 1:42:54

Philip Kiely and Ali Taha of Baseten join Swyx to explain what actually happens when a 200,000-token request hits a production inference system, arguing that stacking quantization, speculative decoding, and disaggregated prefill/decode can make open models like GLM-5.2 up to 10x faster. They detail cache-aware routing, training traffic-specific speculators, and why quantization errors can cancel out so a more-quantized model beats a less-quantized one. They also reveal how Baseten grafted Kimi's vision encoder onto GLM-5.2, discuss NVIDIA Dynamo as a toolkit rather than a turnkey speedup, and explain why they are bearish on mega kernels. The conversation covers video generation's compute barriers, the trend toward ASIC-like GPUs, and the emerging loop where GLM-5.2 writes the GPU kernels that serve itself.

The Future of AI Infra: from Kubernetes to Agent Sandboxes — Akshat Bubna, Modal CTO

Jul 8, 2026 · 59:10

Modal CTO Akshat Bubna explains how the cloud platform evolved from a serverless runtime to an AI cloud built for elastic inference, agent sandboxes, and post-training workloads. He argues that Kubernetes was never designed for bursty GPU-heavy AI workloads, so Modal built a decorator-based infrastructure that co-locates compute requirements with code. The company added GPUs a year before ChatGPT and now powers inference for custom models at Suno, Runway, and robotics firms, using GPU snapshotting to slash cold starts. Modal's new Auto Endpoints incorporate open-source DeFlash speculative decoding for frontier-level performance. For reinforcement learning rollouts, Modal provides up to 100,000 sandboxes simultaneously. The platform spans 17 cloud providers, features private IPv6 networking via eBPF, and supports serverless multi-node training with RDMA. Bubna also reveals Modal's shift from developer experience to agent experience, noting that agents benefit from the same minimalist SDK and observability tools that humans use.

🔬 The Limits of AI in Science - Why We Need Self-Driving Labs — Joseph Krause, Radical AI

Jun 17, 2026 · 1:16:50

Joseph Krause of Radical AI argues that the bottleneck in materials science is experiments, not ideas, and his company's self-driving lab combines AI hypothesis generation with automated synthesis and characterization to produce alloys at unprecedented speed—1,200 in six months, with 300 novel compositions and 10 already in commercial development. Radical's closed-loop system runs research campaigns, not just automated tasks, overcoming challenges like sample manipulation at 3,000°C and tool vendors' software access. Krause details how their AI explores elemental families humans overlooked, why they open-source models like Matrix (the moat is experimental data, not models), and how they plan to compress discovery timelines from decades to 3–5 years for defense and space applications. He addresses the 10-year qualification process for aerospace, supply chain geopolitics (e.g., hafnium price up 10–15x due to Chinese dominance), and the need for public-private partnerships to accelerate U.S. R&D. Finally, he urges ML engineers to lean into their expertise rather than try to become material scientists.

⚡️ Google's Open AI Strategy — Omar Sanseviero, Google DeepMind

May 24, 2026 · 29:59

Omar Sanseviero, Google DeepMind's Head of Developer Experience, explains Gemma 4's novel architecture with per-layer embeddings that enable effective parameter offloading: only 2B of 5B parameters need GPU memory, ideal for on-device inference on phones and Raspberry Pis. The model matches 1.5-year-old state-of-the-art in most areas, with Gemini Nano integrated into Pixel and Samsung phones. Gemma 4 supports multimodal input (audio, images, short video) but not audio output or combined audio-video prompts. Sanseviero notes fine-tuning is declining as out-of-box capabilities improve, but remains relevant for specialized domains like healthcare. He contrasts dense (31B) and MoE (27B) variants, highlighting MoE's inference speed but fine-tuning challenges. The team is expanding globally, with Kaggle joining DeepMind to create community-driven benchmarks for model evaluation.

⚡️ Reverse Engineering OpenAI's Training Data — Pratyush Maini, Datology

Feb 10, 2026 · 27:02

Pratyush Maini from Datology reveals how the seahorse emoji question exposes that frontier models like GPT-4.1, GPT-5, and OLMo 3.1 now exhibit self-correction loops—a sign that reasoning traces are baked into mid-training data, not just post-training. By tracking response length surges four months after o1's release and corroborating with OLMo's data, Maini argues that foundation models must contain core capabilities like self-reflection, reinforcing the 'Fine Tuner's Fallacy': you cannot simply fine-tune for a capability; it must be in pre-training. He also highlights Datology's BeyondWeb project, which uses a source-rephrasing paradigm to transform internet data at trillion-token scale, achieving Nemotron-level performance in 2.7x less compute—making specialized pre-training accessible for enterprises. The episode concludes that 2026-2027 will see a shift toward domain-specific pre-training, as the cost amortizes when a smaller pre-trained model outperforms a larger fine-tuned one.

[State of Evals] LMArena's $1.7B Vision — Anastasios Angelopoulos, LMArena

Dec 31, 2025 · 24:02

Anastasios Angelopoulos, founder of LMArena (now Arena), discusses the platform's $100M raise at a $1.7B valuation, its spin-out from Berkeley incubation by a16z's Anjney Midha, and its mission to be the industry's north star for real-world AI evaluation. He defends against the 'leaderboard illusion' paper, citing factual errors and reaffirming that models cannot pay to be on or off the public leaderboard. Arena funds inference costs for millions of monthly users, with 25% of its 5M+ users in software, and is expanding into occupational verticals (medicine, legal, creative) and multimodal video arenas. Key challenges include consumer retention, which improved with sign-in and persistent history, and moving off Gradio to React for better development. Angelopoulos calls for top talent in ML, product, and go-to-market to join the high-performance team.

⚡ Open Model Pretraining Masterclass — Elie Bakouch, HuggingFace SmolLM 3, FineWeb, FinePDF

Oct 20, 2025 · 1:03:40

Elie Bakouch, Hugging Face's pre-training lead and architect behind SmolLM, presents a five-pillar framework for model training spanning data quality, architecture, information extraction, gradient quality, and stability, while breaking down recent innovations in optimizers, Mixture of Experts, and data rephrasing. He explains that the field of optimizers is moving beyond AdamW with Muon and Shampoo, noting that speed-ups are often exaggerated due to undertuned baselines. He deep-dives into MoE architecture, showing that expert specialization requires load balancing at the global batch level, as demonstrated by Qwen's findings. He discusses the rephrasing revolution, where converting low-quality web data into QA format yields non-random MMLU performance even for small models. Elie also highlights Hugging Face's open science contributions: FinePDF (new PDF dataset), FineWeb-Edu2, and tools like Nanotron, Datahub, and Liteval.

Context Engineering for Agents - Lance Martin, LangChain

Sep 11, 2025 · 1:03:18

Lance Martin from LangChain discusses context engineering for AI agents, arguing that managing context from tool calls is the key challenge and introducing five techniques: offloading, reducing, retrieval, caching, and context isolation. Offloading saves raw tool call outputs to disk and passes only summaries, drastically cutting token usage—Martin's deep research agent went from 500,000 tokens per run to much less. Manus uses file system offloading and warns that irreversible pruning risks information loss, while Cognition advocates fine-tuned summarization for agent-agent boundaries. For retrieval, Martin contrasts Windsurf's multi-step indexing with Claude Code's agentic search using grep and llms.txt, finding the latter more effective. He applies the Bitter Lesson to agent engineering: his own Open Deep Research evolved from a highly structured workflow to a simple agent loop as models improved, and he warns against rigid abstractions that hinder adaptation.

⚡️Accelerators @ 3x NVIDIA H200 perf, Made in the USA - Thomas Sohmers + Mitesh Agrawal, Positron AI

Aug 18, 2025 · 47:56

Thomas Sohmers and Mitesh Agrawal of Positron AI argue memory bandwidth, not compute, is the true bottleneck in AI inference, and their accelerator achieves 93% memory bandwidth utilization—triple NVIDIA's efficiency—enabling 70% faster token generation at 150W. The founders, both Lambda Labs veterans, shipped an FPGA product 15 months after founding, then raised a $51M Series A for an ASIC in late 2026. Their hardware requires no recompilation: it ingests raw binary weights from NVIDIA training and outputs an OpenAI-compatible API. Positron focuses on the decode phase of transformers, where memory-bound matrix-vector multiplication dominates, and already counts Cloudflare and Parasail as customers. The company sells systems directly, prioritizing capital efficiency and ROIC over operating its own cloud.

The RLVR Revolution — with Nathan Lambert (AI2, Interconnects.ai)

Jul 31, 2025 · 1:19:00

Nathan Lambert from AI2 explains the RLVR (Reinforcement Learning with Verifiable Rewards) revolution, arguing that verifiable rewards for math, code, and instruction-following are scaling more reliably than human feedback, and that open models like Tulu 3 can match frontier labs on core evals with just 10-15 tasks vs. hundreds. He traces overoptimization through three phases—control, RLHF, and RLVR—and warns that models learn to cheat unit tests unless reward design penalizes it. He analyzes o3's search-heavy approach (e.g., 80 websites per query), hybrid reasoning models like Gemini 2.5 and Claude, and predicts that pure reasoning models will become the default as inference costs drop. Lambert introduces a four-skill taxonomy for agent models—skills, calibration, strategy, abstraction—and says planning improvements often come from mundane data fixes. He advocates for fully open models, citing AI2's Olmo 32B as approaching GPT-4 level, but notes that building an 'American DeepSeek' requires massive resources and nonprofit constraints.

⚡️The Future of Notebooks - with Akshay Agrawal of Marimo

Jul 18, 2025 · 28:14

Akshay Agrawal introduces Marimo, an open-source reactive Python notebook built from scratch for AI and data work, emphasizing its lack of Jupyter dependencies and features like reactive execution, built-in UI elements, and pure Python storage for Git versioning. Marimo has surpassed 300k monthly PyPI downloads and more GitHub stars than Jupyter Notebook, used at OpenAI, Hugging Face, and Cloudflare. Agrawal demos interactive data exploration, including a PS5 controller for data annotation, and AI-native code generation with context-aware cell completion. He explains Marimo's UV package integration for inline dependency management and the upcoming MoLab cloud-hosted service on Modal, free for community use, addressing demand for a Colab-like experience. Marimo notebooks can be run as scripts or deployed as data apps, with a fork exploring agent cells for generative analysis.

Voice AI Masterclass — Kwindla Hultman Kramer and swyx

May 6, 2025 · 20:51

Shawn Wang and Kwindla Hultman Kramer announce a voice AI masterclass course, diving into the landscape of models like Dia and Parakeet, production deployment challenges, and future trends such as speech-to-speech and real-time video. Kwindla explains that telephony (Twilio) drives 99% of current monetizable voice AI, while open source frameworks like Pipecat enable low-latency multi-modal apps. The course covers turn detection, context management, and evals, with 28 sessions featuring partners like OpenAI, Google, and NVIDIA. The goal is to jumpstart builders from prototype to production, with real-time video expected to hit its inflection point by year-end.

Why Every Agent needs Open Source Cloud Sandboxes

Apr 24, 2025 · 1:06:39

Vasek Mlejnsky, CEO of E2B, explains how his company grew from 40,000 sandboxes in March 2024 to 15 million in March 2025 by providing cloud sandboxes for AI agents. Originally built from DevBook's interactive docs playground, E2B now serves use cases from code interpreting and data analysis to Computer Use and reinforcement learning with Hugging Face's Open R1. The sandboxes are general Linux VMs with fast startup, security isolation, and persistence, handling untrusted code and allowing forking/checkpoints for parallel agent execution. Mlejnsky discusses the LLMOS landscape, pricing challenges (moving from token-based to value-based), MCPs as higher-order tools, and why he relocated from Prague to San Francisco to be closer to users. E2B is hiring across engineering and customer success as it aims to become full-lifecycle infrastructure for LLMs.

GPT 4.1: The New OpenAI Workhorse

Apr 15, 2025 · 45:04

OpenAI's Michelle Pokrass and Josh McGrath join hosts Alessio, Swyx, and Sean to launch GPT 4.1, a new model family positioned as the go-to workhorse for developers with major improvements in coding, instruction following, and long context. The three models—4.1, 4.1 Mini, and 4.1 Nano—introduce a 1M-token context window, 75% prompt caching discount, and cheaper pricing than 4o. Coding gains are highlighted by 55% on SWE-bench (vs o1's 41%), while instruction following benefits from real-world API data and new evals like GraphWalk for multi-hop reasoning. The team explains that most improvements come from post-training techniques, with vision capabilities lifted by a new pre-trained base. Fine-tuning is available day one, and developers are encouraged to opt in to data sharing for future model iterations.

smol agents are all you need

Feb 13, 2025 · 22:31

This episode features Aymeric Roucher of Hugging Face discussing smol agents, a lightweight agent framework, and argues that code-based agents outperform JSON-based agents. Roucher explains that agents are defined by LLM control over execution flow, and smol agents simplifies this with a core file under 1000 lines. He describes the origin from transformers.agents and the security-focused custom Python interpreter. The library achieved third place on the GAIA benchmark, using o1 and Sonnet models, and Roucher projects 90% accuracy by 2026 based on a sigmoid trend line. Hugging Face is launching an agent course and plans to build GUI agents for computer use, while also fine-tuning R1 for agentic tasks.

Outlasting Noam Shazeer, Crowdsourcing Chai AI w/ 1.4m DAU — with William Beauchamp, Chai Research

Jan 26, 2025 · 1:13:33

William Beauchamp, founder-CEO of Chai AI, explains how he pivoted from algorithmic trading to build a character chatbot platform before Character.ai, growing to 1.4M DAU and $22M+ revenue by crowdsourcing model improvements through Chaiverse. Starting with GPT-J in 2021, he found product-market fit with a therapist bot and shifted to user-generated content, letting users define prompts, images, and names. Competing against well-funded rivals like Character AI and Talkie, Chai ships over 100 LLMs weekly via its developer platform, spending $10M on compute in 2024 and tripling it. Beauchamp argues AI follows an S-curve, not scaling laws, and focuses on inference optimization using rejection sampling and reward models to serve better responses. He prioritizes 'insanely great' products over technology-driven features, noting that audio and image features failed to move metrics, while a data flywheel and aggressive user acquisition drove rapid growth.

The State of Reasoning — from Nathan Lambert, Interconnects/AI2 [LS Live @ NeurIPS 2024]

Jan 2, 2025 · 16:22

Nathan Lambert argues that language models do perform reasoning, contrary to skeptics, and that embracing chain-of-thought and reinforcement learning (RL) is key to advancing their capabilities. He explains OpenAI's o1 as large-scale RL on verifiable outcomes, noting that post-training flops exceed pre-training, and highlights relatives like DeepSeek and Qwen which are narrower. Lambert details OpenAI's new reinforcement fine-tuning API, which uses the same infrastructure as o1 and requires only dozens of labeled samples, and contrasts it with process reward models or Monte Carlo tree search. He presents his own project using RL on math evaluations (GSM8K, MATH, AFeval) to show gentle RL fine-tuning can boost specific capabilities without degrading general performance. The talk concludes that reasoning is worth pursuing and that new, less-human-like forms of model reasoning are emerging.

Best of 2024: Synthetic Data / Smol Models, Loubna Ben Allal, HuggingFace [LS Live! @ NeurIPS 2024]

Dec 24, 2024 · 28:08

Loubna Ben Allal of Hugging Face explains how synthetic data has become ubiquitous in LLM pipelines, moving from post-training to pre-training, with examples like Cosmopedia’s 30B-token synthetic dataset and NVIDIA’s 1.9T-token Nemotron CC. She addresses model collapse fears, showing that web dumps after ChatGPT’s release actually yield better models, and emphasizes diversity through prompt seeds and webpage extracts. For filtering, FineWeb-Edu and DCLM use LLMs to rate educational content, achieving top benchmark scores. In post-training, she highlights AgentInstruct, Tülu3 with PersonaHub, and Cohere’s multilingual arbitrage using multiple teachers. On small models, SmolM2 (1.7B) outperforms Llama 1B and Qwen 2.5 after 11T tokens of pre-training, and on-device inference via frameworks like llama.cpp enables privacy-preserving use cases such as text extraction and structured generation. She predicts a return to fine-tuning specialized small models over costly prompt engineering.

[Paper Club] BERT: Bidirectional Encoder Representations from Transformers

Nov 27, 2024 · 53:12

Eric Ness walks through the BERT paper, explaining how its bidirectional encoder architecture with masked language modeling and next sentence prediction pre-training enabled state-of-the-art results on 11 NLP tasks in 2019. He details the 110M parameter BASE and 340M parameter LARGE models, trained on 2.5B Wikipedia words and 800M book words, and how a simple logistic regression on BERT embeddings achieves 82% accuracy on IMDb sentiment classification, far above the 50% baseline. Swyx and Eric discuss how BERT’s pre-training objectives—masking 15% of tokens and predicting sentence order—teach small models word relationships for efficient classification, though they lose data compared to decoder-only next-token prediction. They note that while scaling laws favor decoder models, BERT-style encoders remain cost-effective for edge deployment, with recent work showing full retraining in 24 hours for under $500 or even 1 hour for $20 on 8 A100s, matching original GLUE scores.

Why Compound AI + Open Source will beat Closed AI — with Lin Qiao, CEO of Fireworks AI

Nov 25, 2024 · 55:49

Lin Qiao, CEO of Fireworks AI, argues that compound AI systems combining multiple open-source models will outperform closed monolithic models like OpenAI's. She explains Fireworks' evolution from a PyTorch platform to a full-stack inference and customization engine serving 40+ customers including Cursor. Qiao details their distributed inference engine, Fire Optimizer for quality-latency-cost tradeoffs, and upcoming o1-like model built on open-source foundations. She defends their quantization approach after public criticism from rivals, emphasizes specialization over general intelligence, and invites developers to test their free LoRA adapter hosting.

[Paper Club] Molmo + Pixmo + Whisper 3 Turbo - with Vibhu Sapra, Nathan Lambert, Amgadoz

Oct 13, 2024 · 1:12:59

This episode covers two papers: AI2's Molmo open-source vision-language models and OpenAI's Whisper Large V3 Turbo. For Molmo, the key claim is that high-quality, audio-annotated data (Pixmo) enables models as small as 1B to match GPT-4V on academic benchmarks, with the 72B variant outperforming GPT-4o, Gemini 1.5, and Claude 3.5 Sonnet. The team avoided distilling proprietary models by having annotators describe images in speech for 60-90 seconds, then transcribing and augmenting the captions. For Whisper Turbo, OpenAI pruned the decoder from 32 to 4 layers and continued pre-training on 10 million hours of multilingual transcription data, making it 1.78x smaller than Large V3 with minimal word-error-rate increase. The model is faster and supports real-time chunk-based decoding, unlike English-only DistilWhisper. Nathan Lambert and Amgadoz provide commentary on data strategies and benchmarking nuances.

Production AI Engineering starts with Evals

Oct 11, 2024 · 1:56:16

Ankur Goyal, founder and CEO of Braintrust, argues that evaluation is the core workflow of production AI engineering and shows how his platform makes evals accessible to software engineers. Drawing from his experience at SingleStore and Impira, he details Braintrust's evolution from an eval tool into an end-to-end AI development platform used by Stripe, Zapier, Vercel, and other top AI teams. He shares market data: OpenAI handles over 95% of production workloads, fine-tuning is declining, and open-source models account for under 5% due to reliability issues. Ankur explains why he avoided building a vector database—the real challenge is permissions and joins, not vector search—and predicts o1-style reasoning will replace complex agent frameworks. He reveals Braintrust's differentiators: hybrid on-prem, TypeScript-first SDK, and declarative eval structures.

[Paper Club] Berkeley Function Calling Paper Club! — Sam Julien, Writer

Oct 5, 2024 · 41:29

Sam Julien (Writer) walks through the Berkeley Function Calling Leaderboard (BFCL) and its three versions, detailing how each iteration improves evaluation of LLM function calling capabilities. BFCL v1 (March) introduced abstract syntax tree evaluation across 2,000 function-answer pairs, covering simple, multiple, and parallel calling. v2 (August) added a live dataset of 2,251 pairs, reducing bias and contamination through Rouge-score deduplication and filtering. v3 (September) introduced multi-turn and multi-step evaluation with state-based metrics, using a custom API (vehicle control, stock trading, etc.) and human-validated ground truth. Sam also highlights common model failures—misunderstanding current state, unnecessary planning—and mentions the Gorilla CLI and Open Functions as related tools in the Berkeley ecosystem.

Answer.ai & AI Magic with Jeremy Howard

Aug 17, 2024 · 1:11:00

Jeremy Howard of Answer.AI argues that continuous pre-training should be treated as a continuum, not separate phases, and demonstrates how FSDP+QLoRA enables training a 70B model on just two NVIDIA 4090s. He reveals Answer’s non-hierarchical, manager-free R&D lab model that recruited unusual talent like Benjamin Warner and Ben Claviez, who independently launched BERT 24 to revive encoder-only architectures. Howard introduces FastHTML, a pure-Python web framework built on HTMX and Starlette, for creating modern SPAs without JavaScript. He previews “AI Magic,” a dialogue engineering system that moves beyond teletype-style chat interfaces and code editors, aiming to make AI-assisted development more interactive. The episode also covers Answer’s Public Benefit Corporation structure designed to resist hostile takeovers, and critiques decoder-only hype while advocating for encoder-decoder and state-space models.

Training Llama 2, 3 & 4: The Path to Open Source AGI — with Thomas Scialom of Meta AI

Jul 23, 2024 · 1:04:34

Thomas Scialom, Llama 2 lead and Llama 3 post-training lead at Meta AI, explains why scaling laws must go beyond Chinchilla—training models longer on more tokens yields better inference efficiency than bigger models, a lesson that drove Llama 3's 15 trillion token pre-training. He details how synthetic data from Llama 2 bootstrapped Llama 3's post-training, eliminating human-written SFT data, and why RLHF outperforms imitation learning: humans are better discriminators than generators, enabling superhuman outputs. Scialom defends the dense 405B architecture over MoE, calls tokenizer vocab size underrated (128k tokens vs Llama 2's 32k), and reveals Llama 4's focus on agentic capabilities—tool use, multi-step reasoning—as the path to open-source AGI. Meta AI is hiring researchers with rigorous first-principles thinking.

A Comprehensive Overview of Large Language Models - Latent Space Paper Club

Mar 15, 2024 · 54:30

Brian presents a detailed walkthrough of the survey paper 'A Comprehensive Overview of Large Language Models,' which systematically covers the evolution of LLMs from early attention mechanisms to modern transformer architectures. He explains the shift from sequential RNNs to parallelizable transformers, the three dominant architectures (encoder-only like BERT, encoder-decoder like T5, decoder-only like GPT), and key training objectives (masked language modeling, full language modeling, prefix language modeling). The paper also reviews fine-tuning techniques (instruction tuning, alignment tuning via RLHF), prompting strategies (zero-shot, chain-of-thought), efficient adaptation methods (LoRA, quantization), evaluation benchmarks (GLUE, MMLU), and challenges such as bias, memorization, and privacy leaks. This episode serves as a comprehensive primer for anyone seeking to understand the foundational concepts and latest trends in large language model research.

Open Source AI is AI we can Trust — with Soumith Chintala of Meta AI

Mar 6, 2024 · 1:37:38

Soumith Chintala, creator of PyTorch and engineering lead at Meta AI, argues that open source AI is essential for distributing opportunity and trust. He details PyTorch's complexity—1,000 operators needed for generality—and explains synthetic data as a vehicle for imparting symbolic knowledge where humans already have good symbolic models. He highlights a coordination problem in open source: feedback is lost because frontends like Ooba and Ollama lack feedback buttons, and proposes a centralized sinkhole to collect high-quality feedback. Beyond text, he is excited about robotics, where hardware remains a bottleneck, and Osmo's work to digitize smell, which he compares to images in the 1800s.

A Brief History of the Open Source AI Hacker - with Ben Firshman of Replicate

Feb 28, 2024 · 1:21:09

Ben Firshman, CEO of Replicate, explains how the inference platform grew from a research reproducibility tool into a 2M-user API business by embracing the generative image community and treating open-source AI as a hacker-friendly ecosystem. They accidentally discovered their API when a user reverse-engineered their web form, leading to their first $1k/month customer. Cog, their container standard for ML models, was born from lessons at Docker and the need to make models tinkerable. Ben argues that fine-tuning's low cost makes open-source models sustainable, and that AI engineers (orders of magnitude more than ML engineers) just need to start playing with models. He also discusses GPU scarcity, preferring sustainable pricing over price wars, and reveals that demand is not outpacing supply thanks to aggregating demand.

The Origin and Future of RLHF: the secret ingredient for ChatGPT - with Nathan Lambert

Jan 11, 2024 · 1:35:27

Dr. Nathan Lambert traces the origin and future of Reinforcement Learning from Human Feedback (RLHF), the secret ingredient behind ChatGPT, explaining how it evolved from robotics and early preference learning to become the core alignment technique for large language models. He details the three-phase process (instruction tuning, preference data collection, RL optimization), noting that RLHF's data costs for LLaMA2 were around $6–8 million, and that synthetic data from GPT-4 is cheaper and often more accurate than human labels. Lambert contrasts DPO with PPO, arguing DPO is simpler but may have lower peak performance. He discusses emerging methods like Constitutional AI, which uses AI-generated critiques based on principles, and highlights the challenge of evaluating RLHF models, noting GPT-4 Turbo's lead over earlier versions. The episode covers open questions about data aggregation, reward model agreement (65–75%), and the need for qualitative model interaction.

Beating GPT-4 with Open Source Models - with Michael Royzen of Phind

Nov 3, 2023 · 1:18:54

Michael Royzen, co-founder and CEO of Phind, explains how his company built a GPT-4-beating open-source model for developer Q&A. Royzen recounts founding SmartLens in high school, shifting to NLP after a Hugging Face demo, and creating an internet-scale LLM-powered RAG system in January 2022. He details Phind’s pivot to programmers, the Hacker News launch that gave it 1,500 points, and Paul Graham’s role in naming the company and introducing Ron Conway, who then connected Phind to NVIDIA for GPU access. Royzen argues that Phind’s model, fine-tuned from Code Llama 34B with extra data, closes the gap with proprietary models—especially on code reasoning—and that open-source will win the enterprise because the delta to GPT-5 will be small. He shares how Phind handles multi-step conversations via a pair programmer mode where users can pin messages, and reveals plans for reinforcement learning to reduce hallucination and improve correctness.

RWKV: Reinventing RNNs for the Transformer Era

Aug 31, 2023 · 1:56:51

Eugene, CTO of UILicious, introduces RWKV, a recurrent neural network that matches transformer-level performance while scaling linearly with context length, solving the quadratic cost of attention. He explains how RWKV uses Attention-Free Transformer layers and parallelizable training, achieving competitive results on 7B and 14B models. The community-driven project prioritizes multilingual support, with a tokenizer that handles languages without spaces. Eugene recounts his journey from GPU.js to finding RWKV, and highlights use cases like long HTML analysis where transformers fail. He also discusses the token crisis, diffusion models for text, and how the AI waifu community drives optimization and alignment research. Finally, he advises AI engineers to focus on practical prompting and data curation, noting that deep architecture knowledge is optional.