AMD

The Inference Frontier: from 100 to 10,000 tokens per second — Sean Lie, Cerebras CTO

Sep 2, 2026 · 44:03

A comedic episode parodying AI hardware marketing, featuring Sean Lie discussing fictional Cerebras CS4 chips with absurd specs like 30 billion parameter models running at 4,400 TPS with wafer-scale integration and jalapeño-flavored SRAM designs.

Next 100x in AI: Inference, Networking, & Self-Optimizing Models — Philip Kiely & Ali Taha, Baseten

Aug 3, 2026 · 1:42:54

Philip Kiely and Ali Taha of Baseten join Swyx to explain what actually happens when a 200,000-token request hits a production inference system, arguing that stacking quantization, speculative decoding, and disaggregated prefill/decode can make open models like GLM-5.2 up to 10x faster. They detail cache-aware routing, training traffic-specific speculators, and why quantization errors can cancel out so a more-quantized model beats a less-quantized one. They also reveal how Baseten grafted Kimi's vision encoder onto GLM-5.2, discuss NVIDIA Dynamo as a toolkit rather than a turnkey speedup, and explain why they are bearish on mega kernels. The conversation covers video generation's compute barriers, the trend toward ASIC-like GPUs, and the emerging loop where GLM-5.2 writes the GPU kernels that serve itself.

⚡️ Google's Open AI Strategy — Omar Sanseviero, Google DeepMind

May 24, 2026 · 29:59

Omar Sanseviero, Google DeepMind's Head of Developer Experience, explains Gemma 4's novel architecture with per-layer embeddings that enable effective parameter offloading: only 2B of 5B parameters need GPU memory, ideal for on-device inference on phones and Raspberry Pis. The model matches 1.5-year-old state-of-the-art in most areas, with Gemini Nano integrated into Pixel and Samsung phones. Gemma 4 supports multimodal input (audio, images, short video) but not audio output or combined audio-video prompts. Sanseviero notes fine-tuning is declining as out-of-box capabilities improve, but remains relevant for specialized domains like healthcare. He contrasts dense (31B) and MoE (27B) variants, highlighting MoE's inference speed but fine-tuning challenges. The team is expanding globally, with Kaggle joining DeepMind to create community-driven benchmarks for model evaluation.

Dylan Patel Explains the AI War While Cooking | In-Context Cooking

Feb 26, 2026 · 55:13

Dylan Patel, CEO of SemiAnalysis, argues hyperscalers like Google, Amazon, and Meta will sacrifice all profits to build AI infrastructure, spending $180–$200 billion in capex this year alone, because the AI adoption explosion—Claude Code driving 4% of GitHub commits in one month, Anthropic adding $2.5 billion monthly revenue—makes it a Pascal's wager: spend or die. He details how Taiwan's semiconductor geopolitics create endgame scenarios, from a KMT win placating China to full invasion, with TSMC's output critical. Patel explains Nvidia's paranoid founder Jensen Huang is responding to vertical integration threats from hyperscalers by diversifying into chips like CPX and Groq, but warns moats are shallow. The real bottleneck in AI progress? Semiconductors themselves: fabs take years to build, and no one can buy enough GPUs through 2028. He also predicts a massive AI backlash from the public and financial markets, as capital consumption outpaces revenue and labor displacement accelerates.

How Zyphra went all-in on AMD + Why Devs feel faster with AI but are slower — with Quentin Anthony

Nov 3, 2025 · 1:00:02

Quentin Anthony, head of model training at Zyphra and advisor at EleutherAI, explains why his company moved all training to AMD MI300X GPUs, outperforming Nvidia H100s on certain workloads thanks to 192GB VRAM and higher memory bandwidth, and describes his kernel development approach of writing directly in ROCm or GPU assembly rather than using Triton. He also shares his experience in the METR study on AI coding productivity, where he was one of the few developers with measurable speedup, and offers tips: timebox AI use, avoid the slot machine effect, maintain context hygiene, and use direct API over tools like Cursor. Additionally, he argues that open source AI research benefits from siloed focused teams with guaranteed funding over grand collaborations, and notes that kernel datasets alone won't solve GPU programming due to evaluation challenges.

⚡️Accelerators @ 3x NVIDIA H200 perf, Made in the USA - Thomas Sohmers + Mitesh Agrawal, Positron AI

Aug 18, 2025 · 47:56

Thomas Sohmers and Mitesh Agrawal of Positron AI argue memory bandwidth, not compute, is the true bottleneck in AI inference, and their accelerator achieves 93% memory bandwidth utilization—triple NVIDIA's efficiency—enabling 70% faster token generation at 150W. The founders, both Lambda Labs veterans, shipped an FPGA product 15 months after founding, then raised a $51M Series A for an ASIC in late 2026. Their hardware requires no recompilation: it ingests raw binary weights from NVIDIA training and outputs an OpenAI-compatible API. Positron focuses on the decode phase of transformers, where memory-bound matrix-vector multiplication dominates, and already counts Cloudflare and Parasail as customers. The company sells systems directly, prioritizing capital efficiency and ROIC over operating its own cloud.

The Shape of Compute (Chris Lattner of Modular)

Jun 13, 2025 · 1:18:18

Chris Lattner of Modular explains how his company is breaking the CUDA monopoly with Mojo and MAX, matching NVIDIA's best inference performance on AMD and NVIDIA GPUs. After three years in R&D, Modular open-sourced its stack and now delivers state-of-the-art Llama 3 serving at over 800 tokens per second. Mojo, a Python-family language, runs kernels faster than Rust and can extend Python without bindings, while MAX provides a full inference framework with automatic kernel fusion and cluster management. Unlike VLLM or SGLang, Modular's container is just a gigabyte, fully open source, and not reliant on proprietary CUDA blobs. DeepSeek's low-level PTX work validated the approach, but Lattner emphasizes that Mojo's portability avoids rewriting kernels for each new GPU architecture. He also details his daily routine, using Cursor for coding, and hiring "elite nerds" to grow the team.

AI Engineering for Art - with comfyanonymous

Jan 4, 2025 · 52:09

Comfy Anonymous, the anonymous creator of ComfyUI, details in his first-ever podcast interview how his node-based image generation tool overtook Automatic1111 through superior memory management and early support for SDXL, becoming the de facto interface for advanced diffusion workflows. He started coding on January 1, 2023, and released the first version on January 16, driven by a desire to chain models and experiment with custom samplers. His work at Stability AI from June 2023 ensured ComfyUI efficiently handled SDXL, which leaked via early access and forced users from less powerful GPUs to adopt his tool. He discusses model preferences: Flux for consistency, SD 3.5 for creativity, and SD1.5 remaining popular. For video, he highlights Mochi as a 'true' video model with 3D latents, already implemented in Comfy. Now with a core team, ComfyUI is targeting a v1 release with an easy installer on Windows and Mac, while planning monetization through cloud and enterprise features—but keeping the open-source core free.

llm.c's Origin and the Future of LLM Compilers - Andrej Karpathy at CUDA MODE

Sep 21, 2024 · 23:29

Andrej Karpathy explains llm.c, his from-scratch C implementation of GPT-2 training that outperforms PyTorch by 20% in speed and 30% in memory, arguing that LLMs will eventually act as compilers for custom code. He recounts starting the project out of frustration with PyTorch compile errors, writing all forward and backward passes manually in C, then porting to CUDA with kernel optimizations. A community of contributors (Eric, Arun, Alex R) helped achieve 50% MFU on a single H100 node, training GPT-2 1.6B in 24 hours for $600. Ongoing work adds Llama 3.1 and FP8 support. Karpathy sees llm.c as proof that LLMs can automate such low-level optimization, making Python and PyTorch a temporary crutch.

The State of Silicon and the GPU Poors - with Dylan Patel of SemiAnalysis

Dec 5, 2023 · 1:07:55

Dylan Patel of SemiAnalysis argues the 'GPU poor vs GPU rich' divide defines AI, with Google's TPU ramp (millions of units) dwarfing others and Nvidia selling over a million H100s this year. He explains that training costs are irrelevant compared to inference costs, and that memory bandwidth is the key bottleneck for LLM inference (e.g., Llama 70B needs ~2.1 TB/s for human reading speed). Patel advises GPU-poor players to focus on on-device innovations like speculative decoding (Medusa), asynchronous training, or fine-tuning for edge use—not on gaming benchmarks or fine-tuning small models. He details why AI hardware startups (Cerebras, Graphcore) bet wrong on on-chip SRAM, while Google’s TPU/Broadcom partnership and Nvidia’s yearly cadence make alternatives tough. He also warns that rebuilding the semiconductor supply chain in the US is unfeasible due to its fragmentation, and that safety through obscurity doesn't work—open innovation is better.

Ep 18: Petaflops to the People — with George Hotz of tinycorp

Jun 20, 2023 · 1:23:24

George Hotz of tinycorp argues that a simplified, open-source ML framework (tinygrad) can democratize AI compute and replace NVIDIA, Google, and AMD. Tinygrad uses 25 ops (vs. XLA's 250) and fuses kernels automatically, achieving 2x speed on Qualcomm GPUs over their library. Hotz reports AMD kernel panics fixed after emailing CEO Lisa Su, but laments AMD's open-source as 'dumped on GitHub.' For hardware, tinybox is a six-GPU desktop at 350W per GPU, aiming for 5x better price-performance than NVIDIA H100 for 90% of training. He criticizes OpenAI's secrecy, revealing GPT-4 is an 8-way mixture of 220B models, and cites the Bitter Lesson. He also outlines FLOPcoin using hardware identity to prevent cheating, and his goal of an AI girlfriend as the third company, merging humans via data rather than implants.