Microsoft

🔬 RL with Verifiable Rewards, but the Verifier is a Lab — Lila Sciences

Jul 16, 2026 · 1:41:04

Andy Beam (CTO) and Rafa Gómez-Bombarelli (Co-founder & CSO of Physical Sciences) of Lila Sciences argue that science is an 'infinite token generator' for AI, using reinforcement learning with verifiable rewards where the wet lab acts as the verifier. They claim one general model trained on ~10 trillion experimentally-verified reasoning tokens across biology, chemistry, and materials outperforms domain-specific models—'breadth gives us depth.' Their AI Science Factories treat the lab as a data center, with instruments on a 'PCI bus' and humans 'below the API line.' Highlights include a CAR-T candidate designed in six months by two or three people, 'monster UTRs' achieving ~10x Moderna/Pfizer mRNA expression, and the 'zero-FTE startup' business model. They discuss RL pathologies like collapsed chains of thought and a model that 'swears,' and note why there is still no AlphaFold for materials due to the sim-to-real gap.

🔬 The Limits of AI in Science - Why We Need Self-Driving Labs — Joseph Krause, Radical AI

Jun 17, 2026 · 1:16:50

Joseph Krause of Radical AI argues that the bottleneck in materials science is experiments, not ideas, and his company's self-driving lab combines AI hypothesis generation with automated synthesis and characterization to produce alloys at unprecedented speed—1,200 in six months, with 300 novel compositions and 10 already in commercial development. Radical's closed-loop system runs research campaigns, not just automated tasks, overcoming challenges like sample manipulation at 3,000°C and tool vendors' software access. Krause details how their AI explores elemental families humans overlooked, why they open-source models like Matrix (the moat is experimental data, not models), and how they plan to compress discovery timelines from decades to 3–5 years for defense and space applications. He addresses the 10-year qualification process for aerospace, supply chain geopolitics (e.g., hafnium price up 10–15x due to Chinese dominance), and the need for public-private partnerships to accelerate U.S. R&D. Finally, he urges ML engineers to lean into their expertise rather than try to become material scientists.

Satya Nadella on AI: @NoPriorsPodcast x Latent Space Crossover Special at Microsoft Build 2026

Jun 3, 2026 · 41:27

Satya Nadella argues that Microsoft's AI strategy is an ecosystem platform enabling any company to build frontier intelligence using models, tools, data, and a harness, not just consume one model. He outlines MAI training with clean data lineage, hill-climbing scaffolds, and private evals as core IP. The harness concept features multi-model harnesses with strong context layers, exemplified by GitHub Copilot and Work IQ turning M365 data into a database for agents. He notes coding agents required new IDE/UI, and long-running autopilots create value. Pricing evolves from per-user to consumption, and SaaS will unbundle and rebundle. Engineering generalists gain leverage; infrastructure roles like RLEs remain critical. Datacenter expansion requires community permission, with benefits in healthcare and rethinking education. Microsoft built more Azure capacity in 15 months than its first 15 years.

GitHub’s Agent Era: 14x Commits, 200M Developers, Copilot’s Next Act — Kyle Daigle

Jun 2, 2026 · 1:24:44

GitHub COO Kyle Daigle joins Swyx to unpack how the agent era is transforming GitHub's infrastructure and developer workflows, from 14x commit growth to internal AI micro-skills. Daigle describes using WorkIQ, MCP, and Slack/Teams context to retroactively analyze weeks' work and build executive presentations entirely with AI, undetected. He advocates atomic micro-skills over mega-skills and reveals GitHub now hosts 200M developers generating 275M weekly commits, on pace for 14B this year. Scaling issues stem from MySQL1 permissions and monorepo shifts, forcing rewrites of 10-year-old services. Copilot has evolved from code completion to a unified agent SDK powering CLI, desktop, and cloud agents. Microsoft's deep investment in OpenClaw reflects a need for OS-level sandboxing for enterprise agent deployment.

Dylan Patel Explains the AI War While Cooking | In-Context Cooking

Feb 26, 2026 · 55:13

Dylan Patel, CEO of SemiAnalysis, argues hyperscalers like Google, Amazon, and Meta will sacrifice all profits to build AI infrastructure, spending $180–$200 billion in capex this year alone, because the AI adoption explosion—Claude Code driving 4% of GitHub commits in one month, Anthropic adding $2.5 billion monthly revenue—makes it a Pascal's wager: spend or die. He details how Taiwan's semiconductor geopolitics create endgame scenarios, from a KMT win placating China to full invasion, with TSMC's output critical. Patel explains Nvidia's paranoid founder Jensen Huang is responding to vertical integration threats from hyperscalers by diversifying into chips like CPX and Groq, but warns moats are shallow. The real bottleneck in AI progress? Semiconductors themselves: fabs take years to build, and no one can buy enough GPUs through 2028. He also predicts a massive AI backlash from the public and financial markets, as capital consumption outpaces revenue and labor displacement accelerates.

Claude Code for Finance + The Global Memory Shortage: Doug O'Laughlin, SemiAnalysis

Feb 24, 2026 · 2:07:40

Doug O'Laughlin, SemiAnalysis founder, explains his Claude Code awakening in December 2024: Anthropic's Opus 4.5 one-shotted tasks that used to take 24 hours, leading him to predict Claude Code will write 25–50% of all GitHub code by year-end (already 4% of commits in two weeks). He argues this marks the death of Excel and Bloomberg for analysts—'you can just do things'—but warns of a 'hygiene' crisis as junior analysts lose meta-learning. On the semiconductor side, O'Laughlin details a severe memory squeeze: HBM demand consumes 4× the DRAM capacity, DRAM prices could double again, and CXL is reviving as a workaround. He compares AI infrastructure buildout to the railroad boom (25% of U.S. gross fixed capital investment) and says Microsoft faces an innovator's dilemma—renting GPUs to 'barbarians at the gate' while its own office franchise is disrupted by Claude Code. Google's TPU v7 enjoys a temporary TCO advantage, but Nvidia's supply chain dominance (Jensen doing shots with SK and Samsung) will reassert with Rubin. O'Laughlin also reflects on his 2,800-mile Continental Divide Trail hike as essential self-mastery.

⚡️ Reverse Engineering OpenAI's Training Data — Pratyush Maini, Datology

Feb 10, 2026 · 27:02

Pratyush Maini from Datology reveals how the seahorse emoji question exposes that frontier models like GPT-4.1, GPT-5, and OLMo 3.1 now exhibit self-correction loops—a sign that reasoning traces are baked into mid-training data, not just post-training. By tracking response length surges four months after o1's release and corroborating with OLMo's data, Maini argues that foundation models must contain core capabilities like self-reflection, reinforcing the 'Fine Tuner's Fallacy': you cannot simply fine-tune for a capability; it must be in pre-training. He also highlights Datology's BeyondWeb project, which uses a source-rephrasing paradigm to transform internet data at trillion-token scale, achieving Nemotron-level performance in 2.7x less compute—making specialized pre-training accessible for enterprises. The episode concludes that 2026-2027 will see a shift toward domain-specific pre-training, as the cost amortizes when a smaller pre-trained model outperforms a larger fine-tuned one.

One Year of MCP — with David Soria Parria and AAIF leads from OpenAI, Goose, Linux Foundation

Dec 28, 2025 · 1:39:19

David Soria Parra, MCP lead at Anthropic, along with Jim Zemlin (Linux Foundation CEO), Nick Cooper (OpenAI), and Brad Howes (Block/Goose), recount the one-year evolution of the Model Context Protocol from a local experiment to the de facto standard for agentic systems, now donated to the new Agentic AI Foundation. They detail four spec releases—from local stdio to remote HTTP streaming, OAuth 2.1 authentication (and enterprise lessons learned), long-running tasks, and MCP Apps (iframes for richer UI)—and explain why internal enterprise adoption is exploding faster than expected, mostly invisible and at massive scale. The group reveals how three competitive AI labs came together to donate protocols and agents to a neutral foundation, how the foundation will balance taste-making (curating meaningful projects) with openness, and their 2025 vision: MCP as the communication layer for asynchronous, long-running agents that discover and install their own tools, unlocking another order of magnitude in AI productivity.

[AIEWF Preview] Containing Agent Chaos — Solomon Hykes

Jun 3, 2025 · 27:14

Solomon Hykes, creator of Docker and founder of Dagger, argues that the industry needs an open standard for AI coding agent environments to avoid repeating the fragmentation seen before Docker. Dagger, a workflow engine originally built for post-development automation (CI/CD), is now being pulled by its community into the agentic space to solve environment isolation, portability, and observability for concurrent agent teams. Hykes insists containers are the right base layer but require a new, agent-native UX—unlike legacy Dockerfile or Compose—that is decoupled from any IDE, model, or cloud provider. He critiques current vertical monoliths (e.g., proprietary IDE-hosted agent environments) and advocates for a Lego-like modular approach where the environment is the standard linchpin. Dagger integrates with existing stacks rather than replacing them, and Hykes previews fresh, unreleased content for his upcoming keynote at AI Engineer World's Fair.

SF Compute: Commoditizing Compute

Apr 11, 2025 · 1:12:02

Evan Conrad, co-founder of SF Compute, argues that GPUs behave like a real estate business, not a traditional cloud, because price-sensitive customers value every incremental GPU and will switch for a 10% margin. CoreWeave succeeded by selling locked-in long-term contracts to low-credit-risk customers like Microsoft and OpenAI, ignoring short-term demand. He predicts hyperscalers and providers like Together and DigitalOcean will lose money on GPU clusters because software margins cannot match the hardware costs. SF Compute started as an AI lab forced to sublease its cluster monthly to avoid bankruptcy, then evolved into a market where anyone can buy H100s by the hour via dynamic pricing—often below $1/hour for short bursts. Utilization stays near 100% as prices adjust. Future plans include cash-settled futures to reduce financial risk across the industry, while the brand deliberately stays anti-hype and calm.

The AI Architect: Bret Taylor

Feb 11, 2025 · 1:35:59

Bret Taylor, CEO of Sierra and chairman of OpenAI, recounts engineering AI's future across three eras — rewriting Google Maps' frontend in a weekend (cutting un-gzipped code to 20K), building Sierra's in-house agent platform for consumer brands like Sonos and SiriusXM, and navigating OpenAI's leadership crisis. He argues that AI agents' core abstractions are still pre-React, compares today's agents to jQuery, and predicts that outcome-based pricing will dominate as agents complete jobs rather than assist humans. He urges software engineers to move up the stack toward domain-specific agents and to embrace Rust-like languages for AI-generated code, emphasizing formal verification and safety. Taylor also shares how OpenAI prioritizes mission over product features, and why the relationship with Microsoft remains OpenAI's most important partnership.

Best of 2024: Synthetic Data / Smol Models, Loubna Ben Allal, HuggingFace [LS Live! @ NeurIPS 2024]

Dec 24, 2024 · 28:08

Loubna Ben Allal of Hugging Face explains how synthetic data has become ubiquitous in LLM pipelines, moving from post-training to pre-training, with examples like Cosmopedia’s 30B-token synthetic dataset and NVIDIA’s 1.9T-token Nemotron CC. She addresses model collapse fears, showing that web dumps after ChatGPT’s release actually yield better models, and emphasizes diversity through prompt seeds and webpage extracts. For filtering, FineWeb-Edu and DCLM use LLMs to rate educational content, achieving top benchmark scores. In post-training, she highlights AgentInstruct, Tülu3 with PersonaHub, and Cohere’s multilingual arbitrage using multiple teachers. On small models, SmolM2 (1.7B) outperforms Llama 1B and Qwen 2.5 after 11T tokens of pre-training, and on-device inference via frameworks like llama.cpp enables privacy-preserving use cases such as text extraction and structured generation. She predicts a return to fine-tuning specialized small models over costly prompt engineering.

Why Google failed to make GPT-3 -- with David Luan of Adept

Mar 27, 2024 · 49:27

David Luan, co-founder of Adept and former early OpenAI leader, explains why Google failed to build GPT-3—its 'brain credit marketplace' prevented critical mass—and why Adept builds enterprise AI agents prioritizing reliability over generality. He recounts the GPT-2 demo that helped secure Microsoft's $1B investment and details Adept's goal: an AI teammate that can do anything a human does on a computer, targeting 'nines of reliability' for workflows like dispatching a physical truck. Luan contrasts Adept's vertical integration—training fast multimodal models (Fuyu) for charts and UIs—with pure-play foundation model companies that sell tokens, predicting commoditization. He notes Adept is sold out for Q1 and raised $420M, and explains how an augmentation focus creates a data flywheel from human oversight.

The State of Silicon and the GPU Poors - with Dylan Patel of SemiAnalysis

Dec 5, 2023 · 1:07:55

Dylan Patel of SemiAnalysis argues the 'GPU poor vs GPU rich' divide defines AI, with Google's TPU ramp (millions of units) dwarfing others and Nvidia selling over a million H100s this year. He explains that training costs are irrelevant compared to inference costs, and that memory bandwidth is the key bottleneck for LLM inference (e.g., Llama 70B needs ~2.1 TB/s for human reading speed). Patel advises GPU-poor players to focus on on-device innovations like speculative decoding (Medusa), asynchronous training, or fine-tuning for edge use—not on gaming benchmarks or fine-tuning small models. He details why AI hardware startups (Cerebras, Graphcore) bet wrong on on-chip SRAM, while Google’s TPU/Broadcom partnership and Nvidia’s yearly cadence make alternatives tough. He also warns that rebuilding the semiconductor supply chain in the US is unfeasible due to its fragmentation, and that safety through obscurity doesn't work—open innovation is better.