R.J. Honeckey

🔬 Why Transformers Hit a Wall the Moment Physics Shows Up — Anima Anandkumar, Caltech

Aug 26, 2026 · 1:23:32

Anima Anandkumar, Caltech professor and ex-NVIDIA lead, argues physical-world modeling needs neural operators that bake in physics, not language-style scaling; it already rivals supercomputers. Physics-informed nets fail; Fourier neural operators capture multi-scale non-local phenomena, and FourCastNet, trained on 50,000 reanalysis samples, forecasts weather tens of thousands of times faster on a consumer GPU. FourCastNet 3’s spherical harmonics keep month-long rollouts stable for ensemble climate prediction; the same operators build a fusion digital twin a million times faster than simulation and inverse design. TorchLean ports PyTorch to Lean for formal robustness proofs, and on the UN Science Advisory Board she urges AI for science not be regulated like chatbots. Bottleneck: more compute.

🔬Causal Models Need Causal Data - Xaira’s X-Cell model (Bo Wang & Ci Chu)

Jul 21, 2026 · 1:29:47

Bo Wang and Ci Chu from Xaira Therapeutics present X-Cell, a 4.9-billion-parameter diffusion language model trained on the largest genome-wide CRISPRi Perturb-seq dataset (25.6 million single cells, 16 biological contexts) that predicts cellular responses to genetic perturbations and generalizes from immortalized cell lines to primary T cells from real donors. They explain why observational atlases describe biology but can't predict interventions, why they abandoned autoregression for a diffusion 'editing' approach, and how a model trained on immortalized cells predicted perturbation responses in primary T cells. They highlight a counterintuitive scaling result: X-Cell scales like an LLM on training loss, but generalization is bottlenecked by data diversity, not compute. The episode also covers Xaira's three-platform strategy (protein design, virtual cell, patient representation) and the central thesis that for causal models, the hard part isn't the model, it's the data.