This episode with Lukasz Kaiser, co-author of the seminal "Attention Is All You Need" transformer paper and former researcher at both Google Brain and OpenAI, is a wide-ranging conversation about the fundamental limits of current AI architectures and whether transformers will continue to dominate or eventually give way to something new. Lukasz brings a rare dual perspective: deep belief in how far the current paradigm has taken us (he's an enthusiastic daily Codex user who's seen 10x productivity gains in his own research), while maintaining genuine intellectual humility about whether transformers can truly generalize the way humans do. The episode weaves together questions about data efficiency, the non-verifiable RL frontier, the coding agent revolution, the open vs. closed source gap, and what the next architectural leap might look like: all filtered through the lens of someone who helped build the foundation the entire field is standing on.
(0:00) Intro
(1:12) Transformers vs. Human Learning
(8:37) How Do We Get Physical World Generalization?
(10:52) What Comes After Transformers
(13:59) How Much Have Agents Improved Lukasz's AI Research Productivity?
(17:21) How Close Is an AI Research Intern?
(26:06) RL Beyond Verifiable Tasks
(35:38) App Companies: Build Models or Lean on Labs?
Podden och tillhörande omslagsbild på den här sidan tillhör
by Redpoint Ventures. Innehållet i podden är skapat av by Redpoint Ventures och inte av,
eller tillsammans med, Poddtoppen.
