Bob McGrew, OpenAIs former Head of Research, led OpenAI from the GPT-3 breakthrough to todays reasoning models. Three main pillars of AGITransformers are scaled pre-training, post-training and reasoningand that the fundamentals that will shape the next decade-plus are already in place. He thinks 2025 will be defined by reasoning while pre-training hits diminishing returns. AI agents will eventually price all services at compute costs due to near-infinite supply. This will fundamentally disrupting industries like law and medicine. From robotics breakthroughs to managing brilliant researchers, Bob offers a unique perspective on AIs trajectory and where startups can still find defensible opportunities.
The fundamental concepts that are needed for AGI are pre-training, post-training and reasoning. Those are all that are needed and all that will be developed for the next ten years.
The supply of currently difficult skills (like legal expertise) will rapidly expand. This will make those currently expensive skills cheap. What will be valuable will be human and business relationships.
00:00 Introduction 01:16 The Trifecta of AI: Pre-training, Post-training, and Reasoning 02:19 Deep Dive into AI Reasoning 03:53 Challenges and Future of Pre-training 05:23 Exploring Post-training and Model Personality 06:42 The Future of AI: Predictions and Controversies 11:44 The Rise of AI Agents and Market Opportunities 15:46 Robotics: The Next Frontier 21:31 Proprietary Data and Its Value in AI 24:22 Proprietary Data and Its Importance 24:29 The Rapid Evolution of Coding 25:25 The Future of Coding: Human vs. AI 27:29 The Role of AgTech Software Engineers 29:14 The Concept of Member of the Technical Staff 31:11 Generational Differences in Using ChatGPT 32:41 AIs Role in Enhancing Learning and Curiosity 34:45 Preparing the Next Generation for AI 38:57 Daily Uses of AI 41:03 Managing High-Performing Teams 46:39 Security in an Agentic World
48:17 Conclusion and Final Thoughts
