About this episode
We unpack how continued pre-training turns a general AI into a coding and math specialist. From Meta's CodeLlama to DeepSeek's findings on code-based learning and Nvidia's synthetic debates, we explore model souping, ultra-long contexts (131k tokens), and why training on code can sharpen logic and mathematical reasoning. We discuss what this means for solving real-world scientific and engineering challenges—and what human-style conversation can unlock next in AI.
Note: This podcast was AI-generated, and sometimes AI can make mistakes. Please double-check any critical information.
Sponsored by Embersilk LLC
Get every episode summarized
Each time Intellectually Curious publishes, we email you a written briefing from the transcript — the topics, who appeared, and any specific claims, with the ad reads skipped.
Email me new episodesFree for 3 shows. No card needed.
Hosts & guests
No transcript yet
This episode has not been transcribed. Request it and it moves to the front of the queue.
More episodes
More from Intellectually Curious
Free Pause Tokens Solve AI Multitasking
Intellectually Curious
Claude’s Autonomous Formalization of Fermat’s Last Theorem
Intellectually Curious
Random Attention: How AI Gets Faster by Forgetting
Intellectually Curious
The Alien Anatomy of the Bigfin Squid
Intellectually Curious