Ground-Truth-as-Code: Testing AI Agents on Live Data
Get every episode summarized
Each time Intellectually Curious publishes, we email you a written briefing from the transcript — the topics, who appeared, and any specific claims, with the ad reads skipped.
Email me new episodesFree for 3 shows. No card needed.
About this episode
How do you grade an AI agent when the correct answer changes with the data? We explore ground-truth-as-code: an evaluation approach that reruns reference calculations against live systems instead of relying on stale answer sheets. Discover how breaking responses into individual facts helps measure correctness and completeness—and why reliable testing matters for data science agents in production.
Note: This podcast was AI-generated, and sometimes AI can make mistakes. Please double-check any critical information.
Sponsored by Embersilk LLC
Get every episode summarized
Each time Intellectually Curious publishes, we email you a written briefing from the transcript — the topics, who appeared, and any specific claims, with the ad reads skipped.
Email me new episodesFree for 3 shows. No card needed.
Hosts & guests
No transcript yet
This episode has not been transcribed. Request it and it moves to the front of the queue.
More episodes
More from Intellectually Curious
Hark Pro: From AI Chat to Getting Things Done
Intellectually Curious
Optogenetics: Restoring Sight and Hearing With Light
Intellectually Curious
Centaurs: Wandering Worlds Between the Giant Planets
Intellectually Curious
Google’s AI Film Crew: Keeping Characters and Stories Consistent
Intellectually Curious