A guide to building product in a post-LLM world | Ryan Glasgow and Kevin Mandich from Sprig

Sprig is an AI-powered user insights platform that has raised over $88m. Today’s discussion features two key individuals in Sprig’s journey so far: Ryan Glasgow, Sprig’s CEO and founder; and Kevin Mandich, Sprig’s Head of Machine Learning. Before Sprig, Ryan was an early PM at GraphScience, Vurb, and Weeby (all of which were acquired), and Kevin was an ML Engineer at Incubit, and a Post-Doctoral Researcher at UC San Diego. In today’s episode, we discuss: Key lessons from the Sprig founding story Product development in the pre vs. post-LLM world How to overcome AI skepticism How to evaluate new models and how to know when to switch Why you need an ML engineer Sprig’s “AI Squad” team structure How Sprig upskills all team members on AI Referenced: Auto-GPT: https://github.com/Significant-Gravitas/Auto-GPT Chat GPT: https://chat.openai.com Google’s BERT model: https://en.wikipedia.org/wiki/BERT_(language_model) Jira: https://www.atlassian.com/software/jira Jobs to Be Done Framework: https://hbr.org/2016/09/know-your-customers-jobs-to-be-done Langchain: https://www.langchain.com/ Sprig: https://sprig.com/ Where to find Ryan Glasgow: Twitter: https://twitter.com/ryanglasgow LinkedIn: https://www.linkedin.com/in/ryanglasgow/ Where to find Kevin Mandich: Twitter: https://twitter.com/kevinmandich LinkedIn: https://www.linkedin.com/in/kevinmandich/ Where to find Brett Berson: Twitter: https://twitter.com/brettberson LinkedIn: https://www.linkedin.com/in/brett-berson-9986094/ Where to find First Round Capital: Website: https://firstround.com/ First Round Review: https://review.firstround.com/ Twitter: https://twitter.com/firstround Youtube: https://www.youtube.com/@FirstRoundCapital This podcast on all platforms: https://review.firstround.com/podcast Timestamps (02:50) Intro (04:57) What attracted Kevin to Sprig (05:53) Kevin's background before Sprig (07:56) How Ryan gained conviction about Kevin (09:55) Key technical challenges and how they solved them (18:46) How to overcome AI skepticism (21:47) The early difficulties of building an ML-enabled product (25:06) Evaluating new models and knowing when to switch (35:09) Using Chat GPT (37:23) Product development in the pre vs. post-LLM world (39:53) The impact of AI hype on Sprig's product development (45:36) Balancing AI automation with user-psychology (48:47) Do recent LLMs reduce Sprig's competitive advantage? (51:00) The importance of "selling the vision" to customers (54:40) How Sprig structures teams (57:25) How Sprig upskills all team members on AI (60:25) 3 key tips for companies trying to navigate AI (66:05) Major limitations with LLMs right now (70:27) The future of AI and the future of Sprig