The Product Experience: a Mind the Product podcast

Jonathan Evens is a product lead at Google DeepMind, where he has spent more than a decade applying machine learning and AI across industries — from the smart grid at AutoGrid, to detecting roads and buildings from satellite imagery at Planet, to recommender systems, Google Search's AI Overviews and AI Mode, and now live avatars. He is also an advisor to the Evens Foundation, where he is building a "digital citizenry": a democracy sandbox that uses synthetic citizens to pre-test how the public might react to a policy before it is written.
Jonathan returns to The Product Experience, where hosts Lily Smith and Randy Silver pick up the conversation they started at MTPcon London, to dig further into what actually separates an AI product manager from a product manager who simply uses AI tools, why product principles have to come before evaluations, and how synthetic users can help — and mislead — at very different scales of product.
We discuss:
1. Why "AI product manager" has become a near-meaningless label, and the two distinct roles hiding underneath it: the modelling product manager working on core model capabilities, and the AI feature product manager building AI-powered products
2. Why using an LLM as a thinking partner or a coding assistant does not make someone an AI product manager — it makes them a product manager using AI tools, full stop
3. How Google Search's North Star metrics have stayed constant even as the proxy metrics beneath them — side-by-side win rates, user ratings, RLHF signals — have had to be rebuilt from scratch
4. Why product principles, not evaluations, are the real starting point for any AI feature, and how Google Search resolved the problem of trustworthy sources disagreeing on basic facts
5. How Google Search builds trust into its AI Overviews through sourcing, citation placement and UX cues such as highlighting, so users can judge at a glance what to verify
6. Where synthetic users genuinely help — cold-start problems, privacy-sensitive research, automated regression testing — and where they fall short
7. Building the Evens Foundation's "digital citizenry", and the core technical problem behind it: AI-generated personas that are less diverse and more extreme than real people
8. How team size and structure differ between a fully resourced lab like Google DeepMind and a resource-constrained non-profit team, and why Jonathan resists a single answer for the "right" team size
9. How the product manager's job is shifting as engineers absorb more of the evaluation work themselves through prompting and iteration
10. Jonathan's advice for product managers building AI features, and his case for following the Makers Manifesto
 Key takeaways
"AI product manager" covers two distinct jobs. The modelling product manager defines and measures a model's core capabilities — factuality, reasoning, long context — and that role is concentrated almost entirely inside frontier labs. The AI feature product manager builds a product or feature on top of an existing model, and needs domain expertise and user empathy far more than technical depth. Conflating the two is why the title has become so diluted.
Using an LLM to think faster or write code faster does not make someone an AI product manager. It makes them a product manager using AI as part of their toolkit — the same as any other knowledge worker. The distinction matters because it clarifies what skills are actually being tested.
Product principles have to come before evaluations, not after. Before Jonathan starts building an eval set for a new product, he first asks what the product is meant to feel like and what values it should encode. Google Search's response to sources disagreeing on a monument's construction date, or to large language models hallucinating at scale, came from principles about trustworthiness established before any metric was built.
Trust in an AI feature is built through sourcing and interface design as much as through the model itself. Google Search's AI Overviews are constrained to draw only from ranked, trustworthy documents rather than the model's own memory, and users are given UX signals — citation placement, highlighting — that let them judge at a glance how much to verify.
Synthetic users add genuine value in cold-start scenarios, privacy-sensitive research and automated regression testing. Where they fall short is diversity: AI-generated personas tend to be less varied and more extreme than real people, which is the central technical problem behind the Evens Foundation's digital citizenry project.
There is no fixed answer to the right team size. Jonathan sees a gradient, from a senior developer working entirely alone, up to the Evens Foundation's single product manager with AI-assisted development skills, up to a fully staffed Google team — with the deciding factor being how unsolved the underlying problem is, not company size.
As engineers absorb more evaluation work themselves through prompting and iteration, roles are blending. What still sits with product management is the judgement calls that follow from product principles — deciding, for example, which technical trade-offs actually matter to the use case, rather than which are easiest to measure.
Features links
- Evens Foundation — https://evensfoundation.eu
- Makers Manifesto — https://makersmanifesto.org
- Google DeepMind — https://deepmind.google
- AutoGrid — smart grid AI company where Jonathan began applying machine learning to industry
- Planet — satellite imagery company where Jonathan worked on automated road and building detection
We're refreshing The Product Experience and want your input. Take our two-minute survey and help shape where the show goes next! 
Our Hosts
Lily Smith enjoys working as a consultant product manager with early-stage and growing startups and as a mentor to other product managers. She’s currently Chief Product Officer at BBC Maestro, and has spent 13 years in the tech industry working with startups in the SaaS and mobile space. 
Randy Silver is a Leadership & Product Coach and Consultant. He gets teams unstuck, helping you to supercharge your results. Randy's held interim CPO and Leadership roles at scale-ups and SMEs, advised start-ups, and been Head of Product at HSBC and Sainsbury’s. He participated in Silicon Valley Product Group’s Coaching the Coaches forum, and speaks frequently at conferences and events. You can join one of communities he runs for CPOs (CPO Circles), Product Managers (Product In the {A}ether) and Product Coaches. He’s the author of What Do We Do Now? A Product Manager’s Guide to Strategy in the Time of COVID-19. A recovering music journalist and editor, Randy also launched Amazon’s music stores in the US & UK.

Kirsten Mann is a board director, product leader and, with this episode, a three-time guest on the podcast. Last time she was here, she was building Vizory, an AI tool that helps company directors get through board packs. This time she's back to explain why she isn't building it any more. Talking to Randy Silver, she walks through exiting Vizory, the friction tests she ran before writing a line of code, and why she now believes distribution, not the build, is where most AI products actually die.
We're refreshing The Product Experience and want your input. Take our two-minute survey and help shape where the show goes next! 
Key takeaways
Before writing a line of code, Mann tested for force rather than sentiment: she asked directors for two confidential board packs and an audio recording of themselves reviewing one, and only started building once 10 people agreed to do it. Coffee-shop enthusiasm is cheap; handing over a company's most sensitive document is not.
Vizory's most-loved feature in demos, cross-pack search, was barely used once shipped. Directors said they wanted it and reacted strongly when they saw it, but in practice they went straight to an automated triage view instead — a clean case of stated preference diverging from revealed preference.
Board software runs on quarterly, not weekly, cycles, and trust compounds slower than value does. Mann modelled two review cycles before directors would rely on the tool; in practice it took three, meaning some customers on quarterly boards needed close to a year before the value became obvious, a timeline no 30-day trial can accommodate.
Pricing got caught in a double anchoring trap. Directors first benchmarked Vizory against the $20-a-month ChatGPT and Claude price point, so Mann positioned against the cost of a governance failure instead, then created a second anchor by pricing early pilots at $150 a seat against a $300 target, which made the later increase read as a 100% price rise.
Cutting friction is the wrong instinct early on. Mann deliberately added friction, asking for sensitive documents and charging as early as possible, to separate real demand from polite interest, arguing that most founders run tests designed to confirm they're right rather than to find a reason to stop.
Mann's main lesson for other founders is that distribution is now the product problem. AI has collapsed the cost of building software but done nothing to the cost of reaching people who trust you, which is why she pushed to add distribution, adoption and willingness to pay as a third discipline in the Makers' Manifesto, alongside building the right thing and building it right.
Multi-model products carry a new operational risk. One of Vizory's AI "judges" silently stopped being selected because the underlying model had been deprecated overnight without warning, a reminder that model drift means the QA and evals work never really stops, however fast the initial build was.
We're refreshing The Product Experience and want your input. Take our two-minute survey and help shape where the show goes next! 
Our Hosts
Lily Smith enjoys working as a consultant product manager with early-stage and growing startups and as a mentor to other product managers. She’s currently Chief Product Officer at BBC Maestro, and has spent 13 years in the tech industry working with startups in the SaaS and mobile space. 
Randy Silver is a Leadership & Product Coach and Consultant. He gets teams unstuck, helping you to supercharge your results. Randy's held interim CPO and Leadership roles at scale-ups and SMEs, advised start-ups, and been Head of Product at HSBC and Sainsbury’s. He participated in Silicon Valley Product Group’s Coaching the Coaches forum, and speaks frequently at conferences and events. You can join one of communities he runs for CPOs (CPO Circles), Product Managers (Product In the {A}ether) and Product Coaches. He’s the author of What Do We Do Now? A Product Manager’s Guide to Strategy in the Time of COVID-19. A recovering music journalist and editor, Randy also launched Amazon’s music stores in the US & UK.

April Dunford is the positioning expert behind Obviously Awesome and a former repeat vice-president of marketing across seven venture-backed startups, most of which were acquired. Over the past decade she has advised roughly 300 B2B technology companies, all with sales teams, on how to position themselves against the competition. In this episode she joins hosts Lily Smith and Randy Silver to explain why so many companies are repositioning at once, how a strong point of view on the future underpins good positioning, and why the story a company tells about itself is a decision for product and sales, not just marketing.
We're refreshing The Product Experience and want your input. Take our two-minute survey and help shape where the show goes next! 

Key takeaways
Positioning is a company-wide decision, not a marketing exercise. Dunford's own experience repositioning a spreadsheet tool as an embeddable database shows how it reshapes pricing, sales motion and even the product roadmap, not just the words on a website.
Major shifts such as the move to AI force widespread repositioning because customers no longer have a stable set of alternatives to judge value or price against, and both AI-native companies and traditional SaaS businesses are wrestling with where they now sit.
Intercom's rebuild of its support product around Fin shows how staying anchored to the customer's underlying job, resolving a ticket, can guide a genuine AI-first redesign rather than a bolt-on feature.
Companies need a clear, credible point of view on the future of their market, or customers facing genuine uncertainty will simply freeze and delay buying decisions.
A compelling vision of the future has to flow from a company's real differentiated value, not be reverse-engineered to flatter its current position.
Selling too far into a hyped future risks stalling deals, so the message needs a maturity model that shows customers a credible, low-risk path from where they are today to where the market is heading.
Positioning should be revisited on a disciplined schedule rather than settled by whoever argues loudest in the room, and Dunford recommends a structured check every six months on competitors, differentiators and value themes.
Customer advisory boards, quarterly executive account calls and a company's own sales team are the most reliable early-warning systems for shifts in budget, buying process and competitive threats, long before they show up as lost deals.
Chapters
(00:00) Introduction to April Dunford 
(01:53) The story behind Obviously Awesome 
(05:33) Positioning as a cross-functional decision 
(07:23) Why everyone is repositioning right now 
(09:32) Intercom, Fin and the AI-native rebuild 
(11:53) Building a point of view of the future 
(16:52) Is the vision genuine or self-serving 
(18:00) How to develop your point of view 
(22:13) Stack Overflow, Salesforce and staying silent 
(23:45) Hyping the future without stalling sales 
(27:01) When positioning needs to change 
(28:42) Running a regular positioning check-in 
(30:29) Customer advisory boards and voice of customer 
(34:47) Predicting change after covid and AI 
(37:20) Wrap-up

We're refreshing The Product Experience and want your input. Take our two-minute survey and help shape where the show goes next! 
Our Hosts
Lily Smith enjoys working as a consultant product manager with early-stage and growing startups and as a mentor to other product managers. She’s currently Chief Product Officer at BBC Maestro, and has spent 13 years in the tech industry working with startups in the SaaS and mobile space. 
Randy Silver is a Leadership & Product Coach and Consultant. He gets teams unstuck, helping you to supercharge your results. Randy's held interim CPO and Leadership roles at scale-ups and SMEs, advised start-ups, and been Head of Product at HSBC and Sainsbury’s. He participated in Silicon Valley Product Group’s Coaching the Coaches forum, and speaks frequently at conferences and events. You can join one of communities he runs for CPOs (CPO Circles), Product Managers (Product In the {A}ether) and Product Coaches. He’s the author of What Do We Do Now? A Product Manager’s Guide to Strategy in the Time of COVID-19. A recovering music journalist and editor, Randy also launched Amazon’s music stores in the US & UK.

Julia Barham leads a CX acquisition team at Progressive Insurance and is the author of The Product Management Playbook, published by Rosenfeld Media in July 2026. She wrote it to close what she sees as a coaching gap in the profession: a decade of product transformations has produced far more people with the title than people who have been taught end-to-end craft. In this episode she makes the case for treating ambiguity as a condition of the job rather than a problem to be solved, and walks through the things that quietly break when a product starts to scale.
We're refreshing The Product Experience and want your input. Take our two-minute survey and help shape where the show goes next! 
Key takeaways
Frameworks come and go, but the questions endure. Julia structured her book around why something matters, who it is for, what needs building and when, because no methodology eliminates ambiguity, it only reduces it.
Organisational ambiguity is the least discussed of the four ambiguities PMs face, and the one that separates individual contributors from leaders. Waiting for the perfect operating model, funding model or team shape is a way of not moving.
If you have to be in every room to answer every question, you are scaling chaos. Durable artefacts, bug severity rubrics and self-service dashboards let the team make calls without you.
Synthesis is where shared context gets built, and offloading it to an AI tool quietly removes the thing that made it valuable. Do it together, monthly, with the customer, platform and business data in the same room.
Chasing growth before product market fit turns your economics upside down. Products commonly take two years to find fit, sometimes five, and retention is the strongest signal that you have it.
Julia spent nine months fixing latency on an inherited platform because of shortcuts taken at launch. The cost was measured in millions of lost revenue, not just engineering time.
Attaching a monetary value to technical debt is what turns a product manager from a stick in the mud into a steward of the business. Treat your product like a P&L whether you own one or not.
A product at scale needs a portfolio, not a backlog: optimisation, strategic bets, capability work, and honest capacity set aside for run-the-business support.
AI belongs in the toolkit, not in the chair. If you cannot call BS on the output, you do not know enough to be doing the job.
Chapters
(00:00) Don't scale chaos
(01:01) Introducing Julia Barham
(03:01) The coaching gap that prompted the book
(03:57) Why the book is built around questions, not frameworks
(06:06) Has product management ever been deterministic?
(07:47) The four ambiguities every product manager faces
(10:00) Does organisational ambiguity change with company size?
(12:57) The 21 plays and how to use them
(15:35) Getting your reps in across the product lifecycle
(17:07) What breaks when products and teams scale
(17:59) Mistake one: needing to be in every room
(20:30) Building shared context across distributed teams
(23:51) The ways of working exercise
(24:19) Mistake two: chasing growth before product market fit
(25:56) How to measure product market fit
(27:16) Mistake three: scaling expensive problems
(29:42) Putting a price on technical debt
(31:54) When taking on debt is the right call
(34:22) Naming the mode your product is in
(35:09) Mistake four: running your product as a portfolio
(37:39) The one problem Julia would make disappear
(39:25) What AI changes, and what it does not
(42:26) Wrap-up
Referenced
The Product Management Playbook, Julia Barham, Rosenfeld Media: https://rosenfeldmedia.com/books/product-management-playbook/
Use discount code MTP20 for 20% off at Rosenfeld Media
Sample chapter: https://rosenfeldmedia.com/sample-chapter-the-product-management-playbook/
Dan Olsen and the product market fit pyramid: https://dan-olsen.com
Testing Business Ideas, David Bland
Progressive Insurance: https://www.progressive.com
Pioneers, settlers and town planners, Simon Wardley: https://blog.gardeviance.org/2015/03/on-pioneers-settlers-town-planners-and.html
We're refreshing The Product Experience and want your input. Take our two-minute survey and help shape where the show goes next! 
Our Hosts
Lily Smith enjoys working as a consultant product manager with early-stage and growing startups and as a mentor to other product managers. She’s currently Chief Product Officer at BBC Maestro, and has spent 13 years in the tech industry working with startups in the SaaS and mobile space. 
Randy Silver is a Leadership & Product Coach and Consultant. He gets teams unstuck, helping you to supercharge your results. Randy's held interim CPO and Leadership roles at scale-ups and SMEs, advised start-ups, and been Head of Product at HSBC and Sainsbury’s. He participated in Silicon Valley Product Group’s Coaching the Coaches forum, and speaks frequently at conferences and events. You can join one of communities he runs for CPOs (CPO Circles), Product Managers (Product In the {A}ether) and Product Coaches. He’s the author of What Do We Do Now? A Product Manager’s Guide to Strategy in the Time of COVID-19. A recovering music journalist and editor, Randy also launched Amazon’s music stores in the US & UK.

Tiama Hanson-Drury is CPTO at Opus 2, the legal technology company behind the systems that run some of the world's largest and most high-stakes litigation. She started her career in sales, moved into product in 2010 and has spent the last 16 years split roughly evenly between individual contributor and leadership roles. Her verdict on AI in product organisations is neither evangelism nor scepticism: the tooling has removed software as a bottleneck, but the markers of quality that mattered before AI have not changed, and teams that forget them end up shipping slop and burning out. In this episode, we discuss:
Why a sales background is better preparation for product than an engineering one, and how to lead engineers without pretending to be an expert
How the promise of doing more with AI is producing overload and burnout, and what leaders owe their teams in response
Why high AI adoption scores mean nothing if pull request sizes are ballooning and someone else has to review the output
The three things Tiama looks for in her teams now: outcome focus over craft, curiosity, and a culture of sharing failure as readily as success
Why the right move is not to hand three of your seven steps to AI, but to start again from the outcome you want
How Opus 2 went from shipping every six to 12 weeks to shipping daily, and why that made documentation and planning more important, not less
The case for betting on humans in the loop when your customers explicitly prefer it
Why Opus 2 is still hiring juniors, and how to build commercial judgment when the groundwork can be skipped
The one situation where Tiama tells people to let AI write the first draft rather than the second
We're refreshing The Product Experience and want your input. Take our two-minute survey and help shape where the show goes next! 
Chapters
(01:31) From sales to product
(03:24) Why the move is less obvious than it sounds
(04:34) Leading engineering without an engineering background
(06:28) How outsider questions sharpen engineering teams
(07:36) What AI has changed for a CPTO
(08:16) Overload, burnout and the pressure to token max
(10:32) Adoption scores versus quality signals
(11:39) What AI adoption actually looks like inside law firms
(12:42) Outcomes over output
(14:07) How Opus 2 built its AI strategy
(16:07) Removing friction from an AI rollout
(16:53) Partnering with AI instead of splitting up the steps
(18:08) A message from Mike Belsito
(19:40) Three things AI asks of a team
(21:47) More agency, and why planning matters more
(23:51) Scaling a team on conflicting evidence
(26:09) Betting on humans in the loop
(27:36) Why Opus 2 keeps hiring juniors
(29:44) Building judgment when the work moves faster
(32:36) Sharing your own AI operating system
(33:22) AI slop, CVs and the hiring algorithm
(36:17) Top tips for a product career in the age of AI
(38:36) Wrap-up

Referenced
Opus 2: https://www.opus2.com
Opus 2 announces addition of Tiama Hanson-Drury as chief product and technology officer: https://www.opus2.com/news/opus-2-announces-cpto-tiama-hanson-drury/
Minna Technologies, where Tiama was previously chief product officer: https://minnatechnologies.com
DX, the developer intelligence platform used to track squad performance: https://getdx.com
DORA metrics: https://dora.dev
Vote for Mike Belsito's South by Southwest 2027 session: https://tinyurl.com/belsitosxsw
Where to find Tiama Hanson-Drury
LinkedIn: https://www.linkedin.com/in/tiamahansondrury
We're refreshing The Product Experience and want your input. Take our two-minute survey and help shape where the show goes next! 
Our Hosts
Lily Smith enjoys working as a consultant product manager with early-stage and growing startups and as a mentor to other product managers. She’s currently Chief Product Officer at BBC Maestro, and has spent 13 years in the tech industry working with startups in the SaaS and mobile space. 
Randy Silver is a Leadership & Product Coach and Consultant. He gets teams unstuck, helping you to supercharge your results. Randy's held interim CPO and Leadership roles at scale-ups and SMEs, advised start-ups, and been Head of Product at HSBC and Sainsbury’s. He participated in Silicon Valley Product Group’s Coaching the Coaches forum, and speaks frequently at conferences and events. You can join one of communities he runs for CPOs (CPO Circles), Product Managers (Product In the {A}ether) and Product Coaches. He’s the author of What Do We Do Now? A Product Manager’s Guide to Strategy in the Time of COVID-19. A recovering music journalist and editor, Randy also launched Amazon’s music stores in the US & UK.