Engineering Enablement by DX

In this episode of Engineering Enablement, Brian Houck talks with Andrew Swerdlow, VP of Engineering at Roblox, about how the company is building trust in AI agents to review, test, and deploy code autonomously. Andrew explains how Roblox turns past code review feedback into rules for agents and why engineering is shifting toward specifications and intent. They explore what that means for the skills engineers need, how Roblox measures AI’s impact on productivity, and how it manages costs. They also discuss the guardrails needed to let agents safely ship code with minimal human intervention.

Where to find Andrew Swerdlow:
• LinkedIn: https://www.linkedin.com/in/andrewswerdlow/ 
• X:  https://x.com/andrewswerdlow 
• Website: https://andrewswerdlow.net

Where to find Brian Houck: 
• LinkedIn: ⁠⁠⁠⁠https://www.linkedin.com/in/brianhouck⁠⁠⁠⁠

In this episode, we cover:
(00:00) Intro
(02:20) Building trust in AI across the software development lifecycle
(04:44) When human oversight helps and when it gets in the way
(06:50) What AI agents need to know about your organization
(08:45) How Roblox turns engineering knowledge into rules for AI agents 
(12:40) How AI code review compares to human code review
(14:59) Capturing engineering knowledge beyond documentation
(18:14) How AI changes what engineers need to understand
(20:17) The shift to spec-driven development
(23:24) How AI changes how we evaluate engineers
(26:33) Rethinking engineering interviews in the age of AI
(28:23) How Roblox tracks AI-generated code
(35:56) Model routing and choosing the right AI tools
(39:50) Measuring AI's impact on engineering productivity
(44:44) Optimizing AI costs at scale
(48:12) Builders at Roblox and the future of software engineering
(50:19) Building trust in autonomous code review
(53:11) The cultural shift toward autonomous engineering

Referenced:
• CAFE(S): Your Agent Is Only as Good as Its Context
• Roblox
• Tech Leadership by Andrew Swerdlow
• Margaret-Anne Storey
• Malte Ubl - Vercel | LinkedIn
• Grafana Labs
• Kushan Mehta | LinkedIn 
• Diff Risk Score: AI-driven risk-aware software development - Engineering at Meta

As AI agents become more capable, context quality is increasingly the limiting factor in their performance. In this episode of Engineering Enablement, DX researcher Brian Houck by his CAFE(S) is joined by co-authors Max Kanat-Alexander (Capital One), Eirini Kalliamvakou (DX), and Margaret-Anne Storey (University of Victoria) to discuss CAFE(S), a new framework for improving context quality.
The panelists introduce the five dimensions of CAFE(S): Clarity, Actionability, Fidelity, Efficiency, and Security. They walk through common context failures, how poor context accrues technical and cognitive debt, what it costs in developer productivity and operating spend, how AI is changing the skills engineers need, and why context review could become as important as code review.

Where to find Margaret-Anne Storey:
• LinkedIn: https://www.linkedin.com/in/margaret-anne-storey 
• X:  https://x.com/margaretstorey 
• Website: https://margaretstorey.com
Where to find Max Kanat-Alexander: 
• LinkedIn: https://www.linkedin.com/in/mkanat
• X: https://x.com/mkanat
• Website: https://www.codesimplicity.com
Where to find Eirini Kalliamvakou: 
• LinkedIn: https://www.linkedin.com/in/eirini-kalliamvakou-1016865 
• X: https://x.com/irina_kAl
Where to find Brian Houck: 
• LinkedIn: ⁠⁠⁠⁠https://www.linkedin.com/in/brianhouck⁠⁠⁠⁠

In this episode, we cover:
(00:00) Intro
(01:18) Meet the CAFE(S) authors
(03:13) Introducing CAFE(S): Why context quality matters
(06:32) Common context failure modes
(08:30) The five dimensions of high-quality context
(12:33) When model failures are actually context failures
(18:24) Why the panel prioritizes clarity in CAFE(S)
(22:34) How context quality relates to technical, cognitive, and intent debt
(25:35) How human interventions reveal context failures
(28:11) How context quality relates to AI fluency
(30:28) How AI is changing what makes a good engineer
(33:48) The case for context review as a discipline 
(38:44) Wrap-up

Referenced:
• CAFE(S): Your Agent Is Only as Good as Its Context
• Nicole Forsgren
• The Productivity-Experience Paradox - Annie Vella
• Startup Studio
• From Technical Debt to Cognitive and Intent Debt: Rethinking Software Health in the Age of AI
• The AI-native developer
• CodeScene

Meta researcher Moritz Beller joins Brian Houck to discuss how Meta measures developer productivity. Moritz explains diff authoring time and how Meta uses it to evaluate tools and guide engineering decisions. They explore how AI is changing engineers’ work and revealing gaps in traditional metrics. They also revisit Moritz’s “Mind the Gap” study and discuss the promise and risks of AI agents for testing.

Where to find Moritz Beller:
• LinkedIn: https://www.linkedin.com/in/inventitech 
• X: https://x.com/Inventitech 
• GitHub: https://github.com/Inventitech 
• Website: https://inventitech.com

Where to find Brian Houck: 
• LinkedIn: ⁠⁠⁠https://www.linkedin.com/in/brianhouck⁠⁠⁠

In this episode, we cover:
(00:00) Intro
(02:10) Moritz’s role at Meta
(04:01) Measuring diff authoring time
(08:20) Measuring A/B experiments
(12:11) What diff authoring time reveals
(14:45) Planning and leadership reporting
(18:36) Why Meta measures teams, not individuals
(22:33) Where developer time goes with AI
(26:06) AI’s impact on junior and senior developers
(27:18) Capturing invisible work with AI
(29:32) The “Mind the Gap” study
(33:40) Revisiting “Mind the Gap” in 2026
(38:32) AI agents for software testing
(41:49) What remains hard to measure

Referenced:
• State of AI Impact in Engineering Q2 Report 2026
• GitHub Copilot and Developer Productivity: An Observational Dose-Response Analysis
• From Technical Debt to Cognitive and Intent Debt 
• Mind the Gap: On the Relationship Between Automatically Measured and Self-Reported Productivity
• The Impact of AI Coding Assistants on Software Engineering: A Longitudinal Study
• When, how, and why developers (do not) test in their IDEs | Proceedings of the 2015 10th Joint Meeting on Foundations of Software Engineering
• DORA, SPACE, and DevEx: Which framework should you use
• A Tale of Two Cities: Software Developers Working from Home During the COVID-19 Pandemic

In this episode of Engineering Enablement, Robert Lucero, Chief Architect at Okta, joins host Brian Houck to discuss how agentic AI is changing identity and access management. They explore how organizations should define agent identities, why authentication and authorization remain foundational, and how sandboxing, fine-grained permissions, and just-in-time access can enable agents to operate autonomously without creating unacceptable risk.
Robert shares what Okta has learned from driving AI adoption among security-minded engineers, what makes a repository ready for AI agents, and why strong testing, CI, and review processes matter even more as AI generates more code. They also discuss AI’s role in software validation, the need for human judgment, and whether AI will ultimately give the advantage to security teams or attackers.

Where to find Robert Lucero:
• LinkedIn: https://www.linkedin.com/in/rlucero

Where to find Brian Houck: 
• LinkedIn: ⁠⁠https://www.linkedin.com/in/brianhouck⁠⁠

In this episode, we cover:
(00:00) Intro
(02:16) Meet Robert Lucero
(02:56) Why identity becomes critical as AI agents access more systems
(06:07) Governing automated access for non-deterministic agents
(08:01) Defining and managing agent identities
(10:32) Authentication and authorization for AI agents
(13:33) Determining how much autonomy to give AI agents
(19:44) Identity systems as the control plane for AI agents
(22:40) Driving AI adoption among security-minded engineers
(25:48) Why AI’s impact on engineering extends beyond coding
(28:44) The three components of AI readiness for repositories
(30:55) AI’s role in software testing and validation
(34:26) Whether AI gives defenders or attackers the advantage
(36:01) Where to find the best food in the country

Referenced:
• State of AI Impact in Engineering Q2 Report 2026
• Okta
• The Hugging Face incident and the road ahead | OpenAI
• GitHub Copilot · Your AI pair programmer
• Claude Code by Anthropic | AI Coding Agent, Terminal, IDE
• Eight Myths on Software Engineering and GenAI | Queue
• Adriana Corona

Max Kanat-Alexander is an Executive Distinguished Engineer at Capital One, where he helps lead developer experience and AI-assisted software development.
In this episode of Engineering Enablement, Max joins Brian Houck to discuss how AI is changing software development and why strong engineering fundamentals matter more than ever. They explore how engineering skills are evolving, why AI can amplify weaknesses in the development lifecycle, and how teams should rethink code review, quality, and testing.
Max also shares how leaders can assess whether their organizations are ready for more advanced AI workflows and why developing the next generation of senior engineers remains one of the biggest open questions in software engineering.

Where to find Max Kanat-Alexander:
• LinkedIn: https://www.linkedin.com/in/mkanat 
• X: https://x.com/mkanat 
• Blog: https://www.codesimplicity.com 

Where to find Brian Houck: 
• LinkedIn: ⁠https://www.linkedin.com/in/brianhouck⁠

In this episode, we cover:
(00:00) Intro
(01:54) Max’s role at Capital One
(02:52) Where to invest in engineering organizations  
(06:36) The new entry-level engineering skills to pay attention to 
(10:39) Why deepening your understanding still matters 
(12:29) The bottlenecks around code review 
(19:40) Why human code reviews still have value 
(25:20) Why not all PRs need human review
(26:11) The vicious cycle of AI-driven development
(30:13) Using LLMs for refactoring 
(33:58) AI readiness and why fixing engineering systems is so hard
(38:42) Why research is needed on creating good senior engineers
(41:00) Why AI will increase the need for engineers

Referenced:
• DX Core 4 Productivity Framework
• Capital One 
• Code Review Guidelines at Google 
• Code Simplicity by Max Kanat-Alexander
• How Microsoft sees engineering bottlenecks changing with AI