The Pragmatic Engineer

• Developer productivity with Dr. Nicole Forsgren (creator of DORA, co-creator of SPACE)

  Supported by Our Partners• WorkOS — The modern identity platform for B2B SaaS• CodeRabbit — Cut code review time and bugs in half• Augment Code — AI coding assistant that pro engineering teams love—How do you architect a live streaming system to deal with more load than it’s ever been done before? Today, we hear from an architect of such a system: Ashutosh Agrawal, formerly Chief Architect of JioCinema (and currently Staff Software Engineer at Google DeepMind.)We take a deep dive into video streaming architecture, tackling the complexities of live streaming at scale (at tens of millions of parallel streams) and the challenges engineers face in delivering seamless experiences. We talk about the following topics: • How large-scale live streaming architectures are designed• Tradeoffs in optimizing performance• Early warning signs of streaming failures and how to detect them• Why capacity planning for streaming is SO difficult• The technical hurdles of streaming in APAC regions• Why Ashutosh hates APMs (Application Performance Management systems)• Ashutosh’s advice for those looking to improve their systems design expertise• And much more!—Timestamps(00:00) Intro(01:28) The world record-breaking live stream and how support works with live events(05:57) An overview of streaming architecture(21:48) The differences between internet streaming and traditional television.l(22:26) How adaptive bitrate streaming works(25:30) How throttling works on the mobile tower side (27:46) Leading indicators of streaming problems and the data visualization needed(31:03) How metrics are set (33:38) Best practices for capacity planning (35:50) Which resources are planned for in capacity planning (37:10) How streaming services plan for future live events with vendors(41:01) APAC specific challenges(44:48) Horizontal scaling vs. vertical scaling (46:10) Why auto-scaling doesn’t work(47:30) Concurrency: the golden metric to scale against(48:17) User journeys that cause problems (49:59) Recommendations for learning more about video streaming (51:11) How Ashutosh learned on the job(55:21) Advice for engineers who would like to get better at systems(1:00:10) Rapid fire round—The Pragmatic Engineer deepdives relevant for this episode:• Software architect archetypes https://newsletter.pragmaticengineer.com/p/software-architect-archetypes • Engineering leadership skill set overlaps https://newsletter.pragmaticengineer.com/p/engineering-leadership-skillset-overlaps • Software architecture with Grady Booch https://newsletter.pragmaticengineer.com/p/software-architecture-with-grady-booch—See the transcript and other references from the episode at ⁠⁠https://newsletter.pragmaticengineer.com/podcast⁠⁠—Production and marketing by ⁠⁠⁠⁠⁠⁠⁠⁠https://penname.co/⁠⁠⁠⁠⁠⁠⁠⁠. For inquiries about sponsoring the podcast, email [email protected]. Get full access to The Pragmatic Engineer at newsletter.pragmaticengineer.com/subscribe

  --------  

  1:22:39
• Live streaming at world-record scale with Ashutosh Agrawal

  Supported by Our Partners• WorkOS — The modern identity platform for B2B SaaS• CodeRabbit — Cut code review time and bugs in half• Augment Code — AI coding assistant that pro engineering teams love—How do you architect a live streaming system to deal with more load than it’s ever been done before? Today, we hear from an architect of such a system: Ashutosh Agrawal, formerly Chief Architect of JioCinema (and currently Staff Software Engineer at Google DeepMind.)We take a deep dive into video streaming architecture, tackling the complexities of live streaming at scale (at tens of millions of parallel streams) and the challenges engineers face in delivering seamless experiences. We talk about the following topics: • How large-scale live streaming architectures are designed• Tradeoffs in optimizing performance• Early warning signs of streaming failures and how to detect them• Why capacity planning for streaming is SO difficult• The technical hurdles of streaming in APAC regions• Why Ashutosh hates APMs (Application Performance Management systems)• Ashutosh’s advice for those looking to improve their systems design expertise• And much more!—Timestamps(00:00) Intro(01:28) The world record-breaking live stream and how support works with live events(05:57) An overview of streaming architecture(21:48) The differences between internet streaming and traditional television.l(22:26) How adaptive bitrate streaming works(25:30) How throttling works on the mobile tower side (27:46) Leading indicators of streaming problems and the data visualization needed(31:03) How metrics are set (33:38) Best practices for capacity planning (35:50) Which resources are planned for in capacity planning (37:10) How streaming services plan for future live events with vendors(41:01) APAC specific challenges(44:48) Horizontal scaling vs. vertical scaling (46:10) Why auto-scaling doesn’t work(47:30) Concurrency: the golden metric to scale against(48:17) User journeys that cause problems (49:59) Recommendations for learning more about video streaming (51:11) How Ashutosh learned on the job(55:21) Advice for engineers who would like to get better at systems(1:00:10) Rapid fire round—The Pragmatic Engineer deepdives relevant for this episode:• Software architect archetypes https://newsletter.pragmaticengineer.com/p/software-architect-archetypes • Engineering leadership skill set overlaps https://newsletter.pragmaticengineer.com/p/engineering-leadership-skillset-overlaps • Software architecture with Grady Booch https://newsletter.pragmaticengineer.com/p/software-architecture-with-grady-booch—See the transcript and other references from the episode at ⁠⁠https://newsletter.pragmaticengineer.com/podcast⁠⁠—Production and marketing by ⁠⁠⁠⁠⁠⁠⁠⁠https://penname.co/⁠⁠⁠⁠⁠⁠⁠⁠. For inquiries about sponsoring the podcast, email [email protected]. Get full access to The Pragmatic Engineer at newsletter.pragmaticengineer.com/subscribe

  --------  

  1:02:12
• AI Engineering with Chip Huyen

  Supported by Our Partners• Swarmia — The engineering intelligence platform for modern software organizations.• Graphite — The AI developer productivity platform. • Vanta — Automate compliance and simplify security with Vanta.—On today’s episode of The Pragmatic Engineer, I’m joined by Chip Huyen, a computer scientist, author of the freshly published O’Reilly book AI Engineering, and an expert in applied machine learning. Chip has worked as a researcher at Netflix, was a core developer at NVIDIA (building NeMo, NVIDIA’s GenAI framework), and co-founded Claypot AI. She also taught Machine Learning at Stanford University.In this conversation, we dive into the evolving field of AI Engineering and explore key insights from Chip’s book, including:• How AI Engineering differs from Machine Learning Engineering • Why fine-tuning is usually not a tactic you’ll want (or need) to use• The spectrum of solutions to customer support problems – some not even involving AI!• The challenges of LLM evals (evaluations)• Why project-based learning is valuable—but even better when paired with structured learning• Exciting potential use cases for AI in education and entertainment• And more!—Timestamps(00:00) Intro (01:31) A quick overview of AI Engineering(05:00) How Chip ensured her book stays current amidst the rapid advancements in AI(09:50) A definition of AI Engineering and how it differs from Machine Learning Engineering (16:30) Simple first steps in building AI applications(22:53) An explanation of BM25 (retrieval system) (23:43) The problems associated with fine-tuning (27:55) Simple customer support solutions for rolling out AI thoughtfully (33:44) Chip’s thoughts on staying focused on the problem (35:19) The challenge in evaluating AI systems(38:18) Use cases in evaluating AI (41:24) The importance of prioritizing users’ needs and experience (46:24) Common mistakes made with Gen AI(52:12) A case for systematic problem solving (53:13) Project-based learning vs. structured learning(58:32) Why AI is not the end of engineering(1:03:11) How AI is helping education and the future use cases we might see(1:07:13) Rapid fire round—The Pragmatic Engineer deepdives relevant for this episode:• Applied AI Software Engineering: RAG https://newsletter.pragmaticengineer.com/p/rag • How do AI software engineering agents work? https://newsletter.pragmaticengineer.com/p/ai-coding-agents • AI Tooling for Software Engineers in 2024: Reality Check https://newsletter.pragmaticengineer.com/p/ai-tooling-2024 • IDEs with GenAI features that Software Engineers love https://newsletter.pragmaticengineer.com/p/ide-that-software-engineers-love—See the transcript and other references from the episode at ⁠⁠https://newsletter.pragmaticengineer.com/podcast⁠⁠—Production and marketing by ⁠⁠⁠⁠⁠⁠⁠⁠https://penname.co/⁠⁠⁠⁠⁠⁠⁠⁠. For inquiries about sponsoring the podcast, email [email protected]. Get full access to The Pragmatic Engineer at newsletter.pragmaticengineer.com/subscribe

  --------  

  1:14:44
• Building a best-selling game with a tiny team – with Jonas Tyroller

  Supported by Our Partners• Formation — Level up your career and compensation with Formation. • WorkOS — The modern identity platform for B2B SaaS• Vanta — Automate compliance and simplify security with Vanta.—In today’s episode of The Pragmatic Engineer, I’m joined by Jonas Tyroller, one of the developers behind Thronefall, a minimalist indie strategy game that blends tower defense and kingdom-building, now available on Steam.Jonas takes us through the journey of creating Thronefall from start to finish, offering insights into the world of indie game development. We explore:• Why indie developers often skip traditional testing and how they find bugs• The developer workflow using Unity, C# and Blender• The two types of prototypes game developers build • Why Jonas spent months building game prototypes in 1-2 days• How Jonas uses ChatGPT to build games• Jonas’s tips on making games that sell• And more!—Timestamps(00:00) Intro(02:07) Building in Unity(04:05) What the shader tool is used for (08:44) How a Unity build is structured(11:01) How game developers write and debug code (16:21) Jonas’s Unity workflow(18:13) Importing assets from Blender(21:06) The size of Thronefall and how it can be so small(24:04) Jonas’s thoughts on code review(26:42) Why practices like code review and source control might not be relevant for all contexts(30:40) How Jonas and Paul ensure the game is fun (32:25) How Jonas and Paul used beta testing feedback to improve their game(35:14) The mini-games in Thronefall and why they are so difficult(38:14) The struggle to find the right level of difficulty for the game(41:43) Porting to Nintendo Switch(45:11) The prototypes Jonas and Paul made to get to Thronefall(46:59) The challenge of finding something you want to build that will sell(47:20) Jonas’s ideation process and how they figure out what to build (49:35) How Thronefall evolved from a mini-game prototype(51:50) How long you spend on prototyping (52:30) A lesson in failing fast(53:50) The gameplay prototype vs. the art prototype(55:53) How Jonas and Paul distribute work (57:35) Next steps after having the play prototype and art prototype(59:36) How a launch on Steam works (1:01:18) Why pathfinding was the most challenging part of building Thronefall(1:08:40) Gen AI tools for building indie games (1:09:50) How Jonas uses ChatGPT for editing code and as a translator (1:13:25) The pros and cons of being an indie developer (1:15:32) Jonas’s advice for software engineers looking to get into indie game development(1:19:32) What to look for in a game design school(1:22:46) How luck figures into success and Jonas’s tips for building a game that sells(1:26:32) Rapid fire round—The Pragmatic Engineer deepdives relevant for this episode:• Game development basics https://newsletter.pragmaticengineer.com/p/game-development-basics • Building a simple game using Unity https://newsletter.pragmaticengineer.com/p/building-a-simple-game—See the transcript and other references from the episode at ⁠⁠https://newsletter.pragmaticengineer.com/podcast⁠⁠—Production and marketing by ⁠⁠⁠⁠⁠⁠⁠⁠https://penname.co/⁠⁠⁠⁠⁠⁠⁠⁠. For inquiries about sponsoring the podcast, email [email protected]. Get full access to The Pragmatic Engineer at newsletter.pragmaticengineer.com/subscribe

  --------  

  1:29:32
• Observability: the present and future, with Charity Majors

  Supported by Our Partners• Sonar —  Trust your developers – verify your AI-generated code.• Vanta —Automate compliance and simplify security with Vanta.—In today's episode of The Pragmatic Engineer, I'm joined by Charity Majors, a well-known observability expert – as well as someone with strong and grounded opinions. Charity is the co-author of "Observability Engineering" and brings extensive experience as an operations and database engineer and an engineering manager. She is the cofounder and CTO of observability scaleup Honeycomb.Our conversation explores the ever-changing world of observability, covering these topics:• What is observability? Charity’s take• What is “Observability 2.0?”• Why Charity is a fan of platform teams• Why DevOps is an overloaded term: and probably no longer relevant• What is cardinality? And why does it impact the cost of observability so much?• How OpenTelemetry solves for vendor lock-in • Why Honeycomb wrote its own database• Why having good observability should be a prerequisite to adding AI code or using AI agents• And more!—Timestamps(00:00) Intro (04:20) Charity’s inspiration for writing Observability Engineering(08:20) An overview of Scuba at Facebook(09:16) A software engineer’s definition of observability (13:15) Observability basics(15:10) The three pillars model(17:09) Observability 2.0 and the shift to unified storage(22:50) Who owns observability and the advantage of platform teams (25:05) Why DevOps is becoming unnecessary (27:01) The difficulty of observability (29:01) Why observability is so expensive (30:49) An explanation of cardinality and its impact on cost(34:26) How to manage cost with tools that use structured data (38:35) The common worry of vendor lock-in(40:01) An explanation of OpenTelemetry(43:45) What developers get wrong about observability (45:40) A case for using SLOs and how they help you avoid micromanagement (48:25) Why Honeycomb had to write their database (51:56) Companies who have thrived despite ignoring conventional wisdom(53:35) Observability and AI (59:20) Vendors vs. open source(1:00:45) What metrics are good for (1:02:31) RUM (Real User Monitoring) (1:03:40) The challenges of mobile observability (1:05:51) When to implement observability at your startup (1:07:49) Rapid fire round—The Pragmatic Engineer deepdives relevant for this episode:• How Uber Built its Observability Platform https://newsletter.pragmaticengineer.com/p/how-uber-built-its-observability-platform • Building an Observability Startup https://newsletter.pragmaticengineer.com/p/chronosphere • How to debug large distributed systems https://newsletter.pragmaticengineer.com/p/antithesis • Shipping to production https://newsletter.pragmaticengineer.com/p/shipping-to-production —See the transcript and other references from the episode at ⁠⁠https://newsletter.pragmaticengineer.com/podcast⁠⁠—Production and marketing by ⁠⁠⁠⁠⁠⁠⁠⁠https://penname.co/⁠⁠⁠⁠⁠⁠⁠⁠. For inquiries about sponsoring the podcast, email [email protected]. Get full access to The Pragmatic Engineer at newsletter.pragmaticengineer.com/subscribe

  --------  

  1:14:25

More Technology podcasts

Trending Technology podcasts

About The Pragmatic Engineer