Python Bytes

• #435 Stop with .folders in my ~/

  Topics covered in this episode: platformdirs poethepoet - “Poe the Poet is a batteries included task runner that works well with poetry or with uv.” Python Pandas Ditches NumPy for Speedier PyArrow pointblank: Data validation made beautiful and powerful Extras Joke Watch on YouTube About the show Sponsored by us! Support our work through: Our courses at Talk Python Training The Complete pytest Course Patreon Supporters Connect with the hosts Michael: @[email protected] / @mkennedy.codes (bsky) Brian: @[email protected] / @brianokken.bsky.social Show: @[email protected] / @pythonbytes.fm (bsky) Join us on YouTube at pythonbytes.fm/live to be part of the audience. Usually Monday at 10am PT. Older video versions available there too. Finally, if you want an artisanal, hand-crafted digest of every week of the show notes in email form? Add your name and email to our friends of the show list, we'll never share it. Michael #1: platformdirs A small Python module for determining appropriate platform-specific dirs, e.g. a "user data dir". Why the community moved on from appdirs to platformdirs At AppDirs: Note: This project has been officially deprecated. You may want to check out pypi.org/project/platformdirs/ which is a more active fork of appdirs. Thanks to everyone who has used appdirs. Shout out to ActiveState for the time they gave their employees to work on this over the years. Better than AppDirs: Works today, works tomorrow – new Python releases sometimes change low-level APIs (win32com, pathlib, Apple sandbox rules). platformdirs tracks those changes so your code keeps running. First-class typing – no more types-appdirs stubs; editors autocomplete paths as Path objects. Richer directory set – if you need a user’s Downloads folder or a per-session runtime dir, there’s a helper for it. Cleaner internals – rewritten to use pathlib, caching, and extensive test coverage; all platforms are exercised in CI. Community stewardship – the project lives in the PyPA orbit and gets security/compatibility patches quickly. Brian #2: poethepoet - “Poe the Poet is a batteries included task runner that works well with poetry or with uv.” from Bob Belderbos Tasks are easy to define and are defined in pyproject.toml Michael #3: Python Pandas Ditches NumPy for Speedier PyArrow Pandas 3.0 will significantly boost performance by replacing NumPy with PyArrow as its default engine, enabling faster loading and reading of columnar data. Recently talked with Reuven Lerner about this on Talk Python too. In the next version, v3.0, PyArrow will be a required dependency, with pyarrow.string being the default type inferred for string data. PyArrow is 10 times faster. PyArrow offers columnar storage, which eliminates all that computational back and forth that comes with NumPy. PyArrow paves the way for running Pandas, by default, on Copy on Write mode, which improves memory and performance usage. Brian #4: pointblank: Data validation made beautiful and powerful “With its … chainable API, you can … validate your data against comprehensive quality checks …” Extras Brian: Ruff rules Ruff users, what rules are using and what are you ignoring? Python 3.14.0b2 - did we already cover this? Transferring your Mastodon account to another server, in case anyone was thinking about doing that I’m trying out Fathom Analytics for privacy friendly analytics Michael: Polars for Power Users: Transform Your Data Analysis Game Course Joke: Does your dog bite?

  --------  

  25:34
• #434 Most of OpenAI’s tech stack runs on Python

  Topics covered in this episode: Making PyPI’s test suite 81% faster People aren’t talking enough about how most of OpenAI’s tech stack runs on Python PyCon Talks on YouTube Optimizing Python Import Performance Extras Joke Watch on YouTube About the show Sponsored by Digital Ocean: pythonbytes.fm/digitalocean-gen-ai Use code DO4BYTES and get $200 in free credit Connect with the hosts Michael: @[email protected] / @mkennedy.codes (bsky) Brian: @[email protected] / @brianokken.bsky.social Show: @[email protected] / @pythonbytes.fm (bsky) Join us on YouTube at pythonbytes.fm/live to be part of the audience. Usually Monday at 10am PT. Older video versions available there too. Finally, if you want an artisanal, hand-crafted digest of every week of the show notes in email form? Add your name and email to our friends of the show list, we'll never share it. Brian #1: Making PyPI’s test suite 81% faster Alexis Challande The PyPI backend is a project called Warehouse It’s tested with pytest, and it’s a large project, thousands of tests. Steps for speedup Parallelizing test execution with pytest-xdist 67% time reduction --numprocesses=auto allows for using all cores DB isolation - cool example of how to config postgress to give each test worker it’s on db They used pytest-sugar to help with visualization, as xdist defaults to quite terse output Use Python 3.12’s sys.monitoring to speed up coverage instrumentation 53% time reduction Nice example of using COVERAGE_CORE=sysmon Optimize test discovery Always use testpaths Sped up collection time. 66% reduction (collection was 10% of time) Not a huge savings, but it’s 1 line of config Eliminate unnecessary imports Use python -X importtime Examine dependencies not used in testing. Their example: ddtrace A tool they use in production, but it also has a couple pytest plugins included Those plugins caused ddtrace to get imported Using -p:no ddtrace turns off the plugin bits Notes from Brian: I often get questions about if pytest is useful for large projects. Short answer: Yes! Longer answer: But you’ll probably want to speed it up I need to extend this article with a general purpose “speeding up pytest” post or series. -p:no can also be used to turn off any plugin, even builtin ones. Examples include nice to have developer focused pytest plugins that may not be necessary in CI CI reporting plugins that aren’t needed by devs running tests locally Michael #2: People aren’t talking enough about how most of OpenAI’s tech stack runs on Python Original article: Building, launching, and scaling ChatGPT Images Tech stack: The technology choices behind the product are surprisingly simple; dare I say, pragmatic! Python: most of the product’s code is written in this language. FastAPI: the Python framework used for building APIs quickly, using standard Python type hints. As the name suggests, FastAPI’s strength is that it takes less effort to create functional, production-ready APIs to be consumed by other services. C: for parts of the code that need to be highly optimized, the team uses the lower-level C programming language Temporal: used for asynchronous workflows and operations inside OpenAI. Temporal is a neat workflow solution that makes multi-step workflows reliable even when individual steps crash, without much effort by developers. It’s particularly useful for longer-running workflows like image generation at scale Michael #3: PyCon Talks on YouTube Some talks that jumped out to me: Keynote by Cory Doctorow 503 days working full-time on FOSS: lessons learned Going From Notebooks to Scalable Systems And my Talk Python conversation around it. (edited episode pending) Unlearning SQL The Most Bizarre Software Bugs in History The PyArrow revolution in Pandas And my Talk Python episode about it. What they don't tell you about building a JIT compiler for CPython And my Talk Python conversation around it (edited episode pending) Design Pressure: The Invisible Hand That Shapes Your Code Marimo: A Notebook that "Compiles" Python for Reproducibility and Reusability And my Talk Python episode about it. GPU Programming in Pure Python And my Talk Python conversation around it (edited episode pending) Scaling the Mountain: A Framework for Tackling Large-Scale Tech Debt Brian #4: Optimizing Python Import Performance Mostly pay attention to #'s 1-3 This is related to speeding up a test suite, speeding up necessary imports. Finding what’s slow Use python -X importtime <the reset of the command Ex: python -X importtime ptyest Techniques Lazy imports move slow-to-import imports into functions/methods Avoiding circular imports hopefully you’re doing that already Optimize __init__.py files Avoid unnecessary imports, heavy computations, complex logic Notes from Brian Some questions remain open for me Does module aliasing really help much? This applies to testing in a big way Test collection imports your test suite, so anything imported at the top level of a file gets imported at test collection time, even if you only are running a subset of tests using filtering like -x or -m or other filter methods. Run -X importtime on test collection. Move slow imports into fixtures, so they get imported when needed, but NOT at collection. See also: option -X in the standard docs Consider using import_profile Extras Brian: PEPs & Co. PEP is a ‘backronym”, an acronym where the words it stands for are filled in after the acronym is chosen. Barry Warsaw made this one up. There are a lot of “enhancement proposal” and “improvement proposal” acronyms now from other communities pythontest.com has a new theme More colorful. Neat search feature Now it’s excruciatingly obvious that I haven’t blogged regularly in a while I gotta get on that Code highlighting might need tweaked for dark mode Michael: git-bug Pyrefly follow up Joke: There is hope.

  --------  

  29:01
• #433 Dev in the Arena

  Topics covered in this episode: git-flight-rules Uravelling t-strings neohtop Introducing Pyrefly: A new type checker and IDE experience for Python Extras Joke Watch on YouTube About the show Sponsored by us! Support our work through: Our courses at Talk Python Training The Complete pytest Course Patreon Supporters Connect with the hosts Michael: @[email protected] / @mkennedy.codes (bsky) Brian: @[email protected] / @brianokken.bsky.social Show: @[email protected] / @pythonbytes.fm (bsky) Join us on YouTube at pythonbytes.fm/live to be part of the audience. Usually Monday at 10am PT. Older video versions available there too. Finally, if you want an artisanal, hand-crafted digest of every week of the show notes in email form? Add your name and email to our friends of the show list, we'll never share it. Michael #1: git-flight-rules What are "flight rules"? A guide for astronauts (now, programmers using Git) about what to do when things go wrong. Flight Rules are the hard-earned body of knowledge recorded in manuals that list, step-by-step, what to do if X occurs, and why. Essentially, they are extremely detailed, scenario-specific standard operating procedures. [...] NASA has been capturing our missteps, disasters and solutions since the early 1960s, when Mercury-era ground teams first started gathering "lessons learned" into a compendium that now lists thousands of problematic situations, from engine failure to busted hatch handles to computer glitches, and their solutions. Steps for common operations and actions I want to start a local repository What did I just commit? I want to discard specific unstaged changes Restore a deleted file Brian #2: Uravelling t-strings Brett Cannon Article walks through Evaluating the Python expression Applying specified conversions Applying format specs Using an Interpolation class to hold details of replacement fields Using Template class to hold parsed data Plus, you don’t have to have Python 3.14.0b1 to try this out. The end result is very close to an example used in PEP 750, which you do need 3.14.0b1 to try out. See also: I’ve written a pytest version, Unravelling t-strings with pytest, if you want to run all the examples with one file. Michael #3: neohtop Blazing-fast system monitoring for your desktop Features Real-time process monitoring CPU and Memory usage tracking Beautiful, modern UI with dark/light themes Advanced process search and filtering Pin important processes Process management (kill processes) Sort by any column Auto-refresh system stats Brian #4: Introducing Pyrefly: A new type checker and IDE experience for Python From Facebook / Meta Another Python type checker written in Rust Built with IDE integration in mind from the beginning Principles Performance IDE first Inference (inferring types in untyped code) Open source I mistakenly tried this on the project I support with the most horrible abuses of the dynamic nature of Python, pytest-check. It didn’t go well. But perhaps the project is ready for some refactoring. I’d like to try it soon on a more well behaved project. Extras Brian: Python: The Documentary Official Trailer Tim Hopper added Setting up testing with ptyest and uv to his “Python Developer Tooling Handbook” For a more thorough intro on pytest, check out courses.pythontest.com pocket is closing, I’m switching to Raindrop I got one question about code formatting. It’s not highlighted, but otherwise not bad. Michael: New course! Polars for Power Users: Transform Your Data Analysis Game Apache Airflow 3.0 Released Paste 5 Joke: Theodore Roosevelt’s Man in the Arena, but for programming

  --------  

  28:40
• #432 How To Fix Your Computer

  Topics covered in this episode: pre-commit: install with uv PEP 773: A Python Installation Manager for Windows (Accepted) Changes for Textual The Best Programmers I Know Extras Joke Watch on YouTube About the show Sponsored by NordLayer: pythonbytes.fm/nordlayer Connect with the hosts Michael: @[email protected] / @mkennedy.codes (bsky) Brian: @[email protected] / @brianokken.bsky.social Show: @[email protected] / @pythonbytes.fm (bsky) Join us on YouTube at pythonbytes.fm/live to be part of the audience. Usually Monday at 10am PT. Older video versions available there too. Finally, if you want an artisanal, hand-crafted digest of every week of the show notes in email form? Add your name and email to our friends of the show list, we'll never share it. Brian #1: pre-commit: install with uv Adam Johnson uv tool works great at keeping tools you use on lots of projects up to date quickly, why not use it for pre-commit. The extension of pre-commit-uv will use uv to create virtual environments and install packages fore pre-commit. This speeds up initial pre-commit cache creation. However, Adam is recommending this flavor of using pre-commit because it’s just plain easier to install pre-commit and dependencies than the official pre-commit install guide. Win-win. Side note: No Adam, I’m not going to pronounce uv “uhv”, I’ll stick with “you vee”, even Astral tells me I’m wrong Michael #2: PEP 773: A Python Installation Manager for Windows (Accepted) via pycoders newsletter One manager to rule them all – PyManager. PEP 773 replaces all existing Windows installers (.exe “traditional” bundle, per-version Windows Store apps, and the separate py.exe launcher) with a single MSIX app called Python Install Manager (nick-named PyManager). PyManager should be mainstream by CPython 3.15, and the traditional installer disappears no earlier than 3.16 (≈ mid-2027). Simple, predictable commands. python → launches “the best” runtime already present or auto-installs the latest CPython if none is found. py → same launcher as today plus management sub-commands: py install, py uninstall, py list, py exec, py help. Optional python3 and python3.x aliases can be enabled by adding one extra PATH entry. Michael #3: Changes for Textual Bittersweet news: the business experiment ends, but the code lives on. Textual began as a hobby project layered on top of Rich, but it has grown into a mature, “makes-the-terminal-do-the-impossible” TUI framework with an active community and standout documentation. Despite Textual’s technical success, the team couldn’t pinpoint a single pain-point big enough to sustain a business model, so the company will wind down in the coming weeks. The projects themselves aren’t going anywhere: they’re stable, battle-tested, and will continue under the stewardship of the original author and the broader community. Brian #4: The Best Programmers I Know Matthias Endler “I have met a lot of developers in my life. Lately, I asked myself: “What does it take to be one of the best? What do they all have in common?”” The list Read the reference Know your tools really well Read the error message Break down problems Don’t be afraid to get your hands dirty Always help others Write Never stop learning Status doesn’t matter Build a reputation Have patience Never blame the computer Don’t be afraid to say “I don’t know” Don’t guess Keep it simple Each topic has a short discussion. So don’t just ready the bullet points, check out the article. Extras Brian: I had a great time in Munich last week. I a talk at a company event, met with tons of people, and had a great time. The best part was connecting with people from different divisions working on similar problems. I love the idea of internal conferences to get people to self organize by topic and meet people they wouldn’t otherwise, to share ideas. Also got started working on a second book on the plane trip back. Michael: Talk Python Clips (e.g. mullet) Embrace your cloud firewall (example). Python 3.14.0 beta 1 is here Congrats to the new PSF Fellows. Cancelled faster CPython bsky.app/profile/snarky.ca post Joke: How To Fix Your Computer

  --------  

  25:48
• #431 Nerd Gas

  Topics covered in this episode: pirel: Python release cycle in your terminal FastAPI Cloud Python's new t-strings Extras Joke Watch on YouTube About the show Sponsored by NordLayer: pythonbytes.fm/nordlayer Connect with the hosts Michael: @[email protected] / @mkennedy.codes (bsky) Brian: @[email protected] / @brianokken.bsky.social Show: @[email protected] / @pythonbytes.fm (bsky) Join us on YouTube at pythonbytes.fm/live to be part of the audience. Usually Monday at 10am PT. Older video versions available there too. Finally, if you want an artisanal, hand-crafted digest of every week of the show notes in email form? Add your name and email to our friends of the show list, we'll never share it. Michael #1: pirel: Python release cycle in your terminal pirel check shows release information about your active Python interpreter. If the active version is end-of-life, the program exits with code 1. If no active Python interpreter is found, the program exits with code 2. pirel list lists all Python releases in a table. Your active Python interpreter is highlighted. A picture is worth many words Brian #2: FastAPI Cloud Sebastián Ramírez, creator of FastAPI, announced today the formation of a new Company, FastAPI Cloud. Here’s the announcement blog post: FastAPI Cloud - By The Same Team Behind FastAPI There’s a wait list to try it out. Promises to turns deployment into fastapi login; fastapi deploy Side note: announcement includes quote from Daft Punk: Build Harder, Better, Faster, Stronger I just included this in a talk I’m gave last week (and will again next week), where I modify this to “Build Easier, Better, Faster, Stronger” Sebastian and I are both fans of the rocket emoji. BTW, we first covered FastAPI on episode 123 in 2019 Brian #3: Python's new t-strings Dave Peck, one of the authors of PEP 750, which will go into Python 3.14 We covered t-strings in ep 428 In article t-strings security benefits over f-strings How to work with t-strings A Pig Latin example Also, I think I have always done this wrong Is it the first consonant to the end? or the first consonant cluster? So… Brian → Rianbay? or Ianbray? BTW, this is an example of nerdgassing What’s next once t-strings ship? On thing that’s next (in Python 3.15, maybe, is using t-strings in shlex and subprocess) PEP 787 – Safer subprocess usage using t-strings deferred to 3.15 Michael #4: zev A simple CLI tool to help you remember terminal commands. Examples: # Find running processes zev 'show all running python processes' # File operations zev 'find all .py files modified in the last 24 hours' # System information zev 'show disk usage for current directory' # Network commands zev 'check if google.com is reachable' # Git operations zev 'show uncommitted changes in git' Again, picture worth many words: Extras Brian: Holy Grail turns 50 nerdgassing Michael: Transcripts are a bit better now. Zen is better now Joke: Can my friend come in?

  --------  

  29:27

More Technology podcasts

Trending Technology podcasts

About Python Bytes