Pandas 3.0 Highlights: str dtype and Copy-on-Write

• PEP 798: Unpacking in Comprehensions
• Pandas 3.0.0rc0
• typos
• A couple testing topics
• Extras
• Joke

About the show

Sponsored by us! Support our work through:

• Our courses at Talk Python Training
• The Complete pytest Course
• Patreon Supporters

Connect with the hosts

• Michael: @mkennedy@fosstodon.org / @mkennedy.codes (bsky)
• Brian: @brianokken@fosstodon.org / @brianokken.bsky.social
• Show: @pythonbytes@fosstodon.org / @pythonbytes.fm (bsky)

Join us on YouTube at pythonbytes.fm/live to be part of the audience. Usually Monday at 10am PT. Older video versions available there too.

Finally, if you want an artisanal, hand-crafted digest of every week of the show notes in email form? Add your name and email to our friends of the show list, we'll never share it.

Michael #1: PEP 798: Unpacking in Comprehensions

• After careful deliberation, the Python Steering Council is pleased to accept PEP 798 – Unpacking in Comprehensions.

• Examples

  [*it for it in its]  # list with the concatenation of iterables in 'its'
  {*it for it in its}  # set with the union of iterables in 'its'
  {**d for d in dicts} # dict with the combination of dicts in 'dicts'
  (*it for it in its)  # generator of the concatenation of iterables in 'its'
  

• Also: The Steering Council is happy to unanimously accept “PEP 810, Explicit lazy imports”

Brian #2: Pandas 3.0.0rc0

• Pandas 3.0.0 will be released soon, and we’re on Release candidate 0
• Here’s What’s new in Pands 3.0.0
  • Dedicated string data type by default
    • Inferred by default for string data (instead of object dtype)
    • The str dtype can only hold strings (or missing values), in contrast to object dtype. (setitem with non string fails)
    • The missing value sentinel is always NaN (np.nan) and follows the same missing value semantics as the other default dtypes.
  • Copy-on-Write
    • The result of any indexing operation (subsetting a DataFrame or Series in any way, i.e. including accessing a DataFrame column as a Series) or any method returning a new DataFrame or Series, always behaves as if it were a copy in terms of user API.
    • As a consequence, if you want to modify an object (DataFrame or Series), the only way to do this is to directly modify that object itself.
  • pd.col syntax can now be used in DataFrame.assign() and DataFrame.loc()
    • You can now do this: df.assign(c = pd.col('a') + pd.col('b'))
  • New Deprecation Policy
  • Plus more
-

Michael #3: typos

• You’ve heard about codespell … what about typos?
• VSCode extension and OpenVSX extension.
• From Sky Kasko:

Like codespell, typos checks for known misspellings instead of only allowing words from a dictionary. But typos has some extra features I really appreciate, like finding spelling mistakes inside snake_case or camelCase words. For example, if you have the line:

*connecton_string = "sqlite:///my.db"*

codespell won't find the misspelling, but typos will. It gave me the output:

*error: `connecton` should be `connection`, `connector`  
╭▸ ./main.py:1:1  │1 │ connecton_string = "sqlite:///my.db"  
╰╴━━━━━━━━━*

But the main advantage for me is that typos has an LSP that supports editor integrations like a VS Code extension. As far as I can tell, codespell doesn't support editor integration. (Note that the popular Code Spell Checker VS Code extension is an unrelated project that uses a traditional dictionary approach.)

For more on the differences between codespell and typos, here's a comparison table I found in the typos repo: https://github.com/crate-ci/typos/blob/master/docs/comparison.md

By the way, though it's not mentioned in the installation instructions, typos is published on PyPI and can be installed with uv tool install typos, for example. That said, I don't bother installing it, I just use the VS Code extension and run it as a pre-commit hook. (By the way, I'm using prek instead of pre-commit now; thanks for the tip on episode #448!) It looks like typos also publishes a GitHub action, though I haven't used it.

Brian #4: A couple testing topics

• slowlify
  • suggested by Brian Skinn
  • Simulate slow, overloaded, or resource-constrained machines to reproduce CI failures and hunt flaky tests.
  • Requires Linux with cgroups v2
• Why your mock breaks later
  • Ned Badthelder
  • Ned’s taught us before to “Mock where the object is used, not where it’s defined.”
  • To be more explicit, but probably more confusing to mock-newbies, “don’t mock things that get imported, mock the object in the file it got imported to.”
    • See? That’s probably worse. Anyway, read Ned’s post.
  • If my project myproduct has user.py that uses the system builtin open() and we want to patch it:
    • DONT DO THIS: @patch("builtins.open")
      • This patches open() for the whole system
    • DO THIS: @patch("myproduct.user.open")
      • This patches open() for just the user.py file, which is what we want
  • Apparently this issue is common and is mucking up using coverage.py

Extras

Brian:

• The Rise and Rise of FastAPI - mini documentary
• “Building on Lean” chapter of LeanTDD is out
  • The next chapter I’m working on is “Finding Waste in TDD”
  • Notes to delete before end of show:
    • I’m not on track for an end of year completion of the first pass, so pushing goal to 1/31/26
    • As requested by a reader, I’m releasing both the full-so-far versions and most-recent-chapter

Michael:

• My Vanishing Gradient’s episode is out
• Django 6 is out

Joke: tabloid - A minimal programming language inspired by clickbait headlines