Going Beyond requirements.txt With pylock.toml and PEP 751
May 16, 2025
auto_awesome
Brett Cannon, a Python Core Developer and packaging expert, joins to discuss his journey with PEP 751 and the new pylock.toml format. He highlights the importance of evolving beyond requirements.txt for better dependency management. Cannon shares insights into the influence of lock files on project reproducibility and the collaborative challenges faced by the Python community in standardizing practices. He also reflects on the historical shift toward security-focused packaging, showcasing the growth in Python's ecosystem.
PEP 751 introduces the pylock.toml file format to standardize dependency recording, improving reproducibility beyond the traditional requirements.txt.
The initiative emphasizes security in dependency management by implementing mechanisms that ensure exact package versions are installed.
Collaboration within the Python community has been crucial in developing and accepting PEP 751, fostering a more coherent approach to packaging standards.
Deep dives
The Evolution of Dependency Management
The podcast discusses the introduction of PEP 751, which aims to standardize the process for recording Python dependencies through a new file format called PyLock.toml. This new format moves beyond the outdated requirements.txt and aims to provide an immutable record of dependencies necessary for project reproducibility. The speaker emphasizes that this initiative followed years of effort and discussions within the Python community to address the shortcomings of the previous system. With PEP 751, it is intended that developers will have a more reliable method for installing dependencies consistent across different environments.
Challenges in Python Packaging Standards
The speaker shares insights about the various challenges faced in creating packaging standards within the Python ecosystem, particularly regarding the lack of standardized requirements files. Historically, tools like pip have defined their own specifications, leading to inconsistencies and confusion across different tools within the Python community. This lack of uniformity has complicated dependency management, making it difficult for projects to effectively share and reproduce code installations. The speaker highlights that through PEP 751, the goal is to unify these efforts and promote a more coherent strategy for managing dependencies.
Significance of Secure Dependency Management
The conversation highlights the importance of security in dependency management, asserting that a lock file not only helps in replicating project environments but also strengthens security practices. By introducing mechanisms such as hashes, the new format aims to ensure that developers install the exact versions of packages that they intend. The speaker expresses concern over existing tools that do not prioritize security checks, which can leave applications vulnerable to risks from unchecked dependencies. Thus, the implementation of PEP 751 promotes more secure practices by default, aiming to increase resilience against potential threats.
The Context of Python's Packaging Community
The podcast presents a detailed overview of the current landscape of Python's packaging community, emphasizing collaboration and discussions among core developers. The speaker mentions that feedback from various projects and stakeholders contributed significantly to the drafting and acceptance of PEP 751. The evolution of packaging practices is presented as a collective effort to address long-standing issues, with many developers advocating for improved standards. The successful acceptance of PEP 751 is highlighted as a pivotal moment for the community, fostering a renewed sense of cooperation and progress.
Looking Ahead: Python's Packaging Future
As the conversation wraps up, the speaker reflects on the future of Python packaging, hinting at ongoing discussions concerning the creation of a packaging council within the community. This initiative aims to lessen the burden on individual developers and create an organized structure for guiding the direction of packaging standards moving forward. The need for a collaborative approach is underscored, as the complexity of packaging practices continues to evolve. Additionally, the possibilities around integrating new projects and tools into the established environment are regarded as essential for supporting the Python ecosystem as it grows.
Personal Insights and Future Proofing with Python
In a personal note, the speaker shares his experiences of becoming a parent, acknowledging the time constraints this life change brings to exploring new programming languages and frameworks. He mentions his intention to focus on making Python more accessible for beginners, ensuring that newcomers can engage with strengths while reducing barriers. Additionally, he discusses interest in languages like Gleam and their functional programming concepts as intriguing challenges. The aim is clear: to simplify entry points into Python for future generations and prevent new learners from facing unnecessary hurdles.
What is the best way to record the Python dependencies for the reproducibility of your projects? What advantages will lock files provide for those projects? This week on the show, we welcome back Python Core Developer Brett Cannon to discuss his journey to bring PEP 751 and the pylock.toml file format to the community.
Brett has been working on a way to move beyond the requirements.txt file for over six years. He was on the show previously to discuss his work on PEP 665, which was rejected. He decided to continue to push forward, authoring PEP 751 last year, which was accepted at the end of March this year.
The PEP calls for a new file format to record your project’s dependencies. The goal was to have a standardized immutable record for what should be installed to reproduce your project in a virtual environment. He discusses working with other packaging projects and the compromises involved in creating a standard.
In this video course, you’ll learn how to use Python’s subprocess module to run and control external programs from your scripts. You’ll start with launching basic processes and progress to interacting with them as they execute.
Topics:
00:00:00 – Introduction
00:02:38 – Brett’s roles within the Python community
00:05:41 – How to move beyond requirement.txt?
00:10:58 – What does the community use as project artifacts?
00:15:28 – Building on the success of pyproject.toml
00:17:44 – Introducing PEP 665
00:19:49 – Software Bills of Materials and security
00:25:20 – Back to lock files and security
00:31:08 – Video Course Spotlight
00:32:27 – Not giving up on the idea
00:34:01 – Leading into PEP 751
00:38:54 – Working toward a single multi-platform file
00:43:02 – The final push
00:48:54 – Leaving room for flexibility
00:53:50 – And it’s done, PEP 751 accepted unconditionally
00:58:06 – Keynote speaker at EuroPython 2025
00:58:45 – What are uv workspaces?
01:01:02 – Considering the use of lock files in data science
01:05:23 – Updates about Python for WASI and Emscripten
01:13:51 – Clarification on WASI
01:20:28 – Future conversation about Python launcher
01:23:04 – What are you excited about in the world of Python?
01:24:25 – What do you want to learn next?
01:28:41 – What’s the best way to follow your work online?
