Discussing Python 3.12 new features contributed by Meta like custom JITs, type system improvements, and faster comprehensions. Exploring the challenges of upstreaming changes, including Cinder's impact on machine learning. Insights on Python performance enhancements, upcoming features, and managing API changes within tech companies.
Meta engineers contributed performance improvements to Python 3.12, including Lazy Imports, Cinder, and type system enhancements.
Podcast discusses the process of upstreaming features to Python, covering community engagement, new syntax additions, and performance optimizations.
Deep dives
Python Development at Meta: From Usage to Development
The podcast delves into the transition from discussing Python usage at Meta in a previous episode to focusing on how Python is developed internally. Mentioning the use of a performance-oriented Python version called Cinder, which is open source. Highlighting cool features in Python 3.12 including hooks for customizations, performance improvements, and the type system. It also covers the process of upstreaming features to the C Python project, providing insights into maintaining custom features and leveraging new Python functionalities.
Python 3.12 Exciting Features
The episode discusses the headline features of Python 3.12, such as additions to typing functionalities like new syntax for type parameters and type aliases. Notable implementation of f-strings into the language's grammar, allowing nested f-strings for advanced string interpolation. Introducing a per-interpreter guild to alleviate the global interpreter lock constraint, improving monitoring, debugging support, and immortal objects for optimizing object handling. Additionally, the inclusion of pep 709 on inlining comprehensions for significant speed enhancements.
Contributions and Collaboration at Meta
The summary covers various feature contributions by meta engineers to Python 3.12, including eager async tasks and optimization improvements. It mentions collaborative efforts with external teams, like Microsoft, aimed at enhancing Python's performance. Exploring the process of upstreaming features through PEP submissions, with examples of successful PEP approvals like comprehension inlining, immortal objects, and new typing features. It also touches on community discussions, the importance of collaboration, and the decision-making process for feature integration.
Challenges and Strategies in Project Cinder
The podcast highlights the origins and purpose of the Cinder project, initiated in response to the exponential growth in server capacity usage at Instagram. Addressing the limitations of Python through the creation of a customized Python interpreter that optimized performance. Explaining the challenges faced, such as integrating a JIT and maintaining compatibility with C extensions. Detailing the successful strategy of extracting and upstreaming specific Cinder features, like dictionary watchers, hooks, and APIs to improve Python performance while maintaining stability and compatibility.
For the second time in just a few months, we are talking Python on the Meta Tech Podcast. Python 3.12 features a whole range of new features, many of which were contributed by Meta. Carl and Itamar join Pascal to talk about their contributions to the latest release, including new hooks that allow for custom JITs like Cinder, Immortal Objects, improvements to the type system, faster comprehensions and much more. In their discussion, they talk not just about how and why those features were built but also the process of upstreaming and engaging with the community.