The podcast discusses the recent changes in the Python for VS Code interface for testing, with a focus on pytest. The speakers talk about the background and improvements in testing support, the integration of Python testing in VS Code, the rollout process for new features, bug fixes and user feedback, and how to stay updated on Python testing in VS Code. They emphasize the importance of relying on issue reports and user feedback to ensure the extension is working as intended.
The recent testing interface changes in VS Code, specifically the rewrite using a PyTest plugin, have greatly improved test discovery, error handling, and test output presentation.
A controlled rollout process was implemented to de-risk the testing rewrite, involving gradual release to users, feedback collection, and bug triaging, resulting in refined code, architecture, and a smoother transition.
Deep dives
Background and Role of the Python Extension in VS Code
The podcast episode introduces Courtney and Eleanor, product managers and software engineers at the Python extension in VS Code. They discuss the recent changes in the testing interface of VS Code and how it relates to PyTest and UnitTest. Courtney explains that the Python extension acts as the middle layer between VS Code and the testing frameworks, providing translation and presenting the data to users in an easily readable format. She highlights the existing features of the extension, including running test discovery, debugging tests, and viewing output in the testing panel.
The Testing Rewrite and Benefits
Eleanor talks about the testing rewrite and the improvements it brings. Previously, the testing support relied on parsing output, which was brittle and limited in functionality. With the rewrite, they moved to using a PyTest plugin, which allowed them to connect to the PyTest run flow and receive error-tolerant test discovery. The rewrite also enables dynamic run, where test results are displayed as they finish, instead of waiting for the entire test run to complete. Additionally, the improved testing interface provides better handling of broken code and displays the test output, enhancing the overall user experience.
Controlled Rollout and User Feedback
To de-risk the changes from the testing rewrite, a controlled rollout was implemented. This involved gradually releasing the changes to a percentage of users and actively collecting feedback and bug reports. Users on the pre-release version of the Python extension were asked to provide raw feedback and bug reports, enabling the team to identify and address issues. The controlled rollout process helped them refine the testing code and architecture, as well as validate that the changes met user expectations. Bugs and issues reported by users were triaged and fixed, ensuring a smoother transition for all users.
Future Developments and Customization
In the future, the Python extension plans to introduce new features and enhancements to further improve the testing experience in VS Code. Some upcoming developments include adding test coverage support, enabling compatibility with Django tests, and providing more customization options for test configurations. Users will be able to have multiple test configurations, specify environment variables, and configure different run options for better flexibility. The extension team encourages users to engage through GitHub issues, where they can track progress, provide feedback, and suggest ideas for further improvements.
If you haven't tried running automated tests, especially with pytest, in VS Code recently, you should take another look. The Python for VS Code interface for testing, especially for pytest, has changed recently. On this episode we discuss the change with the software engineer working on the pytest changes, Eleanor Boyd, and the product manager, Courtney Webster.