Options for Packaging Your Python Application: Wheels, Docker, and More
The Real Python Podcast
Reducing Memory Usage in Python Applications
The chapter covers techniques for reducing memory usage and processing large amounts of data. They discuss three techniques: compression, batching, and indexing, and explain how to apply them in Pandas and NumPy. The speakers also emphasize the importance of understanding the underlying infrastructure and hardware.
Have you wondered, how should I package my Python code? You’ve written the application, but now you need to distribute it to the machines it’s intended to run on. It depends on what the code is, the libraries it depends on, and with whom do you want to share it. This week on the show we have Itamar Turner-Trauring, creator of the website pythonspeed.com. We discuss his article “Options for Packaging Your Python Code: Wheels, Conda, Docker, and More,” covering the how of sharing your code.
Itamar also briefly discusses his Python memory profiler named Fil. We talk about his recent PyCon 2020 presentation, “Small Big Data: What to do When Your Data Doesn’t Fit in Memory.” We also cover several of the resources available on his website for data scientists that want to get deeper into Docker.
Course Spotlight: Python Coding Interviews: Tips & Best Practices
In this step-by-step course, you’ll learn how to take your Python coding interview skills to the next level and use Python’s built-in functions and modules to solve problems faster and more easily.
Topics:
- 00:00:00 – Introduction
- 00:01:36 – About the naming of pythonspeed.com
- 00:03:47 – Fil - Python Memory Profiler
- 00:06:44 – Small Big Data: What to do when your data doesn’t fit in memory - PyCon 2020
- 00:12:17 – Options for packaging your Python code: Wheels, Conda, Docker, and more
- 00:15:13 – Python Wheels
- 00:19:22 – pipx: Install and Run Python Applications in Isolated Environments
- 00:20:52 – PEX, and friends
- 00:24:51 – System Package, RPM or DEB
- 00:29:42 – Conda Packaging and conda-forge
- 00:36:09 – Video Course Spotlight
- 00:37:23 – Self-contained executable: PyInstaller, PyOxidizer, Briefcase
- 00:43:45 – Container image (Docker, Singularity)
- 00:54:55 – Why alpine may not be the best choice
- 01:05:28 – Singularity
- 01:07:50 – What are you excited about in the world of Python?
- 01:10:40 – What do you want to learn next?
- 01:13:54 – Thanks and Goodbye
Show Links:
- Python => Speed: Ship Better Python Software, Faster
- Code Without Rules: Helping You Become a Productive Programmer and Get Work/Life Balance
- Talk Python to Me – Episode #274: Profiling Data Science Code with FIL
- Fil: A New Python Memory Profiler for Data Scientists and Scientists
- Small Big Data: What to do When Your Data Doesn’t Fit in Memory - PyCon 2020
- Episode 16: Thinking in Pandas: Python Data Analysis the Right Way
- Options for Packaging Your Python Code: Wheels, Conda, Docker, and More
- What Are Python Wheels and Why Should You Care?: Real Python article
- pipx — Install and Run Python Applications in Isolated Environments
- pex: A Library and Tool for Generating .pex (Python EXecutable) Files
- WTF is PEX?: Twitter Lightning Talk - YouTube
- RPM (Red Hat Package Manager): Wikipedia article
- DEB (Debian Package - file format): Wikipedia article
- Conda: Package, Dependency and Environment Management for Any Language
- conda-forge: A community-led collection of recipes, build infrastructure and distributions for the conda package manager
- PyInstaller: Freezes (packages) Python applications into stand-alone executables
- PyOxidizer: A utility for producing binaries that embed Python
- Briefcase: Convert a Python project into a standalone native application
- Docker: Get Started with Docker
- Just Enough Docker Packaging: Book
- Episode 8: Docker + Python for Data Science and Machine Learning
- Using Alpine can make Python Docker builds 50× slower
- Introduction to Singularity
- Docker vs. Singularity for data processing: UIDs and filesystem access
- Best practices for production-ready Docker packaging: EuroPython 2020 talk
- EuroPython 2020 Talk: Brian Track Stream - Unedited - Starts @08:33:45
- Statistical Rethinking: A Bayesian Course with Examples
- Bayes Theorem: 3Blue1Brown - YouTube
Level up your Python skills with our expert-led courses:
The AI-powered Podcast Player
