The Real Python Podcast

What goes into the decision of how to host your Python code or application in the cloud? Which technology stack is the right size for your project? This week on the show, we have Calvin Hendryx-Parker. Calvin talks about cloud hosting options, infrastructure choices, and deployment tools. Calvin is the co-founder and CTO of Six Feet Up, and co-organizer of the Python Web Conference. We talk about finding the right tools for clients. He also discusses the Python platform they created for hosting a virtual conference. We also discuss hosting personal portfolio projects. That conversation leads to the question, what types of skills you can showcase through creating a hosted project. Course Spotlight: Creating PyQt Layouts for GUI Applications In this step-by-step course, you’ll learn how to use PyQt layouts to arrange and manage the graphical components on your GUI applications. With the help of PyQt’s layout managers, you’ll be able to create polished and professional GUIs with minimal effort. Topics: 00:00:00 – Introduction 00:01:46 – What considerations to start with for deployment? 00:04:02 – What is Saltstack? 00:06:00 – The changing cloud hosting landscape 00:10:12 – Containers, Docker, and other standards 00:11:28 – How do you learn about this technology? 00:15:58 – Concerns of setting up development vs production environments 00:17:41 – Security concerns 00:19:20 – Sponsor: Scout APM 00:20:26 – Deploying a Python portfolio project 00:23:12 – Deploying for a small business project or API 00:29:11 – Cloud formation, Terraform, additional tools 00:30:22 – Deploying a large project 00:35:12 – Frontend frameworks for large web projects 00:39:30 – Video Course Spotlight 00:40:43 – What does your consultancy do? 00:41:37 – What things do you look for in an employee? 00:50:42 – Python Web Conference 2021 00:57:43 – What are you excited about in the world of Python? 00:59:09 – What do you want to learn next? 01:00:49 – What is something you thought you knew about Python, but were wrong about it? 01:02:23 – Thanks and goodbye Show Links: Six Feet Up Python Web Conference 2021 LoudSwarm: Virtual Conference Hosting Platform Salt Project: Open Source Automation Engine The 12 Factor App Heroku Ansible: Agentless IT Automation AWS Free Tier AWS Lambda: Run code without thinking about servers Terraform: open-source infrastructure as code software tool AWS Fargate: Serverless compute for containers Docker: Get Started Snyk: Developer-first Cloud Native Application Security Dependabot: Automated Dependency Updates Continuous Integration With Python: An Introduction - Real Python Article Chris Anderson via Twitter: This is the open source flight code that the NASA Mars drone is running Plone: The Ultimate Enterprise CMS Code With Me: Ultimate collaborative development by JetBrains AWS DeepLens Fluent Python: Luciano Ramalho Level up your Python skills with our expert-led courses: Continuous Integration With Python Using Google Login With Flask Creating PyQt Layouts for GUI Applications Support the podcast & join our community of Pythonistas

Have you wanted to get your Python code to consume data from web-based APIs? Maybe you’ve dabbled with the requests package, but you don’t know what steps to take next. This week on the show, David Amos is back, and he’s brought another batch of PyCoder’s Weekly articles and projects. We discuss an article titled, “Python’s APIs: A Winning Combo for Reading Public Data”. David shares another Real Python article about creating microservices using Google Remote Procedure Calls (gRPC). We also cover several other articles and projects from the Python community including, making a difficult data analysis question easy with pandas, efficiently cleaning text with pandas, the tricky bits of Python concurrency, building rich terminal dashboards, making better assertions for Python tests, and building and managing real-life data science projects with metaflow. Course Spotlight: Making HTTP Requests With Python The “requests” library is the de facto standard for making HTTP requests in Python. It abstracts the complexities of making requests behind a beautiful, simple API so that you can focus on interacting with services and consuming data in your application. This course shows you how to work effectively with “requests”, from start to finish. Topics: 00:00:00 – Introduction 00:01:46 – Python Microservices With gRPC 00:07:49 – Python’s APIs: A Winning Combo for Reading Public Data 00:15:07 – Making a Difficult Data Analysis Question Easy With Pandas 00:21:07 – Efficiently Cleaning Text With Pandas 00:34:20 – Video Course Spotlight 00:35:27 – Python Concurrency: The Tricky Bits 00:41:49 – Building Rich Terminal Dashboards 00:45:08 – python-precisely: Better Assertions for Python Tests 00:48:45 – metaflow: Build and Manage Real-Life Data Science Projects With Ease 00:52:35 – Thanks and goodbye Show Links: Python Microservices With gRPC – Learn how to build a robust and developer-friendly Python microservices infrastructure using gRPC and Kubernetes. You’ll also explore advanced topics such as interceptors and integration testing. Python’s APIs: A Winning Combo for Reading Public Data – Learn what APIs are and how to consume them using Python. You’ll also learn some core concepts for working with APIs, such as status codes, HTTP methods, using the requests library, and much more. Making a Difficult Data Analysis Question Easy With Pandas – A great strategy to use when faced with a tricky data analysis problem is to reshape the dataset into a format that turns it into an easy problem. In this article, you’ll look at an example involving a simple calculation and extensive reshaping in pandas. Efficiently Cleaning Text With Pandas – In this article, you’ll see some examples of cleaning text fields in a large data file and learn several strategies for efficiently cleaning unstructured text fields using Python and pandas. Python Concurrency: The Tricky Bits – An exploration of threads, processes, and coroutines in Python, with interesting examples that illuminate the differences between each. Building Rich Terminal Dashboards – Learn how to use the Rich CLI library’s new terminal dashboard feature. Projects: python-precisely: Better Assertions for Python Tests metaflow: Build and Manage Real-Life Data Science Projects With Ease Additional Links: API design: Understanding gRPC, OpenAPI and REST and when to use them Data Cleaning IS Analysis, Not Grunt Work How to Lie with Statistics: Wikipedia Article Level up your Python skills with our expert-led courses: Building HTTP APIs With Django REST Framework Web Scraping With Beautiful Soup and Python Making HTTP Requests With Python Support the podcast & join our community of Pythonistas

What’s the difference between writing code for yourself and developing for others? What new considerations do you need to take into account as a professional Python developer? This week on the show, we talk to Dane Hillard about his book “Practices of the Python Pro”. Dane discusses his philosophy on the design principles that go into writing code. We talk about namespaces, object-oriented design, and how to keep your code extensible. We also consider the how and when of code optimization. Course Spotlight: Dictionaries and Arrays: Selecting the Ideal Data Structure In this course, you’ll learn about two of Python’s data structures: dictionaries and arrays. You’ll look at multiple types and classes for both of these and learn which implementations are best for your specific use cases. Topics: 00:00:00 – Introduction 00:01:29 – Release and response to Practices of the Python Pro 00:03:12 – What was the writing process like? 00:06:09 – What makes someone a professional? 00:12:30 – How have you and the tools changed in Python testing? 00:14:10 – When did you start to see the change in your career? 00:15:42 – Sponsor: PyCharm 00:16:27 – What topic were you excited to share in the book? 00:17:49 – The importance of code design and ergonomics 00:20:52 – Why is managing and designing namespaces important? 00:26:32 – Expanding that design thought process to object-oriented programming 00:30:02 – Differences of functional vs object-oriented programming 00:34:40 – Video Course Spotlight 00:36:04 – What do you mean by extensible? 00:42:59 – How and when to optimize code? 00:45:57 – Sharing developer philosophy 00:46:52 – What are you excited about in the world of Python? 00:48:31 – What do you want to learn next? 00:51:03 – Thanks and goodbye Show Links: Practices of the Python Pro Dane’s Website Effective Python Testing With Pytest: Real Python Article pytest: helps you write better programs An Effective Python Environment: Making Yourself at Home - Real Python Article FastAPI framework, high performance, easy to learn, fast to code, ready for production Django: The web framework for perfectionists with deadlines Django: GitHub Graphene-Django: Provides Abstractions to Add GraphQL Functionality to Your Django Project Level up your Python skills with our expert-led courses: Getting Started With Django: Building a Portfolio App Test-Driven Development With pytest Dictionaries and Arrays: Selecting the Ideal Data Structure Support the podcast & join our community of Pythonistas

Do you know the initial steps to get your Python script hosted on the web? You may have built something with Flask, but how would you stand it up so that you can share it with others? This week on the show, we have the previous guest Martin Breuss back on the show. Martin shares his recent article titled, “Python Web Applications: Deploy Your Script as a Flask App”. David Amos also returns, and he’s brought another batch of PyCoder’s Weekly articles and projects. David shares a recent mathematical Real Python article about the stochastic gradient descent algorithm with Python. Stochastic gradient descent is an optimization algorithm often used in machine learning applications to find ideal model parameters. We also cover several other articles and projects from the Python community including, property-based testing with hypothesis, Python’s tug of war between beginner-friendly features and support for advanced users, how Python integers work, the steering council accepts PEP 634, a magical full-stack framework for Django named django-unicorn, and a visual programming environment called Math Inspector. Course Spotlight: Simulating Real-World Processes in Python With SimPy In this step-by-step course, you’ll see how you can use the SimPy package to model real-world processes with a high potential for congestion. You’ll create an algorithm to approximate a complex system, and then you’ll design and run a simulation of that system in Python. Topics: 00:00:00 – Introduction 00:02:44 – Property-Based Testing With hypothesis, and Associated Use Cases 00:09:55 – Python’s Tug of War Between Beginner-Friendly Features and Support for Advanced Users 00:18:50 – Sponsor: Scout APM 00:19:54 – How Python Integers Work 00:26:53 – Python Steering Council Accepts PEP 634 00:32:48 – Stochastic Gradient Descent Algorithm With Python and NumPy 00:38:36 – Video Course Spotlight 00:39:39 – Martin Breuss - Followup about Stay at Home Mentorship Program 00:42:13 – Python Web Applications: Deploy Your Script as a Flask App 00:52:25 – django-unicorn: A Magical Full-Stack Framework for Django 00:55:15 – Math Inspector: A Visual Programming Environment for Scientific Computing With NumPy and SciPy 01:00:21 – Thanks and goodbye Show Links: Property-Based Testing With hypothesis, and Associated Use Cases – Testing software is hard. Property-based testing can help you create more effective tests. Learn how to do property-based testing with the hypothesis framework by looking at some real-world use cases. Python’s Tug of War Between Beginner-Friendly Features and Support for Advanced Users – Python has made some big improvements to tracebacks in recent versions. See how tracebacks have evolved over the last couple of major releases and where there’s still some work left to be done. Check out the discussion on Hacker News. How Python Integers Work – Python’s integer datatype is pretty different from most other languages because they allow arbitrary precision. Learn how integers work under the hood in this in-depth article. Python Steering Council Accepts PEP 634 – Pattern matching, which adds a kind of switch-case statement to Python, has been accepted. Stochastic Gradient Descent Algorithm With Python and NumPy – Learn what the stochastic gradient descent algorithm is, how it works, and how to implement it with Python and NumPy. Python Web Applications: Deploy Your Script as a Flask App – In this tutorial, you’ll learn how to go from a local Python script to a fully deployed Flask web application that you can share with the world. Projects: django-unicorn: A Magical Full-Stack Framework for Django Math Inspector: A Visual Programming Environment for Scientific Computing With NumPy and SciPy Additional Links: Episode 47: Unraveling Python’s Syntax to Its Core With Brett Cannon Friendly tracebacks - Simplified Python tracebacks translatable into any language. PythonBytes - Episode #220 Warnings About Dangerous Syntax: Cool New Features in Python 3.8 - Real Python Article PEP 636 – Structural Pattern Matching: Tutorial Django-Unicorn Articles python-utils: The online playground for Python utilities -Powered by Unicorn Level up your Python skills with our expert-led courses: Simulating Real-World Processes in Python With SimPy Using Google Login With Flask Cool New Features in Python 3.8 Support the podcast & join our community of Pythonistas

Python core developer Brett Cannon discusses unraveling Python's syntax to its core and his series of articles exploring the structure and workings of Python. He also talks about his role at Microsoft working on the Python extension for VS Code and recent Python enhancement proposals. The podcast covers topics such as syntactic sugar, Python in different environments, analyzing Python's syntax, and exploring perspectives and learning from different communities.

Are you interested in building interactive dashboards with Python? How about a project that takes a flat data file all the way to a web-hosted interactive dashboard? This week on the show, David Amos is back, and he’s brought another batch of PyCoder’s Weekly articles and projects. Along with the Real Python article about data visualizations using Dash, David covers an article designed to help Python developers understand the fundamentals of C. We discuss a couple of articles about Excel and using Python with Microsoft Office. We also cover several other articles and projects from the Python community including, out-of-memory crashes in Python, updating all packages with pip-review, data science notebooks for teams, and a command-line tool for looking up colors, shades, and palettes. Course Spotlight: Command Line Interfaces in Python Command line arguments are the key to converting your programs into useful and enticing tools that are ready to be used in the terminal of your operating system. In this course, you’ll learn their origins, standards, and basics, and how to implement them in your program. Topics: 00:00:00 – Introduction 00:01:36 – CPython Internals Book 00:03:18 – C for Python Programmers 00:06:11 – Dying, Fast and Slow: Out-Of-Memory Crashes in Python 00:13:29 – Automating Excel File Creation and Distribution With Pandas And Outlook 00:18:33 – Update All Packages With pip-review 00:23:49 – Video Course Spotlight 00:25:04 – Ditching Excel for Python: Lessons Learned From a Legacy Industry 00:30:20 – Develop Data Visualization Interfaces in Python With Dash 00:38:11 – Deepnote: Data Science Notebook for Teams 00:41:25 – colorpedia: Command-Line Tool for Looking Up Colors, Shades and Palettes 00:43:45 – Thanks and goodbye Show Links: C for Python Programmers – In this tutorial, you’ll learn the basics of the C language, which is used in the source code for CPython, the most popular Python implementation. Learning C is important for Python programmers interested in contributing to CPython. Dying, Fast and Slow: Out-Of-Memory Crashes in Python – Learn about the different ways that memory issues can manifest in your Python programs, and how you can debug and fix them. Automating Excel File Creation and Distribution With Pandas And Outlook – See how a little bit of Python can go a long way to automating manual processes and save businesses valuable time. Update All Packages With pip-review – Keeping track of Python dependencies and updates can be tricky. The pip-review tool automates a lot of this process in a convenient command-line interface. Ditching Excel for Python: Lessons Learned From a Legacy Industry – Learn how Python is revolutionizing an industry that’s notoriously resistant to change and fraught with every programmer’s most dreaded tool: Excel spreadsheets. Develop Data Visualization Interfaces in Python With Dash – Learn how to build a dashboard using Python and Dash. Dash is a framework for building data visualization interfaces. It helps data scientists build fully interactive web applications quickly. Projects: Deepnote: Data Science Notebook for Teams colorpedia: Command-Line Tool for Looking Up Colors, Shades and Palettes Additional Links: CPython Internals Book: Your Guide to the Python 3 Interpreter E27: Preparing for an Interview With Python Practice Problems - Guest Jim Anderson E24: Options for Packaging Your Python Application: Wheels, Docker, and More - Guest Itamar Turner-Trauring Fil: A New Python Memory Profiler for Data Scientists and Scientists pip-review: A Tool to Keep Track of Your Python Package Updates pip-tools: Keeps Your Pinned Dependencies Fresh E29: Resolving Package Dependencies With the New Version of Pip pip - The Python Package Installer pyxll: Write Excel Add-Ins in Python Introduction to Dash: Plotly Dash App Gallery Level up your Python skills with our expert-led courses: Using Jupyter Notebooks Command Line Interfaces in Python Editing Excel Spreadsheets in Python With openpyxl Support the podcast & join our community of Pythonistas

Are you interested in processing images in Python? Do you need to load and modify images for your Flask or Django website or CMS? Then you most likely will be working with Pillow, the friendly fork of PIL, the Python imaging library. This week on the show, we have Mike Driscoll, who is writing a new book about image processing in Python. We dive deep into the types of processing Pillow provides. Mike talks about creating Python GUI applications to take advantage of all the library has to offer. We also talk about his PyDev of the week series and his Python Interviews book. Course Spotlight: Editing Excel Spreadsheets in Python With openpyxl In this course, you’ll learn how to handle spreadsheets in Python using the openpyxl package. You’ll learn how to manipulate Excel spreadsheets, extract information from spreadsheets, create simple or more complex spreadsheets, including adding styles, charts, and so on. Topics: 00:00:00 – Introduction 00:01:40 – Update on Python 101 book 00:03:17 – Pillow: Image Processing With Python 00:04:06 – Kickstarter for the book 00:05:35 – What parts of the Pillow library will the book cover? 00:07:49 – What is ImageChops? 00:09:06 – How do you currently use Pillow? 00:11:06 – What is ImageOps? 00:13:15 – Sponsor Scout APM 00:14:18 – Building a GUI interface for Pillow features 00:16:46 – Other uses for Pillow in testing 00:18:01 – Use in web frameworks and file formats 00:20:17 – What is Pillow not good at? 00:22:13 – Batch processing 00:23:12 – Exif Data and GPS information from images 00:26:57 – Creating a watermark 00:28:58 – Video Course Spotlight 00:30:15 – Writing image process methods as modules 00:33:45 – Timeline for the book release 00:35:04 – Using Pillow in a Jupyter notebook 00:38:02 – Python Interviews Book and PyDev of the Week 00:41:57 – What are you excited about in the world of Python? 00:44:41 – What do you want to learn next? 00:46:25 – Thanks and goodbye Show Links: Pillow: Image Processing With Python Python 101: pythonlibrary.org Pillow: Image Processing With Python - Kickstarter Pillow: The Friendly Fork of the Python Imaging Library (PIL) Image Chops (“Channel Operations”) Module PySimpleGUI: Python GUIs for Humans PySimpleGUI: The Simple Way to Create a GUI With Python - Real Python wxPython: The GUI Toolkit for Python Create an EXIF Viewer with PySimpleGUI: Mouse Vs Python Getting GPS EXIF Data with Python: Mouse Vs Python Mouse Vs Python Blog Python Interviews: Discussions with Python Experts: Packt Publishing PyConUS 2021 PyCascades 2021 Python Pizza: Remote Conferences openpyxl - A Python library to read/write Excel 2010 xlsx/xlsm files Editing Excel Spreadsheets in Python With openpyxl: Real Python video course Episode 20: Building PDFs in Python with ReportLab Level up your Python skills with our expert-led courses: How to Work With a PDF in Python Traditional Face Detection Using Python Editing Excel Spreadsheets in Python With openpyxl Support the podcast & join our community of Pythonistas

How do you create a virtual conference that retains the interactivity of an in-person event? What are the tools needed for talk submissions, ticketing, and live hosting? Can you find those tools written in Python? This week on the show, we have several of the organizers of the PyCascades 2021 conference. They share the process of restructuring a Python conference to meet those challenges. Nina Zakharenko and Seb Vetter are co-chairs, and Ashia Zawaduk is the conference program chair. PyCascades will be held online from February 19th through 21st, with a day of virtual social events, one of live-streamed talks, and another of mentored sprints. We discuss ways to recreate the elusive feel of the “hallway” track virtually. They share advice about submitting a talk proposal and ways that you can volunteer for conferences. Tickets are available now. PyCascades is looking for additional sponsors. If you work for an organization that can help, get in contact with them. Course Spotlight: Speed Up Python With Concurrency Learn what concurrency means in Python and why you might want to use it. You’ll see a simple, non-concurrent approach and then look into why you’d want threading, asyncio, or multiprocessing. Topics: 00:00:00 – Introduction 00:01:53 – Introducing the organizers 00:03:27 – Structure and vision for the Conference 00:06:50 – Tools for a virtual conference 00:10:34 – Creating a virtual hallway track 00:12:32 – Testing the platform 00:14:33 – How does a virtual event change the type of audience? 00:15:54 – Opening up the range of available speakers and topics 00:19:35 – Tips for finding success in submitting talk proposals 00:24:28 – Sponsor: PyCharm 00:25:10 – How can someone assist at this and other conferences? 00:26:40 – Preparing first time speakers 00:28:29 – How did each of you get involved? 00:36:13 – Video Course Spotlight 00:37:18 – Currently scheduled talks 00:43:01 – Mentored Sprints for Diverse Beginners 00:49:37 – User groups and meetups 00:52:23 – PyCascades sponsors 00:57:02 – What are you excited about in the world of Python? 01:02:39 – Callout: Get Your Tickets and thanks Show Links: PyCascades 2021 PyCascades: The Team PyConline AU 2020 PyCon AU: YouTube Channel pretalx: From Call for Papers to schedule – build your conference! pretalx: GitHub pretix: Event Ticketing Software pretix: GitHub venueless: Host Your Events Online venueless: GitHub Next Day Video Resources for Virtual Events: PSF The Ultimate Guide To Memorable Tech Talks — Nina’s series of posts with lots of advice on giving excellent tech talks. Volunteer at PyCascades PyColorado 2019 PyCascades 2021: Schedule Mentored Sprints for Diverse Beginners at PyCon US 2020: readthedocs Episode 8: Docker + Python for Data Science and Machine Learning With Tania Allard PyLadies Puget Sound Programming Python (PuPPy): Meetup PyCascades: Sponsors Become Our Sponsor: PyCascades nnjaio: Nina’s Twitch Channel AlSweigart: Twitch Channel anthonywritescode: Anthony Sottile Twitch Channel crazy4pi314: Dr. Sarah Kaiser Twitch Channel TheLiveCoders: Twitch Channel MicrosoftDeveloper: Twitch Channel Architecture Patterns in Python: O’Reilly Episode 7: AsyncIO + Music, Origins of Black, and Managing Python Releases import asyncio: Learn Python’s AsyncIO #1 - The Async Ecosystem: YouTube Wagtail : The Powerful CMS for Modern Websites Episode 159: Volunteering, Organizing, and Finding a Python Community Level up your Python skills with our expert-led courses: Getting Started With Django: Building a Portfolio App Formatting Python Strings Speed Up Python With Concurrency Support the podcast & join our community of Pythonistas

What is it like to design a Python library for three different audiences? This week on the show, we have Nir Aides, creator of Jupylet. His new library is designed for deep reinforcement learning researchers, musicians interested in live music coding, and kids interested in learning to program. Everything is designed to run inside of a Jupyter notebook. Nir’s initial goal was to create a framework to study deep reinforcement learning, and this led to building a framework for 2D and 3D games and graphics. As he continued the development, he realized that this interactive environment could be a useful tool for learning Python. We also talk about how he got interested in live music coding and the advanced mathematics of sound synthesis. Nir also shares some resources for finding graphic assets and tools for creating 3D models. Course Spotlight: Using Jupyter Notebooks In this step-by-step course, you learn how to get started with the Jupyter Notebook, an open source web application that you can use to create and share documents that contain live code, equations, visualizations, and text. Topics: 00:00:00 – Introduction 00:02:25 – When did you start the project? 00:02:50 – What is deep reinforcement learning? 00:06:11 – How is deep reinforcement learning implemented in Jupylet? 00:06:56 – What graphic libraries are being used? 00:09:56 – What are the audiences for Jupylet? 00:14:15 – Why create features for musicians? 00:15:52 – Interactive code 00:19:13 – Were you using Jupyter Notebooks previously? 00:24:01 – Sponsor Digital Ocean 00:24:40 – Scaling features and making it kid friendly 00:28:59 – Outside help and learning about audio synthesis 00:33:31 – Using NumPy for synthesis, effects, and algorithmic reverb 00:39:08 – Video Course Spotlight 00:40:13 – Relying on other packages for your own package 00:42:26 – Assets for game design and working with 3D 00:47:51 – What has feedback been like? 00:48:31 – Looking for contributors 00:49:45 – More on live music looping 00:53:24 – What are you excited about in the world of Python? 00:55:41 – What do you want to learn next? 01:01:13 – Thanks and goodbye Show Links: Jupylet: GitHub Project Page Jupylet: Read the Docs Deep Reinforcement Learning: Wikipedia article DQN Breakout: YouTube Learn OpenGL ModernGL: ModernGL is a high performance rendering module for Python SID (Sound Interface Device) - C64 Wiki Chiptune: Wikipedia article The Best Chiptune Groups/Artists: Ranker.com Elektron SidStation: Wikipedia article Sonic Pi: Welcome to the future of music FoxDot: Live Coding with Python and Super Collider Nyquist frequency: Wikipedia article Aliasing: Wikipedia article Coding a basic reverb algorithm - Part 2: An introduction to audio programming Openair: Demo, download and share acoustic impulse responses Jupylet Docs: Impulse Response Files Versilian Community Sample Library: Virtual Instruments Versilian Community Sample Library: Github Free Sound Samples: One Laptop per Child One Laptop per Child: Wikipedia article Kenney.nl: Free game assets, no strings attached Texture Haven Free PBR Texture Websites: The Graphic Assembly Blender: Open Source 3D Creation Episode 7: AsyncIO + Music, Origins of Black, and Managing Python Releases import asyncio: Learn Python’s AsyncIO #1 - The Async Ecosystem PyTorch: Optimized Tensor Library for Deep Learning Using GPUs and CPUs Level up your Python skills with our expert-led courses: Playing and Recording Sound in Python Using Jupyter Notebooks Histogram Plotting in Python: NumPy, Matplotlib, Pandas & Seaborn Support the podcast & join our community of Pythonistas

Are you familiar with the role data engineers play in the modern landscape of data science and Python? Data engineering is a sub-discipline that focuses on the transportation, transformation, and storage of data. This week on the show, David Amos is back, and he’s brought another batch of PyCoder’s Weekly articles and projects. Along with the Real Python article on data engineering, we talk about a project where researchers downloaded 10 million Jupyter notebooks from Github to gather insights about the current state of data science technology. We also discuss an article about validating data in Python with the package Cerberus. And this led us to a conversation about a set of coding challenges from Advent of Code. We also cover several other articles and projects from the Python community including, building my own chess engine, the visual guide to NumPy, a free and open-source alternative to SAP, a library for working with STL files and 3D objects, and is Python really a bottleneck? Course Spotlight: Building With Django REST Framework This course will get you ready to build with Django REST Framework. The Django REST framework (DRF) is a toolkit built on top of the Django web framework that reduces the amount of code you need to write to create REST interfaces. Topics: 00:00:00 – Introduction 00:01:51 – What Is Data Engineering and Is It Right for You? 00:12:07 – Building My Own Chess Engine 00:17:52 – We Downloaded 10,000,000 Jupyter Notebooks From Github: This Is What We Learned 00:28:12 – Video Course Spotlight 00:29:20 – Is Python Really a Bottleneck? 00:34:01 – Validating Data in Python With Cerberus 00:39:04 – NumPy Illustrated: The Visual Guide to NumPy 00:42:54 – erpnext: Free and Open Source Alternative to SAP 00:48:49 – numpy-stl: Library for Working With STL Files and 3D Objects 00:54:54 – Thanks and goodbye Show Links: What Is Data Engineering and Is It Right for You? — In this article, you’ll get an overview of the discipline of data engineering. You’ll learn what is and isn’t part of a data engineer’s job, who data engineers work with, and why data engineers play a crucial role in many industries. Building My Own Chess Engine — Writing your own chess engine is a great way to explore computational complexity and combinatorial aspects of programming. Not to mention it’s pretty fun! Follow along with this reflection on how one coder created his own Chess engine from scratch. We Downloaded 10,000,000 Jupyter Notebooks From Github: This Is What We Learned — The JetBrains Datalore team downloaded ten million Jupyter Notebooks and analyzed them to determine things like which languages were the most popular, what kinds of content are in notebook cells, and how consistently notebooks can be reproduced. It’s a fascinating look into trends in data science technology! Is Python Really a Bottleneck? — Python is slow. From one perspective, that is. But what are the true bottlenecks in the data engineering/data processing space, and how does Python compare to other technologies when those factors are considered? Validating Data in Python With Cerberus — Thanks to an Advent of Code challenge, author Hector Castro was exposed to the Cerberus Python package for data validation. Get a quick introduction to Cerberus and see Hector’s solution to an Advent of Code challenge in this quick-yet-informative read. NumPy Illustrated: The Visual Guide to NumPy — This illustrated guide to NumPy is a great way to learn NumPy or brush up on the package. Full of great visual aides, this tutorial covers all the basics and more! Projects: erpnext: Free and Open Source Alternative to SAP numpy-stl: Library for Working With STL Files and 3D Objects Additional Links: Range - Why Generalists Triumph In a Specialized World: David Epstein Shannon number: Wikipedia article Apple’s open source chess engine minimum response times: Twitter thread Advent of Code cerberus: Lightweight and Extensible Data Validation Library for Python Cerberus - Greek Mythology: Wikipedia article A Visual Intro to NumPy and Data Representation Generating STL Models With Python Level up your Python skills with our expert-led courses: Getting Started With Django: Building a Portfolio App Building HTTP APIs With Django REST Framework Histogram Plotting in Python: NumPy, Matplotlib, Pandas & Seaborn Support the podcast & join our community of Pythonistas