#132 The Past, Present, and Future, of the Data Science Notebook

10 snips

Apr 3, 2023

In this engaging discussion, Dr. Jodie Burchell, Data Science Developer Advocate at JetBrains and a former clinical psychologist, delves into the evolution of data science notebooks. She highlights how these tools transformed from local scripts to collaborative, remote environments, enhancing accessibility and efficiency. Jodie predicts future trends influenced by generative AI, emphasizing the role of ethical considerations and the human touch in data science. Her insights on the intersection of behavioral science and data advocacy are particularly enlightening!

Ask episode

AI Snips

Chapters

Transcript

Episode notes

ANECDOTE

R Markdown Introduction

Jodie Burchell's initial data science journey began with R and R Markdown.
Literate programming environments felt intuitive and cleaner than traditional scripts.

INSIGHT

Remote Notebooks

The biggest change in data science notebooks has been the shift from local to remote environments.
Remote notebooks enable complex model development and access to larger datasets.

INSIGHT

Reproducibility in Managed Notebooks

Managed notebook environments enhance reproducibility by linking notebooks to specific environments.
Containerization ensures consistent dependencies, making projects more approachable for beginners.

Get the Snipd Podcast app to discover more snips from this episode

Get the app

The concept of literate programming, or the idea of programming in a document, was first introduced in 1984 by Donald Knuth. And as of today, notebooks are now the defacto tool for doing data science work. So as the data tooling space continues to evolve at breakneck speed, what are the possible directions the data science notebook can take?

In this episode of DataFramed, we talk with Dr. Jodie Burchell, Data Science Developer Advocate at JetBrains, to find out how data science notebooks evolved into what they are today, what her predictions are for the future of notebooks and data science, and how generative AI will impact data teams going forward.

Jodie completed a Ph.D. in clinical psychology and a postdoc in biostatistics before transitioning into data science. She has since worked for 7 years as a data scientist, developing products ranging from recommendation systems to audience profiling. She is also a prolific content creator in the data science community.

Throughout the episode, Jodie discusses the evolution of data science notebooks over the last few years, noting how the move to remote-based notebooks has allowed for the seamless development of more complex models straight from the notebook environment.

Jodie and Adel’s conversation also covers tooling challenges that have led to modern IDEs and notebooks, with Jodie highlighting the importance of good database tooling and visibility. She shares how data science notebooks have evolved to help democratize data for the wider organization, the tradeoffs between engineering-led approaches to tooling compared to data science approaches, what generative AI means for the data profession, her predictions for data science, and more.

Tune in to this episode to learn more about the evolution of data science notebooks and the challenges and opportunities facing the data science community today.

Links to mentioned in the show: