Open||Source||Data

This episode features a panel discussion with Charna Parkey, a Real-Time AI Product and Strategy leader at DataStax; and Sam Bean, Staff Engineer at You.com. Charna is a co-author and inventor on several patents, including patent-pending work on ML/coordinated feature engine at the edge. Sam helped create the Spark connector to Weaviate, and is passionate about Big Data, Spark, NLP, Hugging Face, and large language models.In this episode, Charna and Sam discuss adapting to user expectations, what’s missing in the AI stack, and how to become an advanced citizen in open source.-------------------"We've seen these companies start to better understand that these streaming technologies have a place, whether it's Kafka or Flink or Pulsar, but it's still incredibly difficult to use and we need a different level of abstraction. [...] We're starting to see the stack change so that it becomes more interchangeable of the components and try to sort of raise that layer of abstraction so that we can get these types of models and these types of capabilities to more people." – Charna Parkey"I think that a lot of what you need to adjust to are these, what you were discussing as I call interaction data, you were calling it event data. But these interactions that people have with the internet and trying to find ways to model that in a way that even if your models aren't real-time, having ways to featurize real-time data in a way that's interpretable by a model. [...] I think Spark and Kafka and Delta and all of those things, give you a lot more flexibility now to move in different directions and readjust and I think, pivot what you want to do with the system." – Sam Bean-------------------Episode Timestamps:(01:29): Sam explains his background(03:36): Charna explains her background(18:13): Sam explains the problems You.com is solving for(28:21): Changes in user expectations in the AI-native stack(39:09): Advice for becoming an advanced citizen in open source(47:25): What’s missing in the AI stack(54:51): What open source data means to the panelists(58:22): How technologists should prepare for the future(01:03:10): Executive producer, Audra Montenegro's backstage takeaways-------------------Links:LinkedIn - Connect with CharnaVisit DataStaxLinkedIn - Connect with SamVisit You.com

This episode features a panel discussion with Mikiko Bazeley, Head of MLOps at Featureform; Zain Hasan, Senior Developer Advocate at Weaviate; and Tuana Celik, Developer Advocate at deepset.In this episode, Mikiko, Zain, and Tuana discuss what open source data means to them, how their companies fit into the AI-first ecosystem, and how jobs will need to evolve with the AI-native stack.-------------------“We're almost part of a fancy new AI robot kitchen that you'd find in Tokyo, in some ways. I see a virtual feature store as, yes, you can have a bunch of your ingredients tossed into a closet. Or, what you can do is you can essentially have a nice way to organize them. You can have a way to label them, to capture information.” – Mikiko Bazeley“I really like that analogy as well. I like how Mikiko put it where a vector search engine is really extracting value from what you've already got. [...] So where I see vector search engines, really, is if we think of these embedding providers as the translators to take all of our unstructured data and bring it into vector space into a common machine language, vector search engines are essentially the workhorses that allow us to compute and search over these objects in vectorized format. They're essentially the calculators of the AI stack.” – Zain Hasan“Haystack, I would really position as the kitchen. I need Mikiko to bring the apples. I need Zain to bring the pears. I need Hugging Face or OpenAI to bring the oranges to make a good fruit salad. But, Haystack will provide the spoons and the pans and the knives to make that into something that works together.” – Tuana Celik-------------------Episode Timestamps:(02:08): What open source data means to the panelists(08:22): What interested the panelists about AI/ML(23:20): Mikiko explains Featureform(26:11): Zain explains Weaviate(29:34): Tuana explains deepset(35:11): The panelists discuss how their companies fit into the AI-first ecosystem(44:12): How jobs need to evolve with the AI-native stack(53:45): Executive producer, Audra Montenegro's backstage takeaways-------------------Links:LinkedIn - Connect with MikikoVisit FeatureformLinkedIn - Connect with ZainVisit WeaviateLinkedIn - Connect with TuanaVisit deepsetVisit Data-centric AI

This special episode of Open||Source||Data features an interview with Patrick McFadin. Patrick has been a distributed systems hacker since he first plugged a modem into his Atari computer. Looking for adventure, he joined the US Navy, working on the Naval Tactical Data System (NTDS), which cemented his love of distributed systems. He is now an Apache Cassandra Committer, and is the Vice President of Developer Relations at DataStax. Sam catches up with Patrick at Data Day Texas to discuss his book Managing Cloud Native Data on Kubernetes, Cassandra Forward, and the future of Apache Cassandra.-------------------“I can now use my Parquet file in Iceberg or DuckDB, and this is data that I created with Cassandra. And we're not getting to the point where we have to reinvent an entire database. We can just connect the Lego parts together and if they're open, then I don't have these encumbrances. I'm not like, ‘Well, I can connect that if I call a salesperson and get a license.’ [...] That's what's exciting to me about Cassandra, the way that the ecosystem is evolving around Cassandra. It's not, ‘Cassandra's at the center, it's just a player.’ It's at the party." – Patrick McFadin-------------------Episode Timestamps:(01:06): What open source data means to Patrick(02:11): Patrick discusses his book Managing Cloud Native Data on Kubernetes(10:02): Patrick discusses Cassandra Forward(11:09): The future of Apache Cassandra-------------------Links:LinkedIn - Connect with PatrickCassandra Forward

This episode features an interview with Denise Gosnell, Principal Product Manager at Amazon Web Services. At AWS, Denise leads product and strategy for Amazon Neptune, a fully managed graph database service. Her career centers on her passion for examining, applying, and advocating for the applications of graph data. Denise has also authored, patented, and spoken on graph theory, algorithms, databases, and applications across all industry verticals.In this episode, Sam sits down with Denise to discuss graph initiatives, the future of developer models, and what Denise learned from hiking the Appalachian Trail.-------------------“We just open sourced something called graph-explorer, which is something for the community by the community, Apache 2.0 license. graph-explorer is a low-code visualization tool. But, the best part about it is that it works for JanusGraph, it works for Blazegraph, it works for all of these graph models that we've talked about, because we've got this divided graph community, but it was written to work with all graphs. [...] Today it's all, ‘Here's your Lego blocks and build one on your own. If you want to go ahead and fork Jupyter Notebook and figure out a way to get that D3 force-directed graph way out to pop up, have fun.’ It's the first time that we've had a unified way across graph vendors and graph implementations to have a way to visualize your graph data in one tool that's open source.” – Denise Gosnell-------------------Episode Timestamps:(01:17): What open source data means to Denise(04:27): How Denise got interested in computer science(08:39): Denise’s work on graph initiatives(14:30): How Denise’s work at LDBC relates to SQL standards(23:43): The future of developer models(29:43): One question Denise wishes to be asked(34:05): Denise’s advice for graph practitioners(37:37): Executive producer, Audra Montenegro's backstage takeaways-------------------Links:LinkedIn - Connect with DeniseThe Practitioner’s Guide to Graph Data

This episode features an interview with Ben Lorica, Co-founder and Principal of Gradient Flow, a company that provides a wide range of content on data and technology. Ben is an industry expert on data, machine learning, and AI. He is a Technical Advisor for Databricks, a program chair for several data conferences, and he hosts The Data Exchange Podcast.In this episode, Sam and Ben discuss Big Data and the improvements and future opportunities of AI and machine learning.-------------------“The reason I use the word decentralize is because when you try to explain it to someone, let's say you want to train a different model for each user, or region, or sensor, or device. So you can't use necessarily just personalized because recommenders can be personalized, but they're still centralized models.” – Ben Lorica-------------------Episode Timestamps:(01:17): What open source data means to Ben(05:54): What intrigued Ben about Big Data(12:07): What brought Ben to working on Ray(16:15): Ben’s opinion on how far AI and ML have come in the last 5 years(26:38): What Ben sees happening in this space in the next 5 years(39:06): What challenges Ben sees in the next 5 years (43:51): One question Ben’s always wanted to be asked(44:55): Ben’s advice for those starting their open source data adventure(46:34): Executive producer, Audra Montenegro's backstage takeaways-------------------Links:LinkedIn - Connect with BenGradient Flow’s NewsletterGradient Flow’s 2023 Trends ReportVisit Sky Labs

Holden Karau, an Open Source Engineer at Netflix, discusses the data analysis stack, functional programming, and the future of open source software data tooling. They emphasize the importance of testing and cover various approaches. Other topics include open source data, data quality challenges, enhancing Jupyter notebooks, improving healthcare architecture, and appealing counterclaims.

This episode features an interview with Tom Baeyens, Co-founder and CTO of Soda, where he oversees the company's product development, software architecture, and technology strategy. He is passionate about open source and committed to building a community where data engineers can succeed using the Soda Data Monitoring Platform. Tom is the inventor of the widely-used open source project jBPM and Activiti. He also co-founded Effektif, a cloud process automation company.In this episode, Sam and Tom discuss the evolution of open source workflow engines, data contracts, and why data quality needs a language approach.-------------------“Where we're heading is what I think is exactly the same as with software engineering in the testing. Test-driven development was a radical new thing back then. But then it turns out, you can much more reliably release software. And this is exactly the same here. If you don't inject data testing, data observability throughout your data stack, then how are you going to trust the data that you put into your machine learning model? This is something that people are realizing, but we're still figuring out the best practices, the dos, the don'ts. We've come a long way, but there's still a way to go before this is as common and as normal as in the test-driven development software engineering space.” - Tom Baeyens-------------------Episode Timestamps:(01:23): What open source data means to Tom(04:34): Tom’s motivations for creating jBPM(09:39): What led Tom to building Soda(13:57): Why data quality needs a language approach(19:24): The community of Soda(22:47): The future of Soda as a technology(24:59): A question Tom wishes to be asked(30:24): Tom’s advice for engineers who want to leverage data observability tools-------------------Links:LinkedIn - Connect with TomTwitter - Follow TomVisit SodaCL

This episode features an interview with Matthew Rocklin, CEO of Coiled, the scalable Dask-based cloud platform. Prior to founding Coiled, Matthew worked on Dask at Anaconda and then NVIDIA where his teams focused on accelerating Dask through parallel computing and GPUs. Matthew is an industry speaker, author, and founding member of Pangeo, whose mission is to develop open source analysis tools for ocean, atmosphere, and climate science.In this episode, Sam sits down with Matthew to discuss enabling edge workers, the future of data science, and the revolution of AI and ML.-------------------“There's all sorts of fun people using these tools and that's the most fun part of this job. You get to learn so much about so many different applications that are all so different and all so fascinating. You were thinking about all these different tools and technologies and I was talking to someone once, it's like, ‘Oh, it's like you're standing on the shoulders of giants.’ That's not quite right. There's lots of sort of normal size people all standing on each other's shoulders in like a massive pyramid. [...] Dask was designed to scale up an existing ecosystem. There's a legacy Python ecosystem that’ll provide a layer of parallel computing on top of it. You can do that either by rewriting the whole thing, which is not feasible, or you can do it by talking to lots of people and getting them to integrate in interesting, fun ways. That's actually been the fun parts of Dask. I think I've probably talked to every major maintainer group ever. I have worked with them to find out the ways to get everything to work smoothly together. And that's super fun. There's an interesting sort of technical and social hacking that occurs, which I think Python has done pretty well at, historically. Which is why it has success.” – Matthew Rocklin-------------------Episode Timestamps:(00:58): What open source data means to Matthew(03:29): Matthew’s motivations behind Python(18:58): How Matthew is enabling edge workers (34:46): What the future of data Python space looks like(39:29): Matthew’s advice for the technical data audience(41:36): Executive producer, Audra Montenegro's backstage takeaways-------------------Links:LinkedIn - Connect with MatthewTwitter - Follow MatthewVisit Matthew’s WebsiteVisit DaskDask ExamplesVisit CoiledSciPy Mission

This episode features an interview with Nithya Ruff, Head of Open Source Program Office at Amazon. At Amazon, she drives open source culture and coordination and engagement with external communities. Prior to Amazon, Nithya spearheaded and grew Open Source Program Offices (OSPOs) for Comcast and Western Digital. She has also served as the Director-At-Large on the Linux Foundation Board since 2016, where she works to advance the mission of building sustainable ecosystems that are built on open collaboration.In this episode, Sam and Nithya discuss OSPOs, how to measure success, and the evolution of the data ecosystem.-------------------“I think if we look at what matters to customers, which is innovation, trust, and being a force for change with open source, then we can really deliver on the metrics that the company cares about.” – Nithya Ruff-------------------Episode Timestamps:(04:02): What open source data means to Nithya(06:29): What interested Nithya about open source software(12:34): What Nithya learned at Western Digital and Comcast that she uses now at Amazon(18:23): What Nithya teaches people in OSPO curriculum(22:06): How the open source data ecosystem has evolved in the last decade(27:44): One question Nithya wishes to be asked(30:37): Nithya’s advice for folks who want to create an OSPO-------------------Links:LinkedIn - Connect with NithyaTwitter - Follow NithyaOpen Source Law, Policy and PracticeLinkedIn - Connect with AmazonTwitter - Follow AmazonVisit Amazon

This episode features an interview with Jonathan Beri, Founder & CEO of Golioth, a commercial IoT development platform built for scale. Previously, Jonathan was a Product Manager at Particle, Google/Nest, Magneto, and Myspace where he spent his time building IoT solutions.In this episode, Sam sits down with Jonathan to discuss the concept of digital twins, the future of IoT databases, and how to build a real holodeck.-------------------“I think about IoT when I started at Nest, we had some of the best engineers I've ever worked with. Starting from first principles, defining networking protocols, and introducing new specifications that became parts of the fabric of the internet. And fast forward 10 years later, a lot of that exists now as building blocks. Someone who's not a PhD with a lifetime and achievement award from the ITF can go actually design systems that are highly productive, integrated, and enabling. And that's where I get excited. And the through line I think is enabling teams of developers to really create more with their own bare hands. And the technology around it, that is that enabler.” – Jonathan Beri-------------------Episode Timestamps:(01:33): Jonathan’s motivation for starting Golioth(08:59): The role of data in IoT(11:01): What is a digital twin and why does it matter?(17:12): The classes of problems Jonathan is trying to solve(20:35): The future of IoT databases in the next five years(31:04): What open source data means to Jonathan(32:24): Jonathan explains how to build a real holodeck(33:42): Jonathan’s advice for those excited about industrial data-------------------Links:LinkedIn - Connect with JonathanTwitter - Follow JonathanVisit Jonathan’s WebsiteLinkedIn - Connect with GoliothTwitter - Follow GoliothVisit Golioth