Contributor

Eric Anderson (@ericmander) and Dor Laor (@DorLaor) go under the hood of Scylla, the open-source NoSQL database designed for low latency and high throughput in big data applications. Dor and his team have reimplemented Apache Cassandra in C++ from scratch, with additional compatibility for DynamoDB. In today’s episode, Dor shares details on the exciting work coming out of ScyllaDB, including Seastar, their open-source C++ framework. Also, check out Scylla Summit 2021 to learn what’s next for Scylla. In this episode we discuss: Enabling Scylla to “gain control” by implementing Apache Cassandra in C++ How Dor and his co-founder were ahead of the curve with their vision for virtualization Scylla’s unique shard-per-core architecture Working with distributed teams, even before the COVID-19 pandemic The growing significance of separating the interface from the engine in open-source Learn about Project Circe, which is being featured at Scylla Summit 2021 right now Links: Scylla Seastar Scylla Summit 2021 Apache Cassandra DynamoDB MongoDB Redhat QEMU Redis Vectorized Apache Hadoop Apache HBase Apache Beam Apache Flink Apache Spark People mentioned: Avi Kivity (@AviKivity)

Eric Anderson (@ericmander) chats with Sven Efftinge (@svenefftinge), Christian Weichel (@csweichel) and Gero Posmyk-Leinemann (Github: @geropl) about their work on Gitpod, an open-source Kubernetes application that allows engineers to spin up a server-side dev-environment from a Git repository, all within their browser. The three team members are part of TypeFox, a consulting firm that specialized in developer tools for different companies before branching out into open-source projects. Upon Gero’s hiring at TypeFox, he was tasked with creating a minimum viable product for the idea that would eventually become Gitpod. Tune in to hear how shifting from consulting to working on their own open-source projects was a breath of fresh air for the developers at TypeFox. In this episode we discuss: How Gitpod solves the problem of switching between multiple dev environments, and improves deep code review The trap that many open-source founders fall into Why TypeFox wanted to switch from a consulting firm to a product shop Details on how Gitpod handles licensing Learn how you can instantly try out a Gitpod environment for any existing Github repository Links: Gitpod TypeFox Theia Kubernetes People mentioned: Anton Kosyakov (@akosyakov) Sid Sijbrandij (@sytses)

Eric Anderson (@ericmander) interviews Graham Neray (@grahamneray) about oso, the open-source policy engine for authorization. oso was originally born from a desire to make infrastructure and security easier for developers, which is why Graham and his company describe themselves as being in the “friction-removal business.” Listen to today’s episode to learn how the team at oso are working to put security in the hands of developers.  In this episode we discuss: Developers building RBAC (role-based access control) systems over and over again Why open-source is the best way to handle authorization logic The history behind oso’s core policy language, Polar How someone beat Graham to the punch submitting oso to a Python newsletter Comparing oso and OPA (Open Policy Agent) Links: oso Stripe Trulioo MongoDB Auth0 Show HN OPA Polar Adventure People mentioned: Sam Scott (@samososos) Alex Plotnick (Github: @plotnick) Stephen Olsen (@olsenator4) Other episodes: Presto on Contributor OPA on Contributor

Eric Anderson (@ericmander) is joined by Rajat Monga (@rajatmonga), a co-creator of TensorFlow. Originally developed by the Google Brain team, TensorFlow is now one of the most popular open-source libraries for machine learning. The team at TensorFlow seek to “democratize” the world of AI as we know it, and by all accounts, they are succeeding. Listen to today’s episode to get inside one of the largest and most exciting open-source projects of the decade. In this episode we discuss: How TensorFlow compares to other open-source projects at Google Taking bets on launch day numbers Balancing the demands of different kinds of TensorFlow users Lessons from Keras and PyTorch Links: TensorFlow Keras  PyTorch Kafka Kubernetes MapReduce: Simplified Data Processing on Large Clusters Bigtable: A Distributed Storage System for Structured Data People mentioned: Jeff Dean (@JeffDean) Andrew Ng (@AndrewYNg) François Chollet (@fchollet)

Eric Anderson (@ericmander) and Frank McSherry (@frankmcsherry) dive into Materialize, a source-available streaming database that lets engineers build real-time applications. Frank is a data processing expert whose work at Microsoft Research on the Timely and Differential Dataflow models culminated in the Materialize project. Tune in to today’s episode to learn how the team at Materialize are making the technology from cutting-edge data research accessible to a wider swath of users. In this episode we discuss: Sharing early ideas with an “academic open source” approach How Materialize made a commitment to correctness Frank’s developmental philosophy of iterative thinking Novel applications for the Materialize community Changing the way we approach problems with real-time data processing Links: Materialize Naiad: A Timely Dataflow System DryadLINQ Apache Arrow People mentioned: Arjun Narayan (@narayanarjun) Derek Murray (@mrry)

Eric Anderson (@ericmander) speaks with Thomas Graf (@tgraf__) about Cilium, the open-source networking, observability, and security software for cloud-native applications based on eBPF. Thomas is the co-founder and CTO of Isovalent, which maintains both eBPF and Cilium. Listen to today’s episode for a discussion of how Thomas’ work has leveled up the Linux kernel and the possibilities of network infrastructure in a cloud-native world. In this episode we discuss: The impact of simultaneous development on Cilium and eBPF Google’s incorporation of Cilium Shortening the gap between writing kernel code and its deployment What JavaScript and eBPF have in common Cilium’s sister project, Hubble Links: Cilium eBPF Isovalent Red Hat OpenShift Kubernetes Docker New GKE Dataplane V2 increases security and visibility for containers SPIFFE Istio People mentioned: Brendan Gregg (@brendangregg) Other episodes: Istio on Contributor

Eric Anderson (@ericmander) and Jeremiah Lowin (@jlowin) discuss Prefect, a workflow management system and data orchestration tool under development as an open-source project. Jeremiah initially created Prefect to solve a technical challenge specific to his own work, but soon realized that it was appealing to a very wide range of different clients. Listen to today’s episode to learn why Jeremiah believes most attempts to build a unified framework for solving data orchestration fail. In this episode we discuss: Solving the “negative engineering problem” Learning from the complaints of data engineers at Apache Airflow The difficulty of having a product that serves two masters How COVID changed the direction of Prefect Links: Prefect Apache Airflow Why Not Airflow? People mentioned: Jim O'Shaughnessy (@jposhaughnessy) Patrick O’Shaughnessy (@patrick_oshag)

Eric Anderson (@ericmander) catches up with Torin Sandall (@sometorin), co-creator of Open Policy Agent (OPA), the open-source, general-purpose policy engine. By focusing on demonstrating OPA’s value through case studies, targeted interviews, and word-of-mouth, Torin and the folks at Styra were able to grow OPA into the emerging standard for unified policy enforcement across the cloud-native stack. In this episode we discuss: When Netflix stumbled across OPA and delivered its “Cinderella moment” Why OPA was designed to be developer-centric The value of demonstrating OPA’s use cases to the industry How one user created an RPG engine with OPA Links: Open Policy Agent Styra OpenStack LinkerD Hacker News Kubernetes KubeCon OPA Gatekeeper conftest Corrupting the Open Policy Agent to Run My Games Envoy Styra Academy People mentioned: Tim Hinrichs (@tlhinrchs) William Morgan (@wm) Kevin Hoffman (@kevinhoffman) Other episodes: LinkerD on Contributor Envoy on Contributor

Eric Anderson (@ericmander) and Maxim Fateev (@mfateev) trace the development of Temporal, an open-source workflow orchestration engine. At Uber, Maxim co-created the project’s predecessor, Cadence, but Temporal’s roots stretch farther back to include lessons learned at Amazon and Microsoft. In this episode, learn how 18 years of experience in asynchronous messaging and workflows culminated in the foundation of Temporal. In this episode we discuss: Why Maxim quit Uber to start his own company Differences between Temporal and Cadence How Uber is filling the position that Google once had incubating open-source projects Maxim’s advice for aspiring open-source founders Related Links: Temporal Cadence Kafka HashiCorp BanzaiCloud Hacker News Andreesen Horowitz TChannel Hadoop People mentioned: Samar Abbas (@samarabbas77)

Eric Anderson (@ericmander) and Manish Jain (@manishrjain) discuss the impact of Dgraph, an open-source database with a graph backend that Manish describes as “a search engine acting as a database.” Manish took a gamble when he chose GraphQL as his project’s query language shortly after its release by Facebook in 2015. Now, GraphQL has grown immensely in popularity and the bet has paid off, as Dgraph leads the cutting edge of databases in this new space. Make sure to check out the Dgraph team’s conference, “GraphQL In Space,” which will be held virtually on September 10th at graphqlcon.space. In this episode we discuss: How Manish was ahead of the curve at Google The chance circumstances in the Australian job market that led to Dgraph Building trust between open-source developers and their community Why the Dgraph team decided to hold their upcoming conference “In Space” The future of databases and GraphQL Related Links: Dgraph GraphQL In Space GraphQL Badger MongoDB BigTable Cassandra Spanner Elasticsearch People mentioned: Scott Kelly (@StationCDRKelly)