Developer Voices

Building a database is a serious undertaking. There are just so many parts that you have to implement before you even get to a decent prototype, and so many hours of work before you could begin working on the ideas that would make your database unique. Apache DataFusion is a project that hopes to change all that, but building an extensible, composable toolkit of database pieces, which could let you build a viable database extremely quickly, and then innovate from that starting point. And even if you’re not building a database, it’s a fascinating project to explain how databases are built.Joining me to explain it all is Andrew Lamb, one of DataFusion’s core contributors, and he’s going to take us through the whole stack, how it’s built and how you could use it. Along the way we cover everything from who’s building interesting new databases and how you manage a large, open-source Rust project.–DataFusion Homepage: https://datafusion.apache.org/DataFusion on Github: https://github.com/apache/datafusionDataFusion Architecture (with diagrams!): https://youtu.be/NVKujPxwSBA?si=tw9ACxlbdpBuVsnv&t=1045Datalog: https://docs.racket-lang.org/datalog/Tokio: https://tokio.rs/Andrew’s Homepage: http://andrew.nerdnetworks.org/Andrew’s Blog Post about Tokio: https://thenewstack.io/using-rustlangs-async-tokio-runtime-for-cpu-bound-tasks/Velox: https://velox-lib.io/Arroyo: https://www.arroyo.dev/Synnada: https://www.synnada.ai/LanceDB: https://lancedb.com/SDF+DBT: https://docs.sdf.com/integrations/dbt/integratingSupport Developer Voices on Patreon: https://patreon.com/DeveloperVoicesSupport Developer Voices on YouTube: https://www.youtube.com/@developervoices/joinKris on Bluesky: https://bsky.app/profile/krisajenkins.bsky.socialKris on Mastodon: http://mastodon.social/@krisajenkinsKris on LinkedIn: https://www.linkedin.com/in/krisjenkins/

Afshin Darian, a co-creator of JupyterLab, and Sylvain Corlay, a key contributor on Jupyter's Steering Committee, delve into the inner workings of Jupyter. They explore its evolution from a Python-centric tool to a versatile, multilingual platform enhancing scientific communication. The duo discusses integrating the Language Server Protocol for coding efficiency, Jupyter's architectural complexities, and addressing security risks in collaborative environments. They also tackle the future of real-time collaboration and innovative applications, boosting productivity across multiple fields.

In this engaging conversation, Julian Arni, a Nix enthusiast and creator of Garnix, discusses how Nix revolutionizes software builds. He contrasts Nix with Docker, highlighting Nix's superior reproducibility and efficiency. Julian explains Nix's unique features like dev shells for easy onboarding and hash-based naming for reliable deployments. He also addresses the challenges of integrating Nix into tech stacks and shares valuable resources for newcomers. This lively dialogue showcases the potential of Nix in transforming development environments.

Graphite is a new image editor with an interesting architecture - it’s a classic UI-driven app, an image-manipulation language, and a library of programmable graphics primitives that any Rust coder could use, extend or add to. The result is something that you can use like Photoshop or Inkscape, or make use of in batch pipelines, a bit like ImageMagick.Joining me to discuss it are Keavon Chambers & Dennis Kobert, who are hammering away on building a project that’s potentially as demanding as Photoshop, but with a more ambitious architecture. How can they hope to compete? Perhaps in the short term by doing what regular image And is the future of image editing modular?–Graphite Homepage: https://graphite.rs/Graphite Web Version: https://editor.graphite.rs/Graphite on Github: https://github.com/GraphiteEditor/GraphiteSigned Distance Fields: https://jasmcole.com/2019/10/03/signed-distance-fields/Support Developer Voices on Patreon: https://patreon.com/DeveloperVoicesSupport Developer Voices on YouTube: https://www.youtube.com/@developervoices/joinKris on Bluesky: https://bsky.app/profile/krisajenkins.bsky.socialKris on Mastodon: http://mastodon.social/@krisajenkinsKris on LinkedIn: https://www.linkedin.com/in/krisjenkins/

Gabriel Nordeborn, a compiler contributor for ReScript, discusses the strengths and advantages of ReScript compared to TypeScript. He unpacks why JavaScript developers might want to choose ReScript, focusing on its streamlined type system and performance. The conversation touches on the challenges of TypeScript, highlights ReScript’s efficiencies in immutability and pattern matching, and explores methods for enhancing developer productivity. Gabriel also shares insights on the importance of achieving a flow state for optimal coding creativity.

Predrag Gruevski, creator of Trustfall, discusses his innovative work in developing a universal query engine in Rust. He explores the potential of querying diverse data sources, integrating REST APIs and databases with filesystems. The conversation dives into optimizing query performance and the challenges of building a flexible syntax for complex queries. Predrag also touches on the role of semantic versioning and linting tools like Cargo Semver Checks in maintaining code consistency. His insights reveal a vision for streamlined, efficient data interactions.

Dimitris Kyriakoudis, a researcher, programmer, and musician, discusses his innovative music hardware, µseq, which utilizes Lisp to control synthesizers. He dives into the结合烹饪 style of live coding, its role in real-time sound manipulation, and why Lisp is a fitting choice for musical composition. Dimitris elaborates on the challenges of moving academic innovations into accessible products and how Raspberry Pi can simplify modular synthesis. The conversation also touches on ergonomics in instrument design and enhancing coding flow through intuitive tools.

In this discussion, Diana Montalion, a coder and architect renowned for her book Learning Systems Thinking, dives deep into the complexities of large software systems. She emphasizes that software is part of a larger ecosystem involving users and teams, making change challenging but not impossible. Key topics include the necessity of systems thinking over linear approaches, the importance of effective communication in development, and nurturing collaboration within organizations. Diana also shares insights on engaging users directly for better software design.

Dmitry Stepanov, an independent game developer known for creating the Fyrox game engine in Rust, dives into his fascinating journey. He discusses the challenges of building a game engine solo, the advantages of Rust, and innovations like hot reloading. Dmitry shares insights on game architecture, scripting, and managing complex UI elements. He also touches on community contributions and the financial aspects of pursuing creative projects, highlighting the passion driving the Fyrox development.

Dave Lucia, a software engineer at TVLabs, tackles the challenges of testing television software at scale. He discusses the complexities of integration testing amidst a mix of software and hardware. With innovative techniques like real-time video processing and using Erlang and Elixir, he shares insights on enhancing reliability. The conversation also delves into GPU-powered testing devices, power management strategies, and maintaining optimal conditions for testing. Lucia highlights the shift to containerization for software deployment, streamlining updates across diverse TV models.