Distributed systems scientist Jonathan Magen discusses functional programming in distributed systems using languages like Gleam, Elixir, Ballerina, and Jolie. They cover type inference, big data processing, and the advantages of using purely functional languages in ensuring code reliability.
Efficiency in distributed systems may not always improve with task distribution due to coordination inefficiencies.
Security aspects are often neglected in formal methods, focusing on functional correctness and absence of race conditions.
Choreographic programming offers structured messaging interactions to simplify distributed system design and ensure secure communication flows.
Deep dives
Distributed Systems Challenges
Building distributed systems challenges the assumption that distributing tasks automatically leads to performance gains. Research highlights the slowdown caused by coordination protocols, emphasizing that not all algorithms benefit from distribution due to coordination inefficiencies. When data sets exceed single-machine capacities, strategies like computational locality may enhance processing but aren't universally effective. Many machine learning workloads misuse parallelization due to tools' inefficiencies, leading to unnecessary complexity.
Type Inference and Functional Correctness
The focus of formal methods often gravitates towards ensuring functional correctness ahead of time, neglecting the critical domain of security aspects. Tools like TLA plus or Alloy primarily validate algorithms for proper functioning and absence of race conditions. However, foretelling potential security threats, mitigating attacks like double spending, or handling malicious nodes are essential but sidelined concerns in system design.
Networking and Message Passing Abstractions
Choreographic programming offers high-level specifications for messaging interactions to ease distributed system design. Frameworks enforcing communication protocols and implicit capabilities using actors or processes convey a structured approach to managing message passing. Defining service protocols and handles for remote entities facilitates secure and predictable communication flows across distributed environments.
Ergonomics in Distributed Systems Design
Streamlining distributed systems' complexity requires intuitive abstractions that mask underlying intricacies while emphasizing security and fault tolerance. Tools like Zeno and Iro aim to simplify design complexities, offering security features, type specifications based on protocols, and abstracting low-level networking intricacies for enhanced usability. Continuous efforts focus on enhancing usability and reliability in distributed systems.
Closing Thoughts on Distributed Systems Research
The evolving landscape of distributed systems research underscores the importance of user-centric design, security focus, and abstraction layers for simplifying system design. Encouraging practitioners to persevere amidst challenges, advancements in tools like ZeroMQ and innovative projects demonstrate ongoing commitment to enhancing distributed system ergonomics and robustness.
Richard talks with distributed systems scientist Jonathen Magen about functional programming in distributed systems, including languages like Gleam, Elixir, Ballerina, and Jolie. They also talk about type inference, big data, and a few other topics.
Jonathan Magen: https://yonkeltron.com or https://jawns.club/@yonkeltron
Programming languages mentioned:
https://ballerina.io
https://www.jolie-lang.org
https://gleam.run
https://elixir-lang.org
Richard's talk: Why Static Typing Came Back - https://youtu.be/Tml94je2edk