Exploring Rustler and NIF Integration

Rustler Core Team Member Sonny Scroggin joins Elixir Wizards Sundi Myint and Charles Suggs. Rustler serves as a bridge to write Native Implemented Functions (NIFs) in Rust that can be called from Elixir code. This combo leverages Rust's performance and memory safety while maintaining Elixir's fault tolerance and concurrency model, creating a powerful solution for CPU-intensive operations within Elixir applications.

Sonny provides guidance on when developers should consider using NIFs versus other approaches like ports or external services and highlights the considerations needed when stepping outside Elixir's standard execution model into native code.

Looking toward the future, Sonny discusses exciting developments for Rustler, including an improved asynchronous NIF interface, API modernization efforts, and better tooling. While Rust offers tremendous performance benefits for specific use cases, Sonny emphasizes that Elixir's dynamic nature and the BEAM's capabilities for distributed systems remain unmatched for many applications. Rustler simply provides another powerful tool that expands what developers can accomplish within the Elixir ecosystem.

Key topics discussed in this episode:

• Rust as a "high-level low-level language" with memory safety
• NIFs (Native Implemented Functions) in the BEAM virtual machine
• Rustler's role simplifying Rust-Elixir integration with macros
• CPU-intensive operations as primary NIF use cases
• Beam scheduler interaction considerations with native code
• Dirty schedulers for longer-running NIFs in OTP 20+
• Memory safety advantages of Rust for NIFs
• Development workflow using Mix tasks for Rustler
• Common pitfalls when first working with Rust
• Error handling improvements possible with Rustler NIFs
• Differences between ports, NIFs, and external services
• Asynchronous programming approaches in Rust versus Elixir
• Tokyo runtime integration for asynchronous operations
• Static NIFs for mobile device compatibility
• Upcoming CLI tooling to simplify Rustler development
• Rustler's API modernization efforts for better ergonomics
• Thread pool sharing across multiple NIFs
• Wasm integration possibilities for the BEAM
• Compile-time safety versus dynamic runtime capabilities
• Performance considerations when implementing NIFs
• Compiler-assisted memory management in Rust
• Automatic encoding/decoding between Rust and Elixir types
• The importance of proper error handling
• Real-world application in high-traffic authentication servers
• Community resources for learning Rustler

Links mentioned:

https://github.com/rusterlium/rustler
https://github.com/rust-lang/rust
https://www.angelfire.lycos.com/
https://www.webdesignmuseum.org/flash-websites
https://www.php.net/
https://xmpp.org/
https://jabberd2.org/
Geocities: https://cybercultural.com/p/geocities-1995/ (fun fact: when you search Geocities on Google, the results page is in Comic Sans font.)
https://bleacherreport.com/
https://hexdocs.pm/jose/readme.html
https://github.com/rust-lang/rust-bindgen
Erlang Ports: https://www.erlang.org/doc/system/c_port.html
Erlang ETFs (External Term Format): https://www.erlang.org/doc/apps/erts/erl_ext_dist.html
Elixir gRPC https://github.com/elixir-grpc/grpc
gRPC (“Remote Proceduce Call”): https://grpc.io/
dirty_cpu.ex https://github.com/E-xyza/zigler/blob/main/lib/zig/nif/dirty_cpu.ex
ets https://www.erlang.org/doc/apps/stdlib/ets.html
Mnesia https://www.erlang.org/doc/apps/mnesia/mnesia.html
VPPs (Virtual Power Plants): https://www.energy.gov/lpo/virtual-power-plants
https://nixos.org/
WASM WebAssembly with Elixir: https://github.com/RoyalIcing/Orb
Rust Tokio https://tokio.rs/

Getting Started:
https://hexdocs.pm/rustler/0.17.0/Mix.Tasks.Rustler.New.html
https://rustup.rs/

Special Guest: Sonny Scroggin.