Daniel Stenberg, creator of curl, discusses its origins, 20 billion installations, space projects, and coding in C vs. Rust. They explore Rust and C integration challenges, parallel processing, and adapting curl across platforms. The podcast ends with encouragement for the Rust community to participate in open source projects.
Curl's integration of Rust libraries like Hyper and Rust-tls showcases adaptability to modern technologies and the need for testing and refinement.
Rust's potential to replace C as the preferred language for low-level development in curl signifies a promising shift in the tech landscape.
The challenges faced in supporting HTTP/2 with Curl and hyper integration underscore the importance of compliance, debugging, and continuous improvement.
Deep dives
The Evolution of curl - From HTTP Get to 20 Billion Devices
Curl, a powerful tool founded by Daniel Stenberg in late 1996, began its journey when Stenberg sought to download currency rates from the web. Initially taking over the maintenance of HTTP Get, the precursor of curl, Stenberg's dedication and expertise led to curl becoming a fundamental component of internet transfers, garnering approximately 20 billion installations due to its multiple uses across devices like mobile phones.
Curl's Adoption of Rust - Hyper and Rust-tls Integration
Curl's integration of Rust libraries, particularly Hyper for HTTP 1 and 2 support, alongside Rust-tls for TLS capabilities, signifies its adaptability to evolving technologies. While these Rust backends undergo testing, they are considered experimental, awaiting further maturity and refinement. Stenberg emphasizes the challenge of balancing exposure and optimization for these Rust components within curl's test suite.
Rust's Future in Curl and C's Replacement Potential
Assessing Rust's long-term role in curl, Stenberg envisions Rust as a promising alternative to C, potentially evolving as the preferred language for low-level and library development within the project. Considering Rust's capabilities and lack of substantial competitors in this domain, Stenberg highlights its potential to rival C's widespread usage. However, he remains cautious about predicting Rust's enduring popularity, acknowledging the uncertainties of the ever-changing tech landscape.
Integrating Rust in Curl Backends
The podcast delves into the integration of Rust components into Curl, highlighting the benefits of utilizing modern hardware, SIMD, and multiple threads to enhance efficiency and performance. Additionally, it explores how Curl, through event systems like E-Poll or Libevent, manages concurrent transfers seamlessly in a single thread, resembling async programming in Rust. The discussion emphasizes the flexibility of Curl, enabling users to select event systems based on their requirements for high-performance lib-code transfers, ultimately enhancing the overall functionality and performance.
Challenges and Enhancements in HTTP/2 Support
The episode addresses the challenges encountered in supporting HTTP/2 with Curl and hyper integration, revealing a need for a deeper understanding of managing streams over a single connection for HTTP/2 requests. Through debugging and adjustments, such as stricter parsing to align with hyper standards, improvements were made to the native parser, ensuring compliance with HTTP specifications. The conversation highlights the symbiotic relationship between Rust, Curl, and hyper, fostering continuous improvements and mutual benefits across the Rust community.