Daniel Stenberg, creator of curl, discusses its origins, 20 billion installations, space projects, and coding in C vs. Rust. They explore Rust and C integration challenges, parallel processing, and adapting curl across platforms. The podcast ends with encouragement for the Rust community to participate in open source projects.
Curl's integration of Rust libraries like Hyper and Rust-tls showcases adaptability to modern technologies and the need for testing and refinement.
Rust's potential to replace C as the preferred language for low-level development in curl signifies a promising shift in the tech landscape.
The challenges faced in supporting HTTP/2 with Curl and hyper integration underscore the importance of compliance, debugging, and continuous improvement.
Deep dives
The Evolution of curl - From HTTP Get to 20 Billion Devices
Curl, a powerful tool founded by Daniel Stenberg in late 1996, began its journey when Stenberg sought to download currency rates from the web. Initially taking over the maintenance of HTTP Get, the precursor of curl, Stenberg's dedication and expertise led to curl becoming a fundamental component of internet transfers, garnering approximately 20 billion installations due to its multiple uses across devices like mobile phones.
Curl's Adoption of Rust - Hyper and Rust-tls Integration
Curl's integration of Rust libraries, particularly Hyper for HTTP 1 and 2 support, alongside Rust-tls for TLS capabilities, signifies its adaptability to evolving technologies. While these Rust backends undergo testing, they are considered experimental, awaiting further maturity and refinement. Stenberg emphasizes the challenge of balancing exposure and optimization for these Rust components within curl's test suite.
Rust's Future in Curl and C's Replacement Potential
Assessing Rust's long-term role in curl, Stenberg envisions Rust as a promising alternative to C, potentially evolving as the preferred language for low-level and library development within the project. Considering Rust's capabilities and lack of substantial competitors in this domain, Stenberg highlights its potential to rival C's widespread usage. However, he remains cautious about predicting Rust's enduring popularity, acknowledging the uncertainties of the ever-changing tech landscape.
Integrating Rust in Curl Backends
The podcast delves into the integration of Rust components into Curl, highlighting the benefits of utilizing modern hardware, SIMD, and multiple threads to enhance efficiency and performance. Additionally, it explores how Curl, through event systems like E-Poll or Libevent, manages concurrent transfers seamlessly in a single thread, resembling async programming in Rust. The discussion emphasizes the flexibility of Curl, enabling users to select event systems based on their requirements for high-performance lib-code transfers, ultimately enhancing the overall functionality and performance.
Challenges and Enhancements in HTTP/2 Support
The episode addresses the challenges encountered in supporting HTTP/2 with Curl and hyper integration, revealing a need for a deeper understanding of managing streams over a single connection for HTTP/2 requests. Through debugging and adjustments, such as stricter parsing to align with hyper standards, improvements were made to the native parser, ensuring compliance with HTTP specifications. The conversation highlights the symbiotic relationship between Rust, Curl, and hyper, fostering continuous improvements and mutual benefits across the Rust community.
In the season premier we talk to none other than Daniel Stenberg! We focus on integrating Rust modules in curl, their benefits, ways in which Rust and Rust crates helped improve curl, but also how curl helped those crates, and where curl is used in the official Rust toolchain. Along the way we also learn about the early history of curl and Rust, which section of your car’s owner’s-manual you should “re”-read, some weird HTTP edge-cases, and Daniel’s experience in open-source maintainership.
About curl
Curl started as a simple way to download currency conversion rates from the internet and evolved into a general data transfer library and cli tool with support for not only HTTP, but also FTP, IMAP, MQTT, and many more. It rivals with SQLite for the #1 spot on the most deployed software list, leaving Java far behind.
About Daniel Stenberg
Daniel Stenberg has been the lead developer of curl for more than 25 years, and is an avid speaker and famous open source personality. Having worked at Mozilla around the time Rust was created, he now works for WolfSSL.
Proudly Supported by CodeCrafters
CodeCrafters helps you become proficient in Rust by building real-world, production-grade projects. Learn hands-on by creating your own shell, HTTP server, Redis, Kafka, Git, SQLite, or DNS service from scratch.
Start for free today and enjoy 40% off any paid plan by using this link.
