This chapter delves into the architecture and design principles behind Nextflow, emphasizing its modular approach in pipeline development and data flow paradigm. It discusses containerization of tasks, orchestration of tasks to executors like AWS Batch, and the intricate details of scientific computing pipelines, focusing on the RNA seek pipeline example. The chapter also explores scaling and parallelism in scientific computing pipelines, decision-making processes at Secara, and the evolution of the Nextflow community through community-contributed content and industry collaborations.
NextFlow is a tool for managing scientific computation workflows. It’s increasingly popular for bioinformatics, computational biology, and other life science applications.
Evan Floden is the Co-Founder and CEO of Seqera Labs which develops NextFlow. He joins the show today to talk about his background as a scientist and engineer, the modular design of NextFlow pipelines, the unique challenges of genomic sequence data formats, and more.
Sean’s been an academic, startup founder, and Googler. He has published works covering a wide range of topics from information visualization to quantum computing. Currently, Sean is Head of Marketing and Developer Relations at Skyflow and host of the podcast Partially Redacted, a podcast about privacy and security engineering. You can connect with Sean on Twitter @seanfalconer .
The post Biotech Special: Scientific Computing Pipelines with Evan Floden appeared first on Software Engineering Daily.