Challenges and Considerations in Job Processing with Elixir

Shannon Selbert, co-founder of Soren and developer of Oban, and Parker Selbert, creator of the Oban background job framework, chief architect at dscout, and co-founder of Soren, speak with SE Radio host Gavin Henry about robust job processing in Elixir. They explore the reliability, consistency, and observability in relation to job processing, to understand how Oban, Elixir, and PostgreSQL deliver them.

The Selberts describe why Oban was created, its history, which parts of the Elixir ecosystem they use, and why this would not be possible without PostgreSQL and Elixir. They discuss the lessons learned in the 5 years since the first release, as well as use cases, job throughput, the hardest problem to solve so far, workers, queues, CRON, distributed architectures, retry algorithms, just-once methodologies, the reliability the beam brings, consistency across nodes, how PostgreSQL is vital, telemetry data, best use cases for Oban, and the most common issues that new users face. Brought to you by IEEE Computer Society and IEEE Software magazine.