Contributor Data Processing Evolved: OpenLineage with Willy Lulciuc
Nov 18, 2025
Willy Lulciuc, a data engineer and co-creator of OpenLineage, shares his journey in the field from his time at WeWork to founding Oleander, which focuses on AI-enabled data tooling. He discusses the importance of data lineage and the challenges of observability in data processing. Willy explains how OpenLineage emerged from previous projects, the synergy with OpenTelemetry, and how AI can enhance, but not replace, the role of data engineers. He also outlines the future of data platforms and the unique benefits of OpenLineage for engineering teams.
AI Snips
Chapters
Transcript
Episode notes
Origin Story At WeWork
- Willy Lulciuc traces his lineage work back to a whiteboard at WeWork where a metadata model was born.
- He then implemented that model in code and prototyped Marques inside WeWork for two years.
From Marques To Datakin
- Marques was prototyped internally then donated to a foundation as the team formed Datakin.
- Datakin built a POC and struggled with custom integrations across frameworks, motivating a spec-based approach.
Integration Hell Motivated The Spec
- Integration churn was the primary pain: every framework required bespoke lineage emitters and broke on upgrades.
- A standardized spec (OpenLineage) would let framework owners maintain their integrations instead of downstream teams.