x

75% of people globally want unbiased news (Pew Research) but are concerned they're not getting that.

Get balanced coverage with Symby. See how stories are covered across diverse outlets and spot when one side covers a story the other skips.

PODCAST

The Data Flowcast: Mastering Apache Airflow ® for Data Engineering and AI

Welcome to The Data Flowcast: Mastering Apache Airflow ® for Data Engineering and AI— the podcast where we keep you up to date with insights and ideas propelling the Airflow community forward. Join us each week, as we explore the current state, future and potential of Airflow with leading thinkers in the community, and discover how best to leverage this workflow management system to meet the ever-evolving needs of data engineering and AI ecosystems. Podcast Webpage:

technology

All Episodes

24:28

How Airflow and AI Power Investigative Journalism at...

The Data Flowcast: Mastering Apache Airflow ® for... ·

2025/10/30

The Financial Times leverages Airflow and AI to uncover powerful stories hidden within vast, unstructured data.In this episode, Zdravko Hvarlingov, Senior Software Engineer at the Financial Times, discusses building multi-tenant Airflow systems and AI-driven pipelines that surface stories that might otherwise be missed. Zdravko walks through entity extraction and fuzzy matching, linking the UK Register of Members’ Financial Interests with Companies House, and how this work cuts weeks of manual

en

29:36

Inside Vinted’s Code-Generated Airflow Pipelines with...

The Data Flowcast: Mastering Apache Airflow ® for... ·

2025/10/23

The shift from monolithic to decentralized data workflows changes how teams build, connect and scale pipelines.In this episode, we feature Oscar Ligthart, Lead Data Engineer, and Rodrigo Loredo, Lead Analytics Engineer, both at Vinted, as we unpack their YAML-driven abstraction that generates Airflow DAGs and standardizes cross-team orchestration.Key Takeaways:00:00 Introduction.05:28 Challenges of decentralization.06:45 YAML-based generator standardizes pipelines and dependencies.12:28 Declarative

en

28:32

Transforming Data Pipelines at XENA Intelligence with...

The Data Flowcast: Mastering Apache Airflow ® for... ·

2025/10/16

The shift from simple cron jobs to orchestrated AI-powered workflows is reshaping how startups scale. For a small team, these transitions come with unique challenges and big opportunities.In this episode, Naseem Shah, Head of Engineering at Xena Intelligence, shares how he built data pipelines from scratch, adopted Apache Airflow and transformed Amazon review analysis with LLMs.Key Takeaways:00:00 Introduction.03:28 The importance of building initial products that support growth and investment.06:16

en

24:03

Scaling Geospatial Workflows With Airflow at Overture...

The Data Flowcast: Mastering Apache Airflow ® for... ·

2025/10/09

Using Airflow to orchestrate geospatial data pipelines unlocks powerful efficiencies for data teams. The combination of scalable processing and visual observability streamlines workflows, reduces costs and improves iteration speed.In this episode, Alex Iannicelli, Staff Software Engineer at Overture Maps Foundation, and Daniel Smith, Senior Solutions Architect at Wherobots, join us to discuss leveraging Apache Airflow and Apache Sedona to process massive geospatial datasets, build reproducible

en

19:04

Scaling Airflow for Enterprise Data Platforms at...

The Data Flowcast: Mastering Apache Airflow ® for... ·

2025/10/02

PepsiCo’s data platform drives insights across finance, marketing and data science. Delivering stability, scalability and developer delight is central to its success, and engineering leadership plays a key role in making this possible.In this episode, Kunal Bhattacharya, Senior Manager of Data Platform Engineering at PepsiCo, shares how his team manages Airflow at scale while ensuring security, performance and cost efficiency.Key Takeaways:00:00 Introduction.02:31 Enabling developer delight by

en

24:09

Building a Unified Data Platform at Pattern with...

The Data Flowcast: Mastering Apache Airflow ® for... ·

2025/09/25

The orchestration of data workflows at scale requires both flexibility and security. At Pattern, decoupling scheduling from orchestration has reshaped how data teams manage large-scale pipelines.In this episode, we are joined by William Graham, Senior Data Engineer at Pattern, who explains how his team leverages Apache Airflow alongside their open-source tool Heimdall to streamline scheduling, orchestration and access management.Key Takeaways:00:00 Introduction.02:44 Structure of Pattern’s data

en

25:34

How Astronomer Turns Proactive Monitoring Into...

The Data Flowcast: Mastering Apache Airflow ® for... ·

2025/09/18

The evolution of Airflow continues to shape data orchestration and monitoring strategies. Leveraging it beyond traditional ETL use cases opens powerful new possibilities for proactive support and internal operations.In this episode, we are joined by Collin McNulty, Sr. Director of Global Support at Astronomer, who shares insights from his journey into data engineering and the lessons learned from leading Astronomer’s Customer Reliability Engineering (CRE) team.Key Takeaways:00:00 Introduction.03:07

en

19:26

Overcoming Data Engineering Challenges at Daiichi...

The Data Flowcast: Mastering Apache Airflow ® for... ·

2025/09/11

The shift to a unified data platform is reshaping how pharmaceutical companies manage and orchestrate data. Establishing standards across regions and teams ensures scalability and efficiency in handling large-scale analytics.In this episode, Evgenii Prusov, Senior Data Platform Engineer of Daiichi Sankyo Europe GmbH, joins us to discuss building and scaling a centralized data platform with Airflow and Astronomer.Key Takeaways:00:00 Introduction.02:49 Building a centralized data platform for 15

en

23:05

Building a Data-Driven Beauty and Wellness...

The Data Flowcast: Mastering Apache Airflow ® for... ·

2025/09/04

StyleSeat is revolutionizing how beauty and wellness professionals grow their businesses through data-driven tools. From streamlining scheduling to optimizing marketing, their platform empowers professionals to focus on their craft while expanding their client base.In this episode, Paschal Onuorah, Senior Data Engineer at StyleSeat, shares how the company leverages Airflow, dbt, and Cosmos to drive marketplace intelligence, improve client connections and deliver measurable growth for

en

22:25

Building the Future of Airflow Execution at...

The Data Flowcast: Mastering Apache Airflow ® for... ·

2025/08/28

The evolution of orchestration in Airflow continues with innovations that address both scalability and security. From improving executor reliability to enabling remote execution, these advancements reshape how organizations manage data pipelines.In this episode, we’re joined by Ian Buss, Principal Software Engineer at Astronomer, and Piotr Chomiak, Principal Product Manager at Astronomer, who share insights into the Astro Executor and remote execution.Key Takeaways:00:00 Introduction.04:13 How

en

24:17

Scaling On-Prem Airflow With 2,000 DAGs at Numberly...

The Data Flowcast: Mastering Apache Airflow ® for... ·

2025/08/21

Scaling 2,000+ data pipelines isn’t easy. But with the right tools and a self-hosted mindset, it becomes achievable.In this episode, Sébastien Crocquevieille, Data Engineer at Numberly, unpacks how the team scaled their on-prem Airflow setup using open-source tooling and Kubernetes. We explore orchestration strategies, UI-driven stakeholder access and Airflow’s evolving features.Key Takeaways:00:00 Introduction.02:13 Overview of the company’s operations and global presence.04:00 The tech stack and

en

21:32

How Moniepoint Group Uses Airflow for Exposure...

The Data Flowcast: Mastering Apache Airflow ® for... ·

2025/08/14

Managing financial data at scale requires precise orchestration and proactive monitoring to maintain operational efficiency.In this episode, we are joined by Adeolu Adegboye, Data Engineer at Moniepoint Group, who shares how his team uses data pipelines and workflow automation to manage high volumes of transactions, ensure timely alerts and support diverse stakeholders across the business.Key Takeaways:(00:00) Introduction.(02:48) The role of data engineering in supporting all business

en

28:02

Inside Bosch’s Airflow 3 Revolution: Remote Execution...

The Data Flowcast: Mastering Apache Airflow ® for... ·

2025/08/07

The evolution of Airflow has reached a milestone with the introduction of remote execution in Airflow 3, enabling flexible orchestration across distributed environments.In this episode, Jens Scheffler, Test Execution Cluster Technical Architect at Bosch, shares insights on how his team’s need for large-scale, cross-environment testing influenced the development of the Edge Executor and shaped this major release.Key Takeaways:(02:39) The role of remote execution in supporting large-scale testing

en

31:24

Inside Modern Data Infrastructure at Massdriver with...

The Data Flowcast: Mastering Apache Airflow ® for... ·

2025/07/31

Managing modern data platforms means navigating a web of complex infrastructure, competing team needs and evolving security standards. For data teams to truly thrive, infrastructure must become both accessible and compliant without sacrificing velocity or reliability.In this episode, we’re joined by Cory O’Daniel, CEO and Co-Founder at Massdriver, and Jacob Ferriero, Senior Software Engineer at Astronomer, to unpack what it takes to make data platform engineering scalable, sustainable and secure

en

21:55

The Future of Airflow Telemetry at Metyis with Bolke...

The Data Flowcast: Mastering Apache Airflow ® for... ·

2025/07/17

Telemetry has the potential to guide the future of Airflow, but only if it’s implemented transparently and with community trust.In this episode, we’re joined by Bolke de Bruin, Director at Metyis and a long-time Airflow PMC member. Bolke discusses how telemetry has been handled in the past, why it matters now and what it will take to get it right.Key Takeaways:(03:20) The role of foundations in establishing credibility and sustainability.(04:52) Why data collection is critical to open-source project

en

22:28

Transforming the Airflow UI for Cloudera’s Users with...

The Data Flowcast: Mastering Apache Airflow ® for... ·

2025/07/10

Contributing to open-source projects can be daunting, but it can also unlock unexpected innovation. This episode showcases how one engineer’s journey with Apache Airflow led to impactful UI enhancements and infrastructure solutions at scale. Shubham Raj, Software Engineer II at Cloudera, shares how his contributions helped shape Airflow 3.0, including an intuitive drag-and-drop DAG editor and a new REST API endpoint for managing XComs.Key Takeaways:(02:30) Day-to-day responsibilities building

en

19:34

Streamlining Thousands of Data Pipelines at Lyft with...

The Data Flowcast: Mastering Apache Airflow ® for... ·

2025/07/07

Managing data pipelines at scale is not just a technical challenge. It is also an organizational one. At Lyft, success means empowering dozens of teams to build with autonomy while enforcing governance and best practices across thousands of workflows.In this episode, we speak with Yunhao Qing, Software Engineer at Lyft, about building a governed data-engineering platform powered by Airflow that balances flexibility, standardization and scale.Key Takeaways:(03:17) Supporting internal teams with a

en

22:19

Transforming Customer Education in Data Engineering...

The Data Flowcast: Mastering Apache Airflow ® for... ·

2025/06/26

Understanding the complexities of Apache Airflow can be daunting for newcomers and seasoned data engineers. But with the right guidance, mastering the tool becomes an achievable milestone.In this episode, Marc Lamberti, Head of Customer Education at Astronomer, joins us to share his journey from Udemy instructor to driving education at Astronomer, and how he's helping over 100,000 learners demystify Airflow.Key Takeaways:(02:36) Early exposure to Airflow while addressing inefficiencies in data

en

30:09

Embracing Data Mesh and SQL Sensors for Scalable...

The Data Flowcast: Mastering Apache Airflow ® for... ·

2025/06/19

The flexibility of Airflow plays a pivotal role in enabling decentralized data architectures and empowering cross-functional teams.In this episode, we speak with Alberto Crespi, Data Architect at lastminute.com, who shares how his team scales Airflow across 12 teams while supporting both vertical and horizontal structures under a data mesh approach.Key Takeaways:(02:17) Defining responsibilities within data architecture teams.(04:15) Consolidating multiple orchestrators into a single

en

23:21

The AI-Ready Pipeline: Reimagining Airflow at Veyer®...

The Data Flowcast: Mastering Apache Airflow ® for... ·

2025/06/12

Innovation in orchestration is redefining how engineers approach both traditional ETL pipelines and emerging AI workloads. Understanding how to harness Airflow’s flexibility and observability is essential for teams navigating today’s evolving data landscape.In this episode, Anu Pabla, Principal Engineer at The ODP Corporation, joins us to discuss her journey from legacy orchestration patterns to AI-native pipelines and why she sees Airflow as the future of AI workload orchestration.Key

en

79 results

Similar Podcasts

Lead With AI

Tamara Nall

Linux in the Ham Shack

Black Sparrow Media

Data Engineering Podcast

Tobias Macey

Eye On A.I.

Craig S. Smith

Reflections

Livewire Labs