x

75% of people globally want unbiased news (Pew Research) but are concerned they're not getting that.

Get balanced coverage with Symby. See how stories are covered across diverse outlets and spot when one side covers a story the other skips.

PODCAST

The Data Flowcast: Mastering Apache Airflow ® for Data Engineering and AI

Welcome to The Data Flowcast: Mastering Apache Airflow ® for Data Engineering and AI— the podcast where we keep you up to date with insights and ideas propelling the Airflow community forward. Join us each week, as we explore the current state, future and potential of Airflow with leading thinkers in the community, and discover how best to leverage this workflow management system to meet the ever-evolving needs of data engineering and AI ecosystems. Podcast Webpage:

technology

All Episodes

17:43

Open Source Airflow Contributions and Performance...

The Data Flowcast: Mastering Apache Airflow ® for... ·

2026/03/19

Modern Airflow isn’t just orchestration. It's a contribution.In this episode, we explore how open source investment drives real performance gains and deeper observability.We’re joined by Christos Bisias, Open Source Software Engineer, Apache Airflow at G-Research, to discuss how his team uses Airflow for large-scale data transformations, contributes upstream and improves scheduler throughput and OpenTelemetry support. From trace-level observability to CI-enforced metrics governance and a major

en

22:14

Automating Threat Intelligence Using Airflow with...

The Data Flowcast: Mastering Apache Airflow ® for... ·

2026/03/12

In this episode, Karan Alang, Principal Software Engineer at Versa Networks, joins the conversation to discuss how Airflow can be used to automate threat intelligence in modern cybersecurity environments. He explains the growing scale of cloud computing, the profitability of hacking and the shortage of SOC analysts. Karan also outlines a novel architecture that combines Airflow, XDR, graph databases and LLMs to orchestrate automated threat detection and response.Key Takeaways:00:00

en

27:45

Using Plugins To Customize Airflow at Ponder Labs...

The Data Flowcast: Mastering Apache Airflow ® for... ·

2026/03/05

In this episode, we explore how teams scale Apache Airflow in complex environments and what it takes to make orchestration work across many stakeholders. We look at real-world challenges around visibility, ownership and predictability as data platforms grow.Egor Tarasenko, Data and AI Engineer at Ponder Labs, joins us to share how Ponder Labs customizes Airflow for education organizations using plugins, event-driven architectures and AI-powered tooling. He explains how his team supports large

en

18:00

Scaling Airflow at Wix for Analytics and AI with...

The Data Flowcast: Mastering Apache Airflow ® for... ·

2026/02/26

Modern data orchestration at scale demands reliability, speed and thoughtful adoption of new tooling. As organizations grow, keeping pipelines efficient while supporting more teams becomes a critical challenge.In this episode, we’re joined by Ethan Shalev, Data Engineer at Wix, to discuss how Wix operates Airflow at massive scale, migrates to Airflow 3 and uses AI to accelerate development.Key Takeaways:00:00 Introduction.02:13 Wix structures data engineering across multiple product-focused

en

24:49

Using Airflow To Orchestrate Billions of Events at...

The Data Flowcast: Mastering Apache Airflow ® for... ·

2026/02/19

Strong data orchestration is as much about culture and visibility as it is about technology. As data platforms scale, teams need systems that reduce cognitive load while increasing reliability and observability.In this episode, Carlos Daniel Puerto Niño, Senior Analytics Engineer and Data Analyst at Addi, joins us to share how Addi uses Airflow to support batch orchestration, manage organizational complexity and improve monitoring across its data platform.Key Takeaways:00:00 Introduction.01:25

en

18:43

Building Event-Driven Data Pipelines With Airflow 3...

The Data Flowcast: Mastering Apache Airflow ® for... ·

2026/02/12

Real-time data expectations are reshaping how modern data teams think about orchestration and dependencies. As event-driven architectures become more common, teams need to rethink how pipelines react to data changes, rather than schedules.In this episode, Andrea Bombino, Co-Founder and Head of Analytics Engineering at Astrafy, joins us to discuss how event-driven scheduling in Airflow is evolving and how Astrafy applies it to deliver faster, more responsive data pipelines.Key Takeaways:00:00

en

18:55

Uphold’s Approach to Orchestrating Modern Data...

The Data Flowcast: Mastering Apache Airflow ® for... ·

2026/02/05

A strong data-driven mindset underpins how fintech teams scale analytics, infrastructure and decision-making across the business.In this episode, Jaime Oliveira, Lead Data Engineer at Uphold, joins us to discuss how Uphold structures its data organization and orchestration strategy. Jaime shares how the team uses Airflow and dbt to support analytics, reporting and data activation while evolving their approach as the stack grows.Key Takeaways:00:00 Introduction.01:23 A data-driven mindset supports

en

17:10

Modern Airflow Best Practices for Scalable Data...

The Data Flowcast: Mastering Apache Airflow ® for... ·

2026/01/29

Building reliable data pipelines at scale requires more than writing code. It depends on thoughtful design, infrastructure trade-offs and an understanding of how orchestration platforms evolve over time.In this episode, Airflow best practices shaped by real-world implementation are examined. Bhavani Ravi, Independent Software Consultant and Apache Airflow Champion, shares lessons on pipeline design, architectural decisions and the evolution of the Airflow ecosystem in modern data environments.Key

en

21:45

Inside Conviva’s Decision To Power Its Data Platform...

The Data Flowcast: Mastering Apache Airflow ® for... ·

2026/01/22

Conviva operates at a massive scale, delivering outcome-based intelligence for digital businesses through real-time and batch data processing. As new use cases emerged, the team needed a way to extend a streaming-first architecture without rebuilding core systems.In this episode, Han Zhang joins us to explain how Conviva uses Apache Airflow as the orchestration backbone for its batch workloads, how the control plane is designed and what trade-offs shaped their platform decisions.Key Takeaways:00:00

en

24:16

Why Airflow Became the Scheduling Backbone at Condé...

The Data Flowcast: Mastering Apache Airflow ® for... ·

2026/01/15

Data platforms are moving from batch-first pipelines to near real-time systems where orchestration, observability, scalability and governance all have to work together.In this episode, Arun Karthik, Director, Data Solutions Engineering at Condé Nast Technology Lab, joins us to share how data engineering evolves from relational databases and ETL into distributed processing, modern orchestration with Apache Airflow and managed Airflow with Astronomer.Key Takeaways:00:00 Introduction.02:13 Early data

en

18:28

How Snowflake Uses Airflow Sensors To Keep Financial...

The Data Flowcast: Mastering Apache Airflow ® for... ·

2026/01/08

Clean financial reporting depends on orchestration that catches issues early, prevents bad data from spreading and helps teams react fast when something breaks.In this episode, Ayush Pradhan, Senior Analytics Engineer at Snowflake, joins us to explain how Snowflake’s finance data team relies on @Apache Airflow, sensors and dbt to keep revenue, cost and accounting pipelines accurate.Key Takeaways:00:00 Introduction.02:25 Airflow is used to coordinate recurring finance workflows.04:20 Sensors enforce

en

19:30

The Role of Airflow in Building Smarter ML Pipelines...

The Data Flowcast: Mastering Apache Airflow ® for... ·

2025/12/11

The integration of data orchestration and machine learning is critical to operational efficiency in healthcare tech. Vivian Health leverages Airflow to power both its ETL pipelines and ML workflows while maintaining strict compliance standards.Max Calehuff, Lead Data Engineer at Vivian Health, joins us to discuss how his team uses Airflow for ML ops, regulatory compliance and large-scale data orchestration. He also shares insights into upgrading to Airflow 3 and the importance of balancing

en

34:24

Scaling Airflow to 11,000 DAGs Across Three Regions...

The Data Flowcast: Mastering Apache Airflow ® for... ·

2025/12/04

The evolution of Intercom’s data infrastructure reveals how a well-built orchestration system can scale to serve global needs. With thousands of DAGs powering analytics, AI and customer operations, the team’s approach combines technical depth with organizational insight.In this episode, András Gombosi, Senior Engineering Manager of Data Infra and Analytics Engineering, and Paul Vickers, Principal Engineer, both at Intercom, share how they built one of the largest Airflow deployments in production

en

23:10

How Covestro Turns Airflow Into a Simulation Toolbox...

The Data Flowcast: Mastering Apache Airflow ® for... ·

2025/11/20

Building scalable, reproducible workflows for scientific computing often requires bridging the gap between research flexibility and enterprise reliability.In this episode, Anja MacKenzie, Expert for Cheminformatics at Covestro, explains how her team uses Airflow and Kubernetes to create a shared, self-service platform for computational chemistry.Key Takeaways:00:00 Introduction.06:19 Custom scripts made sharing and reuse difficult.09:29 Workflows are manually triggered with user traceability.10:38

en

21:16

Building Secure Financial Data Platforms at...

The Data Flowcast: Mastering Apache Airflow ® for... ·

2025/11/13

The use of Apache Airflow in financial services demands a balance between innovation and compliance. Agile Engine’s approach to orchestration showcases how secure, auditable workflows can scale even within the constraints of regulatory environments.In this episode, Valentyn Druzhynin, Senior Data Engineer at AgileEngine, discusses how his team leverages Airflow for ETF calculations, data validation and workflow reliability within tightly controlled release cycles.Key Takeaways:00:00

en

23:48

How Redica Transformed Their Data With Airflow and...

The Data Flowcast: Mastering Apache Airflow ® for... ·

2025/11/06

The life sciences industry relies on data accuracy, regulatory insight and quality intelligence. Building a unified system that keeps these elements aligned is no small feat.In this episode, we welcome Shankar Mahindar, Senior Data Engineer II at Redica Systems. We discuss how the team restructures its data platform with Airflow to strengthen governance, reduce compliance risk and improve customer experience.Key Takeaways:00:00 Introduction.01:53 A focused analytics platform reduces compliance risk

en

24:28

How Airflow and AI Power Investigative Journalism at...

The Data Flowcast: Mastering Apache Airflow ® for... ·

2025/10/30

The Financial Times leverages Airflow and AI to uncover powerful stories hidden within vast, unstructured data.In this episode, Zdravko Hvarlingov, Senior Software Engineer at the Financial Times, discusses building multi-tenant Airflow systems and AI-driven pipelines that surface stories that might otherwise be missed. Zdravko walks through entity extraction and fuzzy matching, linking the UK Register of Members’ Financial Interests with Companies House, and how this work cuts weeks of manual

en

29:36

Inside Vinted’s Code-Generated Airflow Pipelines with...

The Data Flowcast: Mastering Apache Airflow ® for... ·

2025/10/23

The shift from monolithic to decentralized data workflows changes how teams build, connect and scale pipelines.In this episode, we feature Oscar Ligthart, Lead Data Engineer, and Rodrigo Loredo, Lead Analytics Engineer, both at Vinted, as we unpack their YAML-driven abstraction that generates Airflow DAGs and standardizes cross-team orchestration.Key Takeaways:00:00 Introduction.05:28 Challenges of decentralization.06:45 YAML-based generator standardizes pipelines and dependencies.12:28 Declarative

en

28:32

Transforming Data Pipelines at XENA Intelligence with...

The Data Flowcast: Mastering Apache Airflow ® for... ·

2025/10/16

The shift from simple cron jobs to orchestrated AI-powered workflows is reshaping how startups scale. For a small team, these transitions come with unique challenges and big opportunities.In this episode, Naseem Shah, Head of Engineering at Xena Intelligence, shares how he built data pipelines from scratch, adopted Apache Airflow and transformed Amazon review analysis with LLMs.Key Takeaways:00:00 Introduction.03:28 The importance of building initial products that support growth and investment.06:16

en

24:03

Scaling Geospatial Workflows With Airflow at Overture...

The Data Flowcast: Mastering Apache Airflow ® for... ·

2025/10/09

Using Airflow to orchestrate geospatial data pipelines unlocks powerful efficiencies for data teams. The combination of scalable processing and visual observability streamlines workflows, reduces costs and improves iteration speed.In this episode, Alex Iannicelli, Staff Software Engineer at Overture Maps Foundation, and Daniel Smith, Senior Solutions Architect at Wherobots, join us to discuss leveraging Apache Airflow and Apache Sedona to process massive geospatial datasets, build reproducible

en

95 results

Similar Podcasts

Lead With AI

Tamara Nall

Linux in the Ham Shack

Black Sparrow Media

Data Engineering Podcast

Tobias Macey

Eye On A.I.

Craig S. Smith

Reflections

Livewire Labs