x

75% of people globally want unbiased news (Pew Research) but are concerned they're not getting that.

Get balanced coverage with Symby. See how stories are covered across diverse outlets and spot when one side covers a story the other skips.

PODCAST

Confluent Developer ft. Tim Berglund, Adi Polak & Viktor Gamov

Hi, we’re Tim Berglund, Adi Polak, and Viktor Gamov and we’re excited to bring you the Confluent Developer podcast (formerly “Streaming Audio.”) Our hand-crafted weekly episodes feature in-depth interviews with our community of software developers (actual human beings - not AI) talking about some of the most interesting challenges they’ve faced in their careers. We aim to explore the conditions that gave rise to each person’s technical hurdles, as well as how their experiences transformed their

technology how to

All Episodes

30:42

Git for Data: Managing Data like Code with lakeFS

Confluent Developer ft. Tim Berglund, Adi Polak &... ·

2023/01/19

Is it possible to manage and test data like code? lakeFS is an open-source data version control tool that transforms object storage into Git-like repositories, offering teams a way to use the same workflows for code and data. In this episode, Kris sits down with guest Adi Polak, VP of DevX at Treeverse, to discuss how lakeFS can be used to facilitate better management and testing of data.At its core, lakeFS provides teams with better data management. A theoretical data engineer on a large team runs

en-us

51:06

Using Kafka-Leader-Election to Improve Scalability...

Confluent Developer ft. Tim Berglund, Adi Polak &... ·

2023/01/12

How does leader election work in Apache Kafka®? For the past 2 ½ years, Adithya Chandra, Staff Software Engineer at Confluent, has been working on Kafka scalability and performance, specifically partition leader election. In this episode, he gives Kris Jenkins a deep dive into the power of leader election in Kafka replication, why we need it, how it works, what can go wrong, and how it's being improved.Adithya explains that you can configure a certain number of replicas to be distributed across

en-us

38:56

Real-Time Machine Learning and Smarter AI with Data...

Confluent Developer ft. Tim Berglund, Adi Polak &... ·

2023/01/05

Are bad customer experiences really just data integration problems? Can real-time data streaming and machine learning be democratized in order to deliver a better customer experience? Airy, an open-source data-streaming platform, uses Apache Kafka® to help business teams deliver better results to their customers. In this episode, Airy CEO and co-founder Steffen Hoellinger explains how his company is expanding the reach of stream-processing tools and ideas beyond the world of programmers.Airy

en-us

31:19

The Present and Future of Stream Processing

Confluent Developer ft. Tim Berglund, Adi Polak &... ·

2022/12/28

The past year saw new trends emerge in the world of data streaming technologies, as well as some unexpected and novel use cases for Apache Kafka®. New reflections on the future of stream processing and when companies should adopt microservice architecture inspired several talks at this year’s industry conferences. In this episode, Kris is joined by his colleagues Danica Fine, Senior Developer Advocate, and Robin Moffatt, Principal Developer Advocate, for an end-of-year roundtable on this year’s

en-us

70:58

Top 6 Worst Apache Kafka JIRA Bugs

Confluent Developer ft. Tim Berglund, Adi Polak &... ·

2022/12/21

Entomophiliac, Anna McDonald (Principal Customer Success Technical Architect, Confluent) has seen her fair share of Apache Kafka® bugs. For her annual holiday roundup of the most noteworthy Kafka bugs, Anna tells Kris Jenkins about some of the scariest, most surprising, and most enlightening corner cases that make you ask, “Ah, so that’s how it really works?”She shares a lot of interesting details about how batching works, the replication protocol, how Kafka’s networking stack dances with Linux’s

en-us

31:29

Learn How Stream-Processing Works The Simplest Way...

Confluent Developer ft. Tim Berglund, Adi Polak &... ·

2022/12/20

Could you explain Apache Kafka® in ways that a small child could understand? When Mitch Seymour, author of Mastering Kafka Streams and ksqlDB, wanted a way to communicate the basics of Kafka and event-based stream processing, he decided to author a children’s book on the subject, but it turned into something with a far broader appeal.Mitch conceived the idea while writing a traditional manuscript for engineers and technicians interested in building stream processing applications. He wished he could

en-us

53:06

Building and Designing Events and Event Streams with...

Confluent Developer ft. Tim Berglund, Adi Polak &... ·

2022/12/15

What are the key factors to consider when developing event-driven architecture? When properly designed, events can connect existing systems with a common language and allow data exchange in near real time. They also help reduce complexity by providing a single source of truth that eliminates the need to synchronize data between different services or applications. They enable dynamic behavior, allowing each service or application to respond quickly to changes in its environment. Using events,

en-us

41:23

Rethinking Apache Kafka Security and Account Management

Confluent Developer ft. Tim Berglund, Adi Polak &... ·

2022/12/08

Is there a better way to manage access to resources without compromising security? New employees need access to a variety of resources within a company's tech stack. But manually granting access can be error-prone. And when employees leave, their access must be revoked, thus potentially introducing security risks if an admin misses one. In this podcast, Kris Jenkins talks to Anuj Sawani (Security Product Manager, Confluent) about the centralized identity management system he helped build to

en-us

29:18

Real-time Threat Detection Using Machine Learning and...

Confluent Developer ft. Tim Berglund, Adi Polak &... ·

2022/11/29

Can we use machine learning to detect security threats in real-time? As organizations increasingly rely on distributed systems, it is becoming more important to analyze the traffic that passes through those systems quickly. Confluent Hackathon ’22 finalist, Géraud Dugé de Bernonville (Data Consultant, Zenika Bordeaux), shares how his team used TensorFlow (machine learning) and Neo4j (graph database) to analyze and detect network traffic data in real-time. What started as a research and development

en-us

29:32

Improving Apache Kafka Scalability and Elasticity...

Confluent Developer ft. Tim Berglund, Adi Polak &... ·

2022/11/22

What happens when you need to store more than a few petabytes of data? Rittika Adhikari (Software Engineer, Confluent) discusses how her team implemented tiered storage, a method for improving the scalability and elasticity of data storage in Apache Kafka®. She also explores the motivating factors for building it in the first place: cost, performance, and manageability. Before Tiered Storage, there was no real way to retain Kafka data indefinitely. Because of the tight coupling between compute and

en-us

38:38

Decoupling with Event-Driven Architecture

Confluent Developer ft. Tim Berglund, Adi Polak &... ·

2022/11/15

In principle, data mesh architecture should liberate teams to build their systems and gather data in a distributed way, without having to explicitly coordinate. Data is the thing that can and should decouple teams, but proper implementation has its challenges.In this episode, Kris talks to Florian Albrecht (Solution Architect, Hermes Germany) about Galapagos, an open-source DevOps software tool for Apache Kafka® that Albrecht created with his team at Hermes, a German parcel delivery company. After

en-us

43:58

If Streaming Is the Answer, Why Are We Still Doing...

Confluent Developer ft. Tim Berglund, Adi Polak &... ·

2022/11/09

Is real-time data streaming the future, or will batch processing always be with us? Interest in streaming data architecture is booming, but just as many teams are still happily batching away. Batch processing is still simpler to implement than stream processing, and successfully moving from batch to streaming requires a significant change to a team’s habits and processes, as well as a meaningful upfront investment. Some are even running dbt in micro batches to simulate an effect similar to

en-us

48:33

Security for Real-Time Data Stream Processing with...

Confluent Developer ft. Tim Berglund, Adi Polak &... ·

2022/11/03

Streaming real-time data at scale and processing it efficiently is critical to cybersecurity organizations like SecurityScorecard. Jared Smith, Senior Director of Threat Intelligence, and Brandon Brown, Senior Staff Software Engineer, Data Platform at SecurityScorecard, discuss their journey from using RabbitMQ to open-source Apache Kafka® for stream processing. As well as why turning to fully-managed Kafka on Confluent Cloud is the right choice for building real-time data pipelines at scale

en-us

58:44

Running Apache Kafka in Production

Confluent Developer ft. Tim Berglund, Adi Polak &... ·

2022/10/27

What are some recommendations to consider when running Apache Kafka® in production? Jun Rao, one of the original Kafka creators, as well as an ongoing committer and PMC member, shares the essential wisdom he's gained from developing Kafka and dealing with a large number of Kafka use cases.Here are 6 recommendations for maximizing Kafka in production:1. Nail Down the Operational PartWhen setting up your cluster, in addition to dealing with the usual architectural issues, make sure to also invest

en-us

37:18

Build a Real Time AI Data Platform with Apache Kafka

Confluent Developer ft. Tim Berglund, Adi Polak &... ·

2022/10/20

Is it possible to build a real-time data platform without using stateful stream processing? Forecasty.ai is an artificial intelligence platform for forecasting commodity prices, imparting insights into the future valuations of raw materials for users. Nearly all AI models are batch-trained once, but precious commodities are linked to ever-fluctuating global financial markets, which require real-time insights. In this episode, Ralph Debusmann (CTO, Forecasty.ai) shares their journey of migrating from

en-us

71:42

Optimizing Apache JVMs for Apache Kafka

Confluent Developer ft. Tim Berglund, Adi Polak &... ·

2022/10/13

Java Virtual Machines (JVMs) impact Apache Kafka® performance in production. How can you optimize your event-streaming architectures so they process more Kafka messages using the same number of JVMs? Gil Tene (CTO and Co-Founder, Azul) delves into JVM internals and how developers and architects can use Java and optimized JVMs to make real-time data pipelines more performant and more cost effective, with use cases.Gil has deep roots in Java optimization, having started out building large data centers

en-us

6:42

Apache Kafka 3.3 - KRaft, Kafka Core, Streams, &...

Confluent Developer ft. Tim Berglund, Adi Polak &... ·

2022/10/03

Apache Kafka® 3.3 is released! With over two years of development, KIP-833 marks KRaft as production ready for new AK 3.3 clusters only. On behalf of the Kafka community, Danica Fine (Senior Developer Advocate, Confluent) shares highlights of this release, with KIPs from Kafka Core, Kafka Streams, and Kafka Connect. To reduce request overhead and simplify client-side code, KIP-709 extends the OffsetFetch API requests to accept multiple consumer group IDs. This update has three changes, including

en-us

39:10

Application Data Streaming with Apache Kafka and Swim

Confluent Developer ft. Tim Berglund, Adi Polak &... ·

2022/10/03

How do you set data applications in motion by running stateful business logic on streaming data? Capturing key stream processing events and cumulative statistics that necessitate real-time data assessment, migration, and visualization remains as a gap—for event-driven systems and stream processing frameworks according to Fred Patton (Developer Evangelist, Swim Inc.) In this episode, Fred explains streaming applications and how it contrasts with stream processing applications. Fred and Kris also

en-us

62:22

International Podcast Day - Apache Kafka Edition |...

Confluent Developer ft. Tim Berglund, Adi Polak &... ·

2022/09/30

What’s your favorite podcast? Would you like to find some new ones? In celebration of International Podcast Day, Kris Jenkins invites 12 experts from the Apache Kafka® community to talk about their favorite podcasts. Unlike other episodes where guests educate developers and tell stories about Kafka, its surrounding technological ecosystem, or the Cloud, this special episode provides a glimpse into what these guests have learned through listening to podcasts that you might also find

en-us

1:26

How to Build a Reactive Event Streaming App - Coding...

Confluent Developer ft. Tim Berglund, Adi Polak &... ·

2022/09/20

How do you build an event-driven application that can react to real-time data streams as they happen? Kris Jenkins (Senior Developer Advocate, Confluent) will be hosting another fun, hands-on programming workshop—Coding in Motion: Watching the River Flow, to demonstrate how you can build a reactive event streaming application with Apache Kafka®, ksqlDB using Python.As a developer advocate, Kris often speaks at conferences, and the presentation will be available on-demand through the organizer’s

en-us

292 results

Similar Podcasts

Lead With AI

Tamara Nall

Linux in the Ham Shack

Black Sparrow Media

Data Engineering Podcast

Tobias Macey

Eye On A.I.

Craig S. Smith

Reflections

Livewire Labs