Airflow at Monzo: Evolving our data platform as the bank scales

Опубликовано: 05 Ноябрь 2024
на канале: Apache Airflow
478
8

Presented by by Jonathan Rainer & Ed Sparkes at Airflow Summit 2023.

As a bank Monzo has seen exponential growth in active users, from 1.6 million in 2019 to 5.8 million in 2022. At the same time the number of data users and analysts has expanded from an initial team of 4 to 132. Alongside this growth, our infrastructure and tooling have had to evolve to deliver the same value at a new scale. From an Airflow installation deployed on a single monolithic instance we now deploy atop Kubernetes and have integrated our Airflow setup into the bank’s backend systems.