The Data Pipelines Behind Forest Carbon Credits – Why Pachama Uses Flyte to Orchestrate Workflows

Опубликовано: 27 Декабрь 2024
на канале: CNCF [Cloud Native Computing Foundation]
352
16

Don't miss out! Join us at our next Flagship Conference: KubeCon + CloudNativeCon North America in Salt Lake City from November 12 - 15, 2024. Connect with our current graduated, incubating, and sandbox projects as the community gathers to further the education and advancement of cloud native computing. Learn more at https://kubecon.io

The Data Pipelines Behind Forest Carbon Credits – Why Pachama Uses Flyte to Orchestrate Workflows - Bernhard Stadlbauer, Pachama

Navigating the current landscape of carbon credit projects demands a commitment to transparency and openness in carbon credit computation methods, particularly in the face of growing skepticism towards forest-based initiatives. The essence of trust lies in establishing a clear data lineage, tracing the journey from earth observations, such as satellite imagery, all the way to the generation of carbon credits. This talk is a case study on how Pachama uses Flyte — a Kubernetes-native workflow orchestrator — to handle complex data and machine learning pipelines. Key highlights of this session include: 1. Introduction to Flyte 2. Flyte's Workload Management in Kubernetes 3. Pachama's Flyte Configuration 4. Lessons Learned and Common Pitfalls 5. Comparison to Other Orchestrators Such As Argo or Kubeflow