Last Mile Data Processing for ML Training using Ray
In this talk, we will discuss how Pinterest integrated Ray to streamline ML innovations. With Ray, ML engineers at Pinterest can scale the end-to-end ML lifecycle from data processing to model training all within a single Job, minimizing the need for a workflow framework and greatly enhancing the ML developer velocity.
We’ll visit how we integrated Ray with Pinterest’s in-house Kubernetes infrastructure, and how it supports business-critical use cases. Furthermore, we’ll share insights into the challenges faced, lessons learned, and the benefits Ray brings to our business.
Find the slide deck here: https://drive.google.com/file/d/1mIDG...
About Anyscale
---
Anyscale is the AI Application Platform for developing, running, and scaling AI.
https://www.anyscale.com/
If you're interested in a managed Ray service, check out:
https://www.anyscale.com/signup/
About Ray
---
Ray is the most popular open source framework for scaling and productionizing AI workloads. From Generative AI and LLMs to computer vision, Ray powers the world’s most ambitious AI workloads.
https://docs.ray.io/en/latest/
#llm #machinelearning #ray #deeplearning #distributedsystems #python #genai
