Dmitry Petrov: DataOps & ML automation with DVC
Data Fest Online 2020 https://fest.ai/2020/
ML REPA Track https://ods.ai/tracks/ml-repa-df2020
Recently, automation in software development has reached an unprecedented level of adoption. Infrastructure as code (IaC) principles, supported by tools like Terraform, Puppet, and Ansible, have become a mainstream approach to automating software building, testing, and deployment. In contrast, data science and machine learning projects frequently involve many manual steps, including data transfer and processing, model training and evaluation, and provisioning resources like cloud compute and storage. Each manual step lowers the overall reproducibility of a project and creates another hurdle to productionizing a project.
