Parsl: Enabling Scalable Interactive Computing in Python | SciPy 2018 | Kyle Chard
Scientists and engineers like to use Python for interactive data science, machine learning, and online computing. However, computations that are simple to perform at small scales (e.g., on a laptop) can easily become prohibitively difficult as data sizes and analysis complexity grows. For example, efficient interactive analysis at scale can require real-time management of parallel and/or cloud computing resources, orchestration of remote task execution, and data staging across wide area networks. In this talk we introduce Parsl (Parallel Scripting Library), a pure Python library for orchestrating the concurrent and parallel execution of interactive and many-task workloads, and demonstrate how it integrates with the scientific Python ecosystem and how it is being used in a variety of scientific domains. The talk is intended for attendees interested in interactive and parallel computing.
See the full SciPy 2018 playlist at • SciPy 2018: Scientific Computing with...
