Talks - Pablo Galindo Salgado: Profiling at the speed of light
Did you know that Python 3.12 will include one of the world's smallest just-in-time (JIT) compilers? Also, you will be surprised to learn it is not what you probably think it is.
Python 3.12 will include support for the Linux perf profiler. The Linux perf profiler is a very powerful tool that allows you to profile and obtain information about the performance of your application. perf also has a very vibrant ecosystem of tools that help with the analysis of the data that it produces.
In this talk, we will talk about how this exciting feature was implemented, how the support provided by the perf profiler differs from other performance-oriented profilers for Python, and how it can be used effectively, including how to activate it dynamically to enable production profiling. We will also cover some of the requirements that are needed to obtain the best results, as well as some of the limitations of the implementation and how those can affect your metrics.
Being able to understand where our Python applications are expending their time is crucial to being able to improve the performance characteristics of our applications. Several tools already exist to help with this task, but they all have their own limitations, especially when native code written in C, C++, Rust, etc. is involved. Being able to gather and cross-correlate performance information with other performance-related markers, such as branch mispredictions, cache misses, context switches, and other events, can be key in understanding some of the most challenging profiling puzzles.
