Taming Go's Memory Usage — and Avoiding a Rust Rewrite
🎥 Watch all the P99 CONF 2022 talks here: https://www.p99conf.io/
Last summer, my team and I faced a question many young startups face. Should we rewrite our system in Rust?
At the time of the decision, we were primarily writing in Go. I was working on an agent that passively watches network traffic, parses API calls, and sends obfuscated summaries back to our service for analysis. As users were starting to run more traffic through us, memory usage by the agent grew to an unacceptably high level, impacting performance.
This led me to spend 25 days in despair and immerse myself in the details of Go’s memory management, our technology stack, and the profiling tools available – trying to get our memory footprint back under control. Go’s fully automatic memory management makes this no easy feat.
Spoiler: I emerged victorious and our team still uses Go. In this talk, I’ll talk about key steps and lessons learned from my project. I intend this talk to be helpful for people curious about reducing their memory footprint in Go, or anybody wondering about the tradeoffs of switching to or from Go.
