Radu Gheorghe & Ciprian Hacman – Autoscaling Elasticsearch for Logs on Kubernetes
Elasticsearch (or OpenSearch) clusters likely need to scale to adapt to changes in load. But autoscaling Elasticsearch isn't trivial: indices and shards need to be well sized and well balanced across nodes. Otherwise the cluster will have hotspots and scaling it further will be less and less efficient.
This talk focuses on two aspects:
best practices around scaling Elasticsearch for logs and other time-series data
how to apply them when deploying Elasticsearch on Kubernetes. In the process, a new (open-source) operator will be introduced (yes, there will be a demo!). This operator will autoscale Elasticsearch while keeping a good balance of load. It does so by changing the number of shards in the index template and rotating indices when the number of nodes changes.
More: https://pretalx.com/bbuzz22/talk/QJZPYZ/
Web: https://2022.berlinbuzzwords.de/
Twitter: / berlinbuzzwords
LinkedIn: / berlin-buzzwords
