OpenSearchCon India 2025

Cost for running OpenSearch also always been a huge factor. In spite of OpenSearch being completely open-source and no licensing cost, when you are dealing with terabytes and petabytes of data, cost can quickly stack.

This talk aims to empower OpenSearch users by taking a holistic view using best practices and with practical steps & checklist for reducing the infrastructure cost and to do more with less with OpenSearch regardless of the workload (be it Search/Log Analytics/Observability/Semantic Search).

We'll discuss

1/ Gaining optimal performance from existing resources
2/ How to right size a domain & when to Horizontal scale vs. Vertical scale
3/ How to scale down without impacting performance
4/ How much heap memory is too much or too little and what is the correct vCPU ratio for the given heap memory
5/ When to de-compose larger clusters into smaller clusters Vs. When to aggregate smaller clusters to few larger ones
6/ How to write cache efficient queries, optimize mapping and the index settings
7/ Knobs to improve ingestion and search speed
8/ Budget friendly ways to implement Semantic Search

Learn with examples the variety of OpenSearch driven - Vector, Neural, Conversational, Semantic and Multimodal Search and the difference to cost , resiliency and response time in a Managed Vs Self run Cluster. Lessons, Challenges and Solutions would enable you to architect and run it successfully to be the data exhaust fingerprint for your product.

OpenSearch is a powerful open-source search and analytics engine, but running it effectively on Kubernetes requires a well-structured approach. In this session, I will explore how to deploy, configure, and operate OpenSearch in a Kubernetes-native way. I will dive into best practices for maintaining OpenSearch clusters, optimizing performance, and handling scaling challenges.

Additionally, I will cover how to ship Kubernetes logs to OpenSearch, enabling centralized log management for a single cluster and also across multiple clusters. I will discuss scalable log ingestion strategies involving off the shelf as well as custom made tools. The session will also include practical demonstrations on building insightful dashboards for observability and setting up alerts to proactively monitor system health.

This talk will provide a comprehensive guide to managing OpenSearch effectively within Kubernetes.

I hope to share our experience of managing petabyte-scale log data on OpenSearch, achieving 30% cost savings while ensuring performance and scalability. I’ll cover key strategies, challenges, and optimisations, including searchable snapshots, compression techniques, and index management to efficiently store and query massive datasets. I’m also looking forward to engaging with the OpenSearch community to exchange ideas and refine best practices for large-scale deployments.

Attendees will learn practical insights into migrating large workloads with no downtime, optimising storage through compression and index management, and balancing cost and performance using searchable snapshots. I’ll also share best practices for ensuring high availability across multiple availability zones and building strong disaster recovery strategies in OpenSearch.

By sharing real-world challenges and solutions, I aim to help organisations adopt OpenSearch confidently. Demonstrating its scalability and cost efficiency can encourage wider adoption while driving improvements in data management and operational resilience within the OpenSearch ecosystem.