OpenSearchCon India 2025

Snapshots are the backbone of data resilience and portability in OpenSearch. This session will provide a comprehensive exploration of snapshot capabilities in open-source OpenSearch, covering recent advancements and best practices for implementing robust backup, restore, and data migration strategies.

Key areas we'll delve into:
1. Fundamentals: Core architecture and mechanics of snapshot operations.
2. Repositories: Comparing storage options across file system, S3, Azure, and GCS implementations.
3. Management: Automated policy-based snapshot scheduling and lifecycle management.
4. Cross-cluster: Efficient data movement techniques for disaster recovery and migrations.
5. Searchable Snapshots: Instant access to historical data without full restoration.
6. Performance: Optimizing snapshot and restore operations for minimal cluster impact.
7. Scaling: Advanced techniques for managing snapshots in large-scale deployments.
8. Security: Repository access controls, encryption, and multi-tenant considerations.
9. Monitoring: Tools and strategies for maintaining healthy snapshot operations.
Walk away with practical knowledge to master OpenSearch snapshots, from basic to advanced solutions.

In the era of vector search and semantic similarity, returning highly relevant results is only half the battle. When search results are too similar, users must wade through redundant information to find diverse perspectives. This talk introduces Maximum Marginal Relevance (MMR) implementation in OpenSearch, a powerful technique that optimally balances result relevance with diversity.

Key takeaways will include:
- Implementing MMR reranking with OpenSearch's vector search
- Optimizing performance for large-scale deployments
- Measuring and tuning diversity metrics
- Real-world applications and success patterns

I’ll share how we built a highly available, multi-tenant, multi-AZ search platform using OpenSearch, supporting modern capabilities like vector search for semantic search. I’ll cover key challenges and how we ensured fault tolerance and scalability across a distributed environment. Additionally, I look forward to learning from the community’s experience in scaling OpenSearch for advanced search use cases.

Attendees will understand how to design a fault-tolerant OpenSearch architecture and integrate vector search to enhance accuracy and search performance. I’ll provide insights into optimising distributed setups for high availability and resilience, ensuring OpenSearch meets enterprise-scale demands.

By sharing best practices and challenges, I hope to drive discussions on multi-region deployments and hybrid search solutions. Highlighting OpenSearch’s capabilities for scalable, AI-powered search can help expand its adoption and evolution for next-gen search applications.

Learn the story of AWS OpenSearch Service, and how we evolved the OpenSearch engine to offer high durability at Petabyte scale without sacrificing on performance. We’ll share our journey of establishing core principles and breaking down complex challenges into manageable deliverables. We will cover some of the key design decisions that helped us to allow new and prevalent software engines to co-exist, without major overhaul of the system. We will walk through our open-source first journey, illuminating common pitfalls and lessons learnt in scaling, correctness and performance, which helped shape the eventual solution. Finally, we’ll reveal how we revolutionised our periodic data backup strategy, transforming a process that handles half a million partitions into one that completes in mere seconds—enabling granular, minute-by-minute backups.

Key Take-aways
- Define your tenets
- Don’t try building an ideal system
- Don’t re-invent the wheel
- Why high judgement calls matters
- Invent to simplify

If you’re still manually sifting through logs to spot issues, it’s time to rethink your observability strategy. This session will show how AI-driven OpenSearch transforms your observability practices by turning logs, metrics, and traces into actionable insights.

Using vector and neural search, how AI models can detect patterns, anomalies, and trends in real time. Learn how to reduce troubleshooting time, predict potential issues before they become critical, and empower data-driven decisions, all while scaling your observability pipeline with OpenSearch’s powerful capabilities.

Operating OpenSearch in multi-tenant environments presents challenges in performance, security, and data isolation. This session explores best practices for optimizing OpenSearch for high-scale, multi-tenant architectures, covering index sharding, access control, and query performance tuning. Attendees will gain insights from real-world deployments and learn how to ensure cost efficiency and high availability in OpenSearch clusters.