OpenSearchCon India 2025

In this talk, we’ll share Uber’s journey in scaling OpenSearch to handle large scale data across 100+ clusters while ensuring high availability, fault tolerance, performance and zero-downtime upgrades. We’ll dive deep into our strategies for overcoming regional and zonal failures, observability and fleet-upgrade strategy with zero-downtime, at scale. Attendees will learn about our approaches to sharding, indexing, and data distribution that enables resilience across a global infrastructure. Additionally, we’ll cover key best practices for managing OpenSearch clusters at Uber’s scale, ensuring reliability, and achieving cost-efficiency.

Query Insights delivers deep visibility into search query execution, offering detailed metrics and actionable recommendations to help users optimize their search operations. This feature analyzes query characteristics and execution patterns, providing clear insights into system behavior at each processing stage. Users can proactively identify potential performance bottlenecks, implement recommended optimizations, and maintain optimal query performance. These capabilities contribute to improved system efficiency, better user experience, and increased operational reliability.

Overview

- Why use RAG (Retrieval-Augmented Generation)?
- How does OpenSearch contribute to building intelligent agents?

Barriers to Deploying RAG Systems in Production:

- Key concerns: Quality, Cost, Safety, Maintenance, and Knowledge/Time.
- The biggest challenge remains quality.
- What leads to quality degradation in RAG systems, and how does it happen?

Solving the Issues:

- Identifying challenges with different chunking techniques: Which one should you use, or should you develop a custom approach tailored to your needs using OpenSearch?
- Addressing problems with search/querying techniques to optimize results.

By understanding these challenges and adopting the suggested solutions, you can significantly enhance the accuracy of RAG-based applications.

If needed, we can provide a deeper dive, a demo, or even hands-on workshops on this topic.

During our college days, my team and I embarked on an ambitious project—a Scheme Recommendation Chatbot to help users find relevant government schemes using intelligent search. As students new to Retrieval-Augmented Generation (RAG), we faced a big challenge: finding an open-source, scalable, and efficient search solution that didn’t lock us into proprietary or managed tools.

That’s when we discovered OpenSearch—and it completely changed the game!

In this talk, I’ll share:
1. Our journey to OpenSearch and why we made the switch.
2. How we leveraged OpenSearch’s vector search to improve search accuracy.
3. Challenges we faced as students implementing OpenSearch for the first time.
4. Lessons learned and tips for others looking to integrate OpenSearch into their projects.

Now, as a professional working in the industry, I look back at how OpenSearch not only powered our chatbot but also taught us real-world skills. If you’re a student, developer, or someone curious about open-source search and relevance, this talk is for you.

As OpenSearch continues to gain popularity for search and analytics, understanding its security landscape becomes crucial. This session introduces threat modeling for OpenSearch deployments, helping attendees identify potential vulnerabilities and implement effective mitigation strategies. We'll explore common threats, leverage the OpenSearch Security plugin, and discuss best practices for deploying opensearch clusters.

Vector search is a rapidly growing area within search and analytics, and the k-NN (k-Nearest Neighbors) plugin in OpenSearch offers multiple engines to support this capability, with FAISS (Facebook AI Similarity Search), which provides efficient similarity search and clustering of dense vectors, being a preferred choice.

To speed up vector operations, FAISS engine utilizes SIMD (Single Instruction Multiple Data) processing, leveraging AVX (Advanced Vector Extensions) instructions on x86 architecture. With the latest AVX-512 (512-bit wide vector processing) support in FAISS, OpenSearch achieves further performance improvements in vector indexing, quantization, and search operations.

This presentation will delve into key technical advancements, including an in-depth look at AVX, the impact of AVX-512 on vector search performance and real-world benefits for OpenSearch users. Additionally, we will discuss how these enhancements lead to better price-performance and reduced TCO (Total Cost of Ownership).

Search is no longer just a feature—it’s a critical capability that drives innovation, efficiency, and decision-making across enterprises. However, scaling OpenSearch across an organization presents challenges, from infrastructure management to governance and adoption. How do you enable diverse teams—from engineering to business functions—to harness the power of OpenSearch without deep operational overhead?

This session explores how to elevate OpenSearch from a tool to an internal platform offering, allowing teams to seamlessly integrate search into their workflows. By treating OpenSearch as a self-service, scalable, and well-governed enterprise platform, organizations can unlock its potential for everything from complex knowledge engines to simpler, ad-hoc use cases.

We’ll discuss key architectural principles, governance models, automation strategies, and best practices for building an OpenSearch-powered internal platform. Whether you're dealing with enterprise-wide search initiatives, analytics, or operational insights, this talk will provide actionable strategies to ensure OpenSearch adoption is efficient, scalable, and accessible to all teams—without friction.

GDPR. HIPAA. CCPA. Red tape? Not anymore. We’ll showcase how we transformed OpenSearch into a privacy-aware analytics engine using tokenization, dynamic data masking, and a custom plugin that auto-redacts PII before indexing. Ever seen a dashboard where every number is differentially private? You will. We’ll also debate the ethics of search in a post-Roe world and open-source our “privacy score” audit tool.

Warning: This talk may ruin your faith in anonymized data… but we’ll fix it.