OpenSearchCon India 2025

Retrieval-Augmented Generation (RAG) has emerged as a powerful paradigm for grounding large language models (LLMs) in external knowledge, enabling more accurate and contextually relevant responses. However, traditional RAG pipelines often operate in a static, pre-defined manner, limiting their ability to adapt to complex queries and dynamically explore information spaces. This talk introduces "Multimodal Agentic RAG", an innovative approach that integrates autonomous agents into the RAG framework to enhance its reasoning and retrieval capabilities. This talk will delve into the architecture and implementation of multimodal RAG with agentic workflows, showcasing its potential to address limitations of traditional RAG. The architecture is based on OpenSearch's multimodal capabilities, DeepSeek-R1's reasoning capabilities, and custom-tuned embedding models. Furthermore, we will discuss the challenges and future directions of Agentic RAG, including the development of robust planning mechanisms, efficient knowledge source management, and effective agent coordination.

With the rise of Agentic AI, search is no longer just about retrieving information—it’s about enabling AI-driven reasoning, contextual awareness, and decision-making. As AI applications become more autonomous, the need for efficient, scalable, and intelligent search platforms has never been greater.

This session explores how OpenSearch can be leveraged as a foundational platform for Agentic AI applications, enabling capabilities like semantic search, vector-based retrieval, hybrid search, and retrieval-augmented generation (RAG). We’ll discuss architectural patterns, indexing strategies, multi-modal search, and real-time data enrichment, ensuring AI agents can access and utilize knowledge effectively.

By adopting a platform-centric approach, organizations can standardize and scale OpenSearch to serve diverse AI-driven use cases—from chatbots and virtual assistants to complex enterprise automation systems. Join us to discover how search is evolving into a key enabler for the next generation of AI applications.

HPCM (HPE Performance Cluster Manager) is a suite for monitoring HPC systems, consisting of rack, chassis, compute nodes(GPU, CPU), power,cooling components. OpenSearch, deployed in a distributed manner, enables efficient indexing and querying of large volumes of data. For real-time monitoring, events across the system is collected and streamed to Kafka, then persisted in OpenSearch via Logstash, allowing for efficient log management. OpenSearch Dashboards, integrated with Fluent Bit, provide real-time log observability, enabling users to visualize and analyze system logs for insightful cluster monitoring. We have developed a user-friendly wrapper around OpenSearch Alerting that uses custom YAML-based configurations to define alerting rules, categorize alerts by severity and group, and trigger notifications to endpoints like email,alertmanager. Argonne National Laboratory's Aurora, the third-fastest supercomputer in the world, uses 21,248 CPUs and 63,744 GPUs across 166 racks. HPCM, along with its 10-instance OpenSearch database deployed on 10 nodes, successfully manages and monitors events and logs, with OpenSearch Alerting effectively deployed to track critical system issues.

Struggling to tame the latency of your OpenSearch query? Atlassian was facing the exact problem with a query which had grown organically over a number of years of relevance improvements. Learn how they dove into the internals of OpenSearch to discover how their query worked, reducing their query latency by 70% in the process.

This session will cover:

* Techniques for identifying problem areas
* Shard topology
* Conjunctive and disjunctive boolean queries and how they impact latency
* track_total_hits, how it works and subtleties of when it doesn't
* Concurrent segment search
* Joins, when denormalisation is faster and when it isn't
* Index sorting
* initial query scoring vs rescoring
* Cross field match vs copy_to
* Parallel recall paths using hybrid queries or multi-search
* Query cache vs pre-computing
* LTR plugin, feature logging and ranking and impact on latency

Walk through our journey of creating a developer-centric documentation search platform that understands code snippets, technical context, and user intent. Learn how we enhanced OpenSearch's capabilities with custom analyzers for programming languages, implemented semantic code search, and built an intelligent query understanding system. We'll share our unique approach to ranking technical content and how we achieved a 40% improvement in search success rates for our developer community.

In modern applications, observability is essential. OpenTelemetry (OTel) has emerged as the standard for telemetry data, providing a unified way to collect, process, and export logs, metrics, and traces. But organizations also need a scalable way to store, search, and analyze this data for actionable insights.

This is where OpenSearch comes in. As an open-source search and analytics engine, it provides a robust observability backend, enabling real-time indexing, querying, and visualization of logs and metrics. Its distributed architecture ensures scalability and high availability, making it ideal for modern observability stacks.

This session is for developers looking to deepen their understanding of OpenTelemetry's architecture and key pillars. We will explore how OpenSearch seamlessly integrates into an OpenTelemetry-based observability stack, playing a key role in logging, metrics storage, and real-time analytics.

Audience Takeaway: A clear understanding of OpenTelemetry’s core components, how OpenSearch enhances observability, and its role in logging and metrics.

What if OpenSearch could manage itself? This session explores how AI, automation, and cloud-native tools are transforming OpenSearch operations. Learn how Kubernetes, Terraform, and AI-driven monitoring enable self-healing clusters, automated scaling, and hands-free management—bringing the NoOps vision to search and analytics infrastructure

I hope to share how we extended OpenSearch Dashboards (OSD) beyond traditional log analytics to create a single pane of glass for tracing, profiling, and alert management. I’ll discuss how we optimised query performance to handle massive log volumes while keeping dashboards responsive. By presenting our journey, I also aim to engage with the OpenSearch community, exchange ideas, and learn how others are extending OSD to drive observability further.

Attendees will learn how OSD can be extended beyond visualisation to become a comprehensive observability platform. I’ll share strategies for enhancing query efficiency, scaling dashboards for large workloads, and integrating real-time traces and alerts—all within a single interface.

By showcasing how OSD can evolve into a full-stack monitoring solution, I hope to inspire teams to push its capabilities further. Highlighting these extensions will encourage more organisations to adopt OpenSearch for observability, leading to new innovations and a stronger ecosystem.

I hope to share our experience of managing petabyte-scale log data on OpenSearch, achieving 30% cost savings while ensuring performance and scalability. I’ll cover key strategies, challenges, and optimisations, including searchable snapshots, compression techniques, and index management to efficiently store and query massive datasets. I’m also looking forward to engaging with the OpenSearch community to exchange ideas and refine best practices for large-scale deployments.

Attendees will learn practical insights into migrating large workloads with no downtime, optimising storage through compression and index management, and balancing cost and performance using searchable snapshots. I’ll also share best practices for ensuring high availability across multiple availability zones and building strong disaster recovery strategies in OpenSearch.

By sharing real-world challenges and solutions, I aim to help organisations adopt OpenSearch confidently. Demonstrating its scalability and cost efficiency can encourage wider adoption while driving improvements in data management and operational resilience within the OpenSearch ecosystem.

In this talk, we’ll share Uber’s journey in scaling OpenSearch to handle large scale data across 100+ clusters while ensuring high availability, fault tolerance, performance and zero-downtime upgrades. We’ll dive deep into our strategies for overcoming regional and zonal failures, observability and fleet-upgrade strategy with zero-downtime, at scale. Attendees will learn about our approaches to sharding, indexing, and data distribution that enables resilience across a global infrastructure. Additionally, we’ll cover key best practices for managing OpenSearch clusters at Uber’s scale, ensuring reliability, and achieving cost-efficiency.

Query Insights delivers deep visibility into search query execution, offering detailed metrics and actionable recommendations to help users optimize their search operations. This feature analyzes query characteristics and execution patterns, providing clear insights into system behavior at each processing stage. Users can proactively identify potential performance bottlenecks, implement recommended optimizations, and maintain optimal query performance. These capabilities contribute to improved system efficiency, better user experience, and increased operational reliability.

Overview

- Why use RAG (Retrieval-Augmented Generation)?
- How does OpenSearch contribute to building intelligent agents?

Barriers to Deploying RAG Systems in Production:

- Key concerns: Quality, Cost, Safety, Maintenance, and Knowledge/Time.
- The biggest challenge remains quality.
- What leads to quality degradation in RAG systems, and how does it happen?

Solving the Issues:

- Identifying challenges with different chunking techniques: Which one should you use, or should you develop a custom approach tailored to your needs using OpenSearch?
- Addressing problems with search/querying techniques to optimize results.

By understanding these challenges and adopting the suggested solutions, you can significantly enhance the accuracy of RAG-based applications.

If needed, we can provide a deeper dive, a demo, or even hands-on workshops on this topic.

During our college days, my team and I embarked on an ambitious project—a Scheme Recommendation Chatbot to help users find relevant government schemes using intelligent search. As students new to Retrieval-Augmented Generation (RAG), we faced a big challenge: finding an open-source, scalable, and efficient search solution that didn’t lock us into proprietary or managed tools.

That’s when we discovered OpenSearch—and it completely changed the game!

In this talk, I’ll share:
1. Our journey to OpenSearch and why we made the switch.
2. How we leveraged OpenSearch’s vector search to improve search accuracy.
3. Challenges we faced as students implementing OpenSearch for the first time.
4. Lessons learned and tips for others looking to integrate OpenSearch into their projects.

Now, as a professional working in the industry, I look back at how OpenSearch not only powered our chatbot but also taught us real-world skills. If you’re a student, developer, or someone curious about open-source search and relevance, this talk is for you.

As OpenSearch continues to gain popularity for search and analytics, understanding its security landscape becomes crucial. This session introduces threat modeling for OpenSearch deployments, helping attendees identify potential vulnerabilities and implement effective mitigation strategies. We'll explore common threats, leverage the OpenSearch Security plugin, and discuss best practices for deploying opensearch clusters.

Vector search is a rapidly growing area within search and analytics, and the k-NN (k-Nearest Neighbors) plugin in OpenSearch offers multiple engines to support this capability, with FAISS (Facebook AI Similarity Search), which provides efficient similarity search and clustering of dense vectors, being a preferred choice.

To speed up vector operations, FAISS engine utilizes SIMD (Single Instruction Multiple Data) processing, leveraging AVX (Advanced Vector Extensions) instructions on x86 architecture. With the latest AVX-512 (512-bit wide vector processing) support in FAISS, OpenSearch achieves further performance improvements in vector indexing, quantization, and search operations.

This presentation will delve into key technical advancements, including an in-depth look at AVX, the impact of AVX-512 on vector search performance and real-world benefits for OpenSearch users. Additionally, we will discuss how these enhancements lead to better price-performance and reduced TCO (Total Cost of Ownership).

Search is no longer just a feature—it’s a critical capability that drives innovation, efficiency, and decision-making across enterprises. However, scaling OpenSearch across an organization presents challenges, from infrastructure management to governance and adoption. How do you enable diverse teams—from engineering to business functions—to harness the power of OpenSearch without deep operational overhead?

This session explores how to elevate OpenSearch from a tool to an internal platform offering, allowing teams to seamlessly integrate search into their workflows. By treating OpenSearch as a self-service, scalable, and well-governed enterprise platform, organizations can unlock its potential for everything from complex knowledge engines to simpler, ad-hoc use cases.

We’ll discuss key architectural principles, governance models, automation strategies, and best practices for building an OpenSearch-powered internal platform. Whether you're dealing with enterprise-wide search initiatives, analytics, or operational insights, this talk will provide actionable strategies to ensure OpenSearch adoption is efficient, scalable, and accessible to all teams—without friction.

GDPR. HIPAA. CCPA. Red tape? Not anymore. We’ll showcase how we transformed OpenSearch into a privacy-aware analytics engine using tokenization, dynamic data masking, and a custom plugin that auto-redacts PII before indexing. Ever seen a dashboard where every number is differentially private? You will. We’ll also debate the ethics of search in a post-Roe world and open-source our “privacy score” audit tool.

Warning: This talk may ruin your faith in anonymized data… but we’ll fix it.