OpenSearchCon India 2025

Retrieval-Augmented Generation (RAG) has emerged as a powerful paradigm for grounding large language models (LLMs) in external knowledge, enabling more accurate and contextually relevant responses. However, traditional RAG pipelines often operate in a static, pre-defined manner, limiting their ability to adapt to complex queries and dynamically explore information spaces. This talk introduces "Multimodal Agentic RAG", an innovative approach that integrates autonomous agents into the RAG framework to enhance its reasoning and retrieval capabilities. This talk will delve into the architecture and implementation of multimodal RAG with agentic workflows, showcasing its potential to address limitations of traditional RAG. The architecture is based on OpenSearch's multimodal capabilities, DeepSeek-R1's reasoning capabilities, and custom-tuned embedding models. Furthermore, we will discuss the challenges and future directions of Agentic RAG, including the development of robust planning mechanisms, efficient knowledge source management, and effective agent coordination.

With the rise of Agentic AI, search is no longer just about retrieving information—it’s about enabling AI-driven reasoning, contextual awareness, and decision-making. As AI applications become more autonomous, the need for efficient, scalable, and intelligent search platforms has never been greater.

This session explores how OpenSearch can be leveraged as a foundational platform for Agentic AI applications, enabling capabilities like semantic search, vector-based retrieval, hybrid search, and retrieval-augmented generation (RAG). We’ll discuss architectural patterns, indexing strategies, multi-modal search, and real-time data enrichment, ensuring AI agents can access and utilize knowledge effectively.

By adopting a platform-centric approach, organizations can standardize and scale OpenSearch to serve diverse AI-driven use cases—from chatbots and virtual assistants to complex enterprise automation systems. Join us to discover how search is evolving into a key enabler for the next generation of AI applications.

HPCM (HPE Performance Cluster Manager) is a suite for monitoring HPC systems, consisting of rack, chassis, compute nodes(GPU, CPU), power,cooling components. OpenSearch, deployed in a distributed manner, enables efficient indexing and querying of large volumes of data. For real-time monitoring, events across the system is collected and streamed to Kafka, then persisted in OpenSearch via Logstash, allowing for efficient log management. OpenSearch Dashboards, integrated with Fluent Bit, provide real-time log observability, enabling users to visualize and analyze system logs for insightful cluster monitoring. We have developed a user-friendly wrapper around OpenSearch Alerting that uses custom YAML-based configurations to define alerting rules, categorize alerts by severity and group, and trigger notifications to endpoints like email,alertmanager. Argonne National Laboratory's Aurora, the third-fastest supercomputer in the world, uses 21,248 CPUs and 63,744 GPUs across 166 racks. HPCM, along with its 10-instance OpenSearch database deployed on 10 nodes, successfully manages and monitors events and logs, with OpenSearch Alerting effectively deployed to track critical system issues.

Struggling to tame the latency of your OpenSearch query? Atlassian was facing the exact problem with a query which had grown organically over a number of years of relevance improvements. Learn how they dove into the internals of OpenSearch to discover how their query worked, reducing their query latency by 70% in the process.

This session will cover:

* Techniques for identifying problem areas
* Shard topology
* Conjunctive and disjunctive boolean queries and how they impact latency
* track_total_hits, how it works and subtleties of when it doesn't
* Concurrent segment search
* Joins, when denormalisation is faster and when it isn't
* Index sorting
* initial query scoring vs rescoring
* Cross field match vs copy_to
* Parallel recall paths using hybrid queries or multi-search
* Query cache vs pre-computing
* LTR plugin, feature logging and ranking and impact on latency

Walk through our journey of creating a developer-centric documentation search platform that understands code snippets, technical context, and user intent. Learn how we enhanced OpenSearch's capabilities with custom analyzers for programming languages, implemented semantic code search, and built an intelligent query understanding system. We'll share our unique approach to ranking technical content and how we achieved a 40% improvement in search success rates for our developer community.

In modern applications, observability is essential. OpenTelemetry (OTel) has emerged as the standard for telemetry data, providing a unified way to collect, process, and export logs, metrics, and traces. But organizations also need a scalable way to store, search, and analyze this data for actionable insights.

This is where OpenSearch comes in. As an open-source search and analytics engine, it provides a robust observability backend, enabling real-time indexing, querying, and visualization of logs and metrics. Its distributed architecture ensures scalability and high availability, making it ideal for modern observability stacks.

This session is for developers looking to deepen their understanding of OpenTelemetry's architecture and key pillars. We will explore how OpenSearch seamlessly integrates into an OpenTelemetry-based observability stack, playing a key role in logging, metrics storage, and real-time analytics.

Audience Takeaway: A clear understanding of OpenTelemetry’s core components, how OpenSearch enhances observability, and its role in logging and metrics.

What if OpenSearch could manage itself? This session explores how AI, automation, and cloud-native tools are transforming OpenSearch operations. Learn how Kubernetes, Terraform, and AI-driven monitoring enable self-healing clusters, automated scaling, and hands-free management—bringing the NoOps vision to search and analytics infrastructure

I hope to share how we extended OpenSearch Dashboards (OSD) beyond traditional log analytics to create a single pane of glass for tracing, profiling, and alert management. I’ll discuss how we optimised query performance to handle massive log volumes while keeping dashboards responsive. By presenting our journey, I also aim to engage with the OpenSearch community, exchange ideas, and learn how others are extending OSD to drive observability further.

Attendees will learn how OSD can be extended beyond visualisation to become a comprehensive observability platform. I’ll share strategies for enhancing query efficiency, scaling dashboards for large workloads, and integrating real-time traces and alerts—all within a single interface.

By showcasing how OSD can evolve into a full-stack monitoring solution, I hope to inspire teams to push its capabilities further. Highlighting these extensions will encourage more organisations to adopt OpenSearch for observability, leading to new innovations and a stronger ecosystem.

I hope to share our experience of managing petabyte-scale log data on OpenSearch, achieving 30% cost savings while ensuring performance and scalability. I’ll cover key strategies, challenges, and optimisations, including searchable snapshots, compression techniques, and index management to efficiently store and query massive datasets. I’m also looking forward to engaging with the OpenSearch community to exchange ideas and refine best practices for large-scale deployments.

Attendees will learn practical insights into migrating large workloads with no downtime, optimising storage through compression and index management, and balancing cost and performance using searchable snapshots. I’ll also share best practices for ensuring high availability across multiple availability zones and building strong disaster recovery strategies in OpenSearch.

By sharing real-world challenges and solutions, I aim to help organisations adopt OpenSearch confidently. Demonstrating its scalability and cost efficiency can encourage wider adoption while driving improvements in data management and operational resilience within the OpenSearch ecosystem.