OpenSearchCon India 2025

In an era where cyberattacks are growing in sophistication and scale, traditional security tools are no longer sufficient. AI-driven threat detection is enhancing how organizations safeguard their digital assets. This session will showcase how OpenSearch, combined with cutting-edge AI and machine learning, can transform raw log data into actionable security insights.

Discover how OpenSearch’s ML-powered analytics can:
* Automatically detect anomalous behavior and insider threats in real-time.
* Identify patterns indicative of fraud, ransomware, and zero-day attacks.
* Enable predictive threat hunting by correlating data across multiple sources.

We’ll also explore trending advancements, such as integrating Generative AI for threat intelligence enrichment and leveraging vector search to identify similar attack patterns across datasets. Attendees will walk away with practical strategies to implement AI-driven security analytics, reduce false positives, and enhance their organization’s cyber resilience in an increasingly hostile digital landscape.

Learn with examples the variety of OpenSearch driven - Vector, Neural, Conversational, Semantic and Multimodal Search and the difference to cost , resiliency and response time in a Managed Vs Self run Cluster. Lessons, Challenges and Solutions would enable you to architect and run it successfully to be the data exhaust fingerprint for your product.

In high-performance computing (HPC) environments, the need for robust observability mechanisms is critical to ensure optimal performance, fault tolerance, and resource utilization. Airgapped systems, often used in sensitive or high-security domains, require innovative solutions to capture, analyze, and visualize system metrics, application performance, and infrastructure health without compromising security.

Challenges arise in airgapped HPC environments, where network isolation adds complexity to monitoring and diagnostics. I explore methodologies for establishing observability in airgapped HPC environments, focusing on the unique challenges of data collection, real-time monitoring, and offline analytics. Additionally, I discuss the role of automated anomaly detection, resource optimization, and performance tuning in maintaining the efficiency and reliability of these systems. The findings highlight the importance of maintaining a balance between high security and comprehensive observability, paving the way for more resilient and performant airgapped HPC environments.

Explore how Generative AI enhances OpenSearch by improving search relevance, personalization, and contextual understanding. This session covers implementing vector search, integrating LLMs for smarter query processing, and real-world use cases of AI-powered search applications. Learn how to bring next-gen AI capabilities to OpenSearch for better user experiences.

OpenSearch is a powerful open-source search and analytics engine, but running it effectively on Kubernetes requires a well-structured approach. In this session, I will explore how to deploy, configure, and operate OpenSearch in a Kubernetes-native way. I will dive into best practices for maintaining OpenSearch clusters, optimizing performance, and handling scaling challenges.

Additionally, I will cover how to ship Kubernetes logs to OpenSearch, enabling centralized log management for a single cluster and also across multiple clusters. I will discuss scalable log ingestion strategies involving off the shelf as well as custom made tools. The session will also include practical demonstrations on building insightful dashboards for observability and setting up alerts to proactively monitor system health.

This talk will provide a comprehensive guide to managing OpenSearch effectively within Kubernetes.

As enterprises grapple with unstructured data, Retrieval-Augmented Generation (RAG) is transforming search from simple retrieval to context-aware AI-driven knowledge generation.

This session explores how DeepSeek-R1, a powerful reasoning model, integrates with Amazon OpenSearch Service’s vector database to enable scalable, intelligent information retrieval.

Attendees will learn:
How RAG enhances traditional search using vector embeddings and semantic understanding.
How to connect DeepSeek-R1 on SageMaker with OpenSearch for AI-powered responses.
How OpenSearch’s flexible connectors integrate with models like DeepSeek, Cohere, and OpenAI.

By the end of this session, you'll gain actionable insights on implementing RAG workflows in OpenSearch, improving search accuracy, and advancing the AI ecosystem with knowledge-augmented AI.

Learn the story of OpenSearch, and how we evolved the OpenSearch snapshot process to support sub-minutely granularity at Petabyte scale without sacrificing on cost-effectiveness, performance and availability. We'll share our journey of establishing core principles and breaking down complex challenges into manageable deliverables.

We will start by talking about how snapshots work in the OpenSearch world. We will cover some of the key design decisions that helped us to allow backward compatibility with existing snapshots, without major overhaul of the system. We will walk through our open-source first journey, illuminating common pitfalls and lessons learnt in scaling, correctness and performance, which helped shape the eventual solution.
We will lastly cover step by step as to how users can get started on enabling the new sub-minutely snapshot mechanism for their workload to achieve high data protection for their use cases

In modern big data ecosystems like Databricks, logs from different jobs, stages, and components can be highly diverse, making it difficult to gain comprehensive insights into system health, performance, and operational flow. This presentation will explore how to consolidate these disparate logs into a unified OpenSearch platform, enabling advanced log analysis and real-time visibility. By leveraging large language models (LLMs) and contextual dashboards, we will demonstrate how to create a "single pane of glass" for monitoring entire operational workflows. Attendees will learn how this approach improves visibility, traceability, and transparency, enabling quicker actions and decisions based on knowledge-driven insights. Key topics include log mapping, lineage tracking, and using OpenSearch to facilitate both broad and granular insights for operational teams. By the end of this session, you'll understand how to streamline complex log data from systems like Databricks, empowering your teams with smarter, faster decision-making and more efficient troubleshooting.

Hybrid search has become a powerful method for enhancing result quality. However, tuning hybrid search is a complex optimization problem due to the need to balance lexical and semantic relevance while adapting to diverse query intents.

In this talk, we'll present a scalable, data-driven framework for optimizing hybrid search using machine learning techniques, focusing on real-time query adaptation to maximize relevance. We’ll explore how we transitioned from a static one-size-fits-all approach to query-specific optimization, using machine learning to dynamically adjust search parameters.

Attendees will learn:
* How to address the challenges of hybrid search optimization.
* Techniques to predict search parameters using query and query result-specific signals.
* Improvements in search quality metrics (DCG, NDCG, Precision) of more than 7% compared to the static approach.
* Steps to integrate query-specific optimization into your existing search infrastructure.

After this session, attendees will be equipped to build adaptable, machine learning-driven hybrid search systems that respond dynamically to user queries, resulting in measurable gains in search relevance and user satisfaction.

At Uber, OpenSearch is a core component of our search infrastructure. We manage ~100 clusters across several thousands of nodes. We platformized OpenSearch to ensure scalability and reliability for Uber’s search needs.

This talk covers:

Automated Build & Deployment: Uber maintains a custom OpenSearch fork with performance optimizations, scalability improvements, and bug fixes. We’ll discuss our build and release pipelines for deploying OpenSearch at scale.

Batch & Streaming Ingestion: We built batch ingestion using Spark and OpenSearch-Hadoop and real-time streaming with Kafka/Flink, scaling to ingest a billion vectors in X hours.

Elasticsearch to OpenSearch Migration: Insights into compatibility challenges, query performance tuning, and scaling ingestion throughput.

Security: Implementing fine-grained authentication and authorization for multi-tenant OpenSearch clusters.

Community Contributions: How we contribute optimizations, bug fixes, and features upstream to improve OpenSearch for everyone.

Explore how OpenSearch enhances AI applications through vector stores, similarity search, and retrieval-augmented generation (RAG). Learn to efficiently store and retrieve high-dimensional data, improve recommendation systems with similarity search, and integrate robust indexing for context-aware responses using RAG. This session provides practical insights and examples for developers, data scientists, and AI enthusiasts to optimize their AI solutions with OpenSearch. Join us to elevate your AI capabilities with advanced search technologies.