ElasticSearch Index

author: Jael

Description

The index operator index the given documents in ElasticSearch to get ready for retrieval. It accepts a single document in dictionary or a list of documents (dictionaries) as input. For each document, the index automatically generates a unique id. To use this operator, you need to set up ElasticSearch in advance.

Code Example

Insert an example document into ElasticSearch with address of localhost:9200 and index of 'test_index'.

from datetime import datetime
from towhee import pipe, ops, DataCollection


example_doc = {
    'title': 'Test Title',
    'author': 'Towhee',
    'content': 'This is an example.',
    'timestamp': datetime.now()
    }

es_insert = (
    pipe.input('doc')
        .map('doc', 'res', ops.elasticsearch.index_client(
            host='localhost', port=9200, index_name='test_index'
        ))
        .output('doc', 'res')
)

res = es_insert(example_doc) # OR: es_insert([example_doc])
DataCollection(res).show() # Optional: display output data

Factory Constructor

Create the operator via the following factory method:

elasticsearch.search(host='localhost', port=9200, index_name='test_index', user=None, password=None, ca_certs=None)

Parameters:

host: str

The host to connect ElasticSearch client.

port: int

The port to connect ElasticSearch client.

user: str

The username to connect ElasticSearch client if needed, defaults to None.

password: str

The user password to connect ElasticSearch client if needed, defaults to None.

ca_certs: str

The path to CA certificates to connect ElasticSearch client if needed, defaults to None.

Returns:

Index response wrapped by elastic_transport.ObjectApiResponse.

More Resources

• Milvus support for multiple Index types - Zilliz blog: One of the essential features of Milvus is the support for various Index types; Indexes help to optimize data querying and retrieval.
• Everything You Need to Know about Vector Index Basics - Zilliz blog: This tutorial analyzes the components of a modern indexer before going over two simplest indexing strategies - flat indexing and inverted file (IVF) indexes.
• What is Information Retrieval? A Comprehensive Guide. - Zilliz blog: Information retrieval (IR) is the process of efficiently retrieving relevant information from large collections of unstructured or semi-structured data.
• Accelerating Similarity Search on Big Data with Vector Indexing - Zilliz blog: Discover how indexes dramatically accelerate vector similarity search, different index types, and how to choose the right index for your next ML application.
• Accelerate Similarity Search on Big Data with Vector Indexing - II - Zilliz blog: Discover how indexes dramatically accelerate vector similarity search, different types of indexes, and how to choose the right index for your next AI application.
• Simplifying Legal Research with RAG, Milvus, and Ollama - Zilliz blog: In this blog post, we will see how we can apply RAG to Legal data. Legal research can be time-consuming. You usually need to review a large number of documents to find the answers you need. Retrieval-Augmented Generation (RAG) can help streamline your research process.
• How to Pick a Vector Index in Your Milvus Instance: A Visual Guide - Zilliz blog: In this post, we'll explore several vector indexing strategies that can be used to efficiently perform similarity search, even in scenarios where we have large amounts of data and multiple constraints to consider.
• Choosing the Right Vector Index for Your Project - Zilliz blog: Understanding in-memory vector search algorithms, indexing strategies, and guidelines on choosing the right vector index for your project.