osschat-index/README.md

ElasticSearch Index

Description

The index operator index the given documents in ElasticSearch to get ready for retrieval. It accepts a single document in dictionary or a list of documents (dictionaries) as input. To use this operator, you need to set up ElasticSearch in advance.

Code Example

Insert an example document into ElasticSearch with address of localhost:9200 and index of 'test_index'.

from towhee import pipe, ops, DataCollection


example_doc = {
    'sentence': 'This is an example.',
    }

es_insert = (
    pipe.input('index_name', 'doc')
        .map(('index_name', 'doc'), 'res', ops.elasticsearch.osschat_index(
            host='localhost', port=9200
        ))
        .map('doc', 'doc', lambda x: str(x))
        .output('doc', 'res')
)

res = es_insert('test_index_5', example_doc)
DataCollection(res).show()

Factory Constructor

Create the operator via the following factory method:

elasticsearch.search(host='localhost', port=9200, user=None, password=None, ca_certs=None)

Parameters:

host: str

The host to connect ElasticSearch client.

port: int

The port to connect ElasticSearch client.

user: str

The username to connect ElasticSearch client if needed, defaults to None.

password: str

The user password to connect ElasticSearch client if needed, defaults to None.

ca_certs: str

The path to CA certificates to connect ElasticSearch client if needed, defaults to None.

Returns:

Index response wrapped by elastic_transport.ObjectApiResponse.

More Resources

• Milvus support for multiple Index types - Zilliz blog: One of the essential features of Milvus is the support for various Index types; Indexes help to optimize data querying and retrieval.
• Everything You Need to Know about Vector Index Basics - Zilliz blog: This tutorial analyzes the components of a modern indexer before going over two simplest indexing strategies - flat indexing and inverted file (IVF) indexes.
• What is Information Retrieval? A Comprehensive Guide. - Zilliz blog: Information retrieval (IR) is the process of efficiently retrieving relevant information from large collections of unstructured or semi-structured data.
• Accelerating Similarity Search on Big Data with Vector Indexing - Zilliz blog: Discover how indexes dramatically accelerate vector similarity search, different index types, and how to choose the right index for your next ML application.
• Accelerate Similarity Search on Big Data with Vector Indexing - II - Zilliz blog: Discover how indexes dramatically accelerate vector similarity search, different types of indexes, and how to choose the right index for your next AI application.
• Mastering BM25: A Deep Dive into the Algorithm and Its Application in Milvus - Zilliz blog: We can easily implement the BM25 algorithm to turn a document and a query into a sparse vector with Milvus. Then, these sparse vectors can be used for vector search to find the most relevant documents according to a specific query.
• How to Pick a Vector Index in Your Milvus Instance: A Visual Guide - Zilliz blog: In this post, we'll explore several vector indexing strategies that can be used to efficiently perform similarity search, even in scenarios where we have large amounts of data and multiple constraints to consider.
• Choosing the Right Vector Index for Your Project - Zilliz blog: Understanding in-memory vector search algorithms, indexing strategies, and guidelines on choosing the right vector index for your project.