ANN Search Operator: Faiss

author: shiyu

Desription

Only for local test. If you want to use a vector database in a production environment, you can use Milvus(https://github.com/milvus-io/milvus).

Code Example

please insert data into faiss first.

Example

Write a same pipeline with explicit inputs/outputs name specifications:

from towhee import pipe, ops

p = pipe.input('vec')  \
    .flat_map('vec', 'rows', ops.ann_search.faiss('./data_dir', 5))  \
    .map('rows', ('id', 'score'), lambda x: (x[0], x[1]))  \
    .output('id', 'score')

p(<your-vector>)

Factory Constructor

Create the operator via the following factory method:

ops.ann_search.faiss_index('./data_dir', 5)

Parameters:

data_dir: str

The path to faiss index and meta data.

Interface

Parameters:

query: ndarray

Query embedding in Faiss

Returns: Entity

Return the results in Faiss with key and score.

More Resources

• Zilliz Triumphed in Billion-Scale ANN Search Challenge of NeurIPS 2021 - Zilliz Newsroom; Zilliz Triumphed in Billion-Scale ANN Search Challenge of NeurIPS 2021: Zilliz team has won the first place in the Disk-based ANN Search track in NeurIPS 2021. The performance of BBAnn developed by Zilliz research team peaked during the search in the SimSearchNet++ dataset.
• What is Faiss (Facebook AI Similarity Search)? - Zilliz blog: Faiss (Facebook AI similarity search) is an open-source library for efficient similarity search of unstructured data and clustering of dense vectors.
• What is approximate nearest neighbor search (ANNS)?: Learn how to use Approximate nearest neighbor search (ANNS) for efficient nearest-neighbor search in large datasets.
• Setting Up With Facebook AI Similarity Search (FAISS) - Zilliz blog: Here's your FAISS tutorial that helps you set up FAISS, get it up and running, and demonstrate its power through a sample search program.