|
|
|
# ANN Search Operator: Faiss
|
|
|
|
|
|
|
|
*author: shiyu*
|
|
|
|
|
|
|
|
<br />
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
## Desription
|
|
|
|
|
|
|
|
Search embedding in [Faiss](https://github.com/facebookresearch/faiss), **please make sure you have inserted data to Faiss before search**.
|
|
|
|
|
|
|
|
<br />
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
## Code Example
|
|
|
|
|
|
|
|
### Insert data into Faiss first
|
|
|
|
|
|
|
|
```python
|
|
|
|
import numpy as np
|
|
|
|
import towhee
|
|
|
|
|
|
|
|
vec = np.random.random((10, 100)).astype('float32')
|
|
|
|
ids = list(i for i in range(10))
|
|
|
|
|
|
|
|
x = towhee.dc['id'](ids) \
|
|
|
|
.runas_op['id', 'vec'](func=lambda x: vec[x]) \
|
|
|
|
.to_faiss['id', 'vec'](findex='index.bin')
|
|
|
|
```
|
|
|
|
|
|
|
|
### Example
|
|
|
|
|
|
|
|
*Write the pipeline in simplified style:*
|
|
|
|
|
|
|
|
```python
|
|
|
|
query = vec[0:2]
|
|
|
|
towhee.dc(query) \
|
|
|
|
.ann_search.faiss(findex='index.bin')
|
|
|
|
```
|
|
|
|
|
|
|
|
*Write a same pipeline with explicit inputs/outputs name specifications:*
|
|
|
|
|
|
|
|
```python
|
|
|
|
query = vec[0:2]
|
|
|
|
towhee.dc['vec'](query) \
|
|
|
|
.ann_search.faiss['vec', 'results'](findex='index.bin') \
|
|
|
|
.show()
|
|
|
|
```
|
|
|
|
|
|
|
|
<img src="./result.png" height="100px"/>
|
|
|
|
|
|
|
|
<br />
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
## Factory Constructor
|
|
|
|
|
|
|
|
Create the operator via the following factory method:
|
|
|
|
|
|
|
|
***ann_search.faiss(findex)***
|
|
|
|
|
|
|
|
<br />
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
**Parameters:**
|
|
|
|
|
|
|
|
***findex:*** *str* or *faiss.INDEX*
|
|
|
|
|
|
|
|
The path to faiss index file or faiss index.
|
|
|
|
|
|
|
|
|
|
|
|
<br />
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
## Interface
|
|
|
|
|
|
|
|
**Parameters:**
|
|
|
|
|
|
|
|
***query:*** *list*
|
|
|
|
|
|
|
|
Query embeddings in Faiss
|
|
|
|
|
|
|
|
<br />
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
**Returns:** *Entity*
|
|
|
|
|
|
|
|
Return the results in Faiss with `key` and `score`.
|