# Text Embedding with DPR

*author: Kyle He*

<br />

## Description

This operator uses Dense Passage Retrieval (DPR) to convert long text to embeddings.

Dense Passage Retrieval (DPR) is a set of tools and models for state-of-the-art open-domain Q&A research.
It was introduced in Dense Passage Retrieval for Open-Domain Question Answering by Vladimir Karpukhin,
Barlas Oğuz, Sewon Min, Patrick Lewis, Ledell Wu, Sergey Edunov, Danqi Chen, Wen-tau Yih[1].

**DPR** models were proposed in "Dense Passage Retrieval for Open-Domain Question Answering"[2].

In this work, they show that retrieval can be practically implemented using dense representations alone,
where embeddings are learned from a small number of questions and passages by a simple dual-encoder framework[2].

### References

[1].https://huggingface.co/docs/transformers/model_doc/dpr

[2].https://arxiv.org/abs/2004.04906

<br />

## Code Example

Use the pre-trained model "facebook/dpr-ctx_encoder-single-nq-base"
to generate a text embedding for the sentence "Hello, world.". 

*Write the pipeline*:

```python
from towhee import pipe, ops, DataCollection

p = (
    pipe.input('text')
        .map('text', 'vec', ops.text_embedding.dpr(model_name='facebook/dpr-ctx_encoder-single-nq-base'))
        .output('text', 'vec')
)

DataCollection(p('Hello, world.')).show()
```

<img src="./result.png" width="800px"/>

<br />

## Factory Constructor

Create the operator via the following factory method:

***text_embedding.dpr(model_name="facebook/dpr-ctx_encoder-single-nq-base")***

**Parameters:**

***model_name***: *str*

The model name in string.
The default value is "facebook/dpr-ctx_encoder-single-nq-base".

Supported model names:
- facebook/dpr-ctx_encoder-single-nq-base
- facebook/dpr-ctx_encoder-multiset-base

<br />

## Interface

The operator takes a text in string as input.
It loads tokenizer and pre-trained model using model name
and then return text embedding in ndarray.

**Parameters:**

***text***: *str*

The text in string.


**Returns**:

*numpy.ndarray*

The text embedding extracted by model.

# More Resources

- [The guide to text-embedding-ada-002 model  | OpenAI](https://zilliz.com/ai-models/text-embedding-ada-002): text-embedding-ada-002: OpenAI's legacy text embedding model; average price/performance compared to text-embedding-3-large and text-embedding-3-small.
- [Sentence Transformers for Long-Form Text - Zilliz blog](https://zilliz.com/learn/Sentence-Transformers-for-Long-Form-Text): Deep diving into modern transformer-based embeddings for long-form text.
- [Building Open Source Chatbots with LangChain and Milvus in 5m - Zilliz blog](https://zilliz.com/blog/building-open-source-chatbot-using-milvus-and-langchain-in-5-minutes): A start-to-finish tutorial for RAG retrieval and question-answering chatbot on custom documents using Milvus, LangChain, and an open-source LLM.
- [Tutorial: Diving into Text Embedding Models | Zilliz Webinar](https://zilliz.com/event/tutorial-text-embedding-models): Register for a free webinar diving into text embedding models in a presentation and tutorial
- [Tutorial: Diving into Text Embedding Models | Zilliz Webinar](https://zilliz.com/event/tutorial-text-embedding-models/success): Register for a free webinar diving into text embedding models in a presentation and tutorial
- [The guide to text-embedding-3-small | OpenAI](https://zilliz.com/ai-models/text-embedding-3-small): text-embedding-3-small: OpenAI’s small text embedding model optimized for accuracy and efficiency with a lower cost.
- [Evaluating Your Embedding Model - Zilliz blog](https://zilliz.com/learn/evaluating-your-embedding-model): Review some practical examples to evaluate different text embedding models.
- [The guide to voyage-large-2 | Voyage AI](https://zilliz.com/ai-models/voyage-large-2): voyage-large-2: general-purpose text embedding model; optimized for retrieval quality; ideal for tasks like summarization, clustering, and classification.