opus-mt/README.md

# Machine Translation with Opus-MT

*author: David Wang*

<br />

## Description

A machine translation operator translates a sentence, paragraph, or document from source language 
to the target language. This operator is trained on [OPUS](https://opus.nlpl.eu/) data by Helsinki-NLP. 
More detail can be found in [ Helsinki-NLP/Opus-MT ](https://github.com/Helsinki-NLP/Opus-MT). 

<br />

## Code Example

Use the pre-trained model 'opus-mt-en-zh'
to generate the Chinese translation for the sentence "Hello, world.".

*Write a pipeline with explicit inputs/outputs name specifications:*

```python
from towhee import pipe, ops, DataCollection

p = (
    pipe.input('text')
    .map('text', 'translation', ops.machine_translation.opus_mt(model_name='opus-mt-en-zh')) 
    .output('text', 'translation')
)

DataCollection(p('hello, world.')).show()

```

<img src="./result.png" width="800px"/>

<br />

## Factory Constructor

Create the operator via the following factory method:

***machine_translation.opus_mt(model_name="opus-mt-en-zh")***

**Parameters:**

***model_name***: *str*

The model name in string.
The default model name is "opus-mt-en-zh".

Supported model names:
 - opus-mt-en-zh  
 - opus-mt-zh-en
 - opus-tatoeba-en-ja
 - opus-tatoeba-ja-en
 - opus-mt-ru-en
 - opus-mt-en-ru

<br />

## Interface

The operator takes a piece of text in string as input.
It loads tokenizer and pre-trained model using model name.
and then return translated text in string.

***__call__(text)***

**Parameters:**

***text***: *str*

​	The source language text in string.

**Returns**:

*str*

​	The target language text.


# More Resources

- [What is a Transformer Model? An Engineer's Guide](https://zilliz.com/glossary/transformer-models): A transformer model is a neural network architecture. It's proficient in converting a particular type of input into a distinct output. Its core strength lies in its ability to handle inputs and outputs of different sequence length. It does this through encoding the input into a matrix with predefined dimensions and then combining that with another attention matrix to decode. This transformation unfolds through a sequence of collaborative layers, which deconstruct words into their corresponding numerical representations. 

At its heart, a transformer model is a bridge between disparate linguistic structures, employing sophisticated neural network configurations to decode and manipulate human language input. An example of a transformer model is GPT-3, which ingests human language and generates text output.
- [Experiment with 5 Chunking Strategies via LangChain for LLM  - Zilliz blog](https://zilliz.com/blog/experimenting-with-different-chunking-strategies-via-langchain): Explore the complexities of text chunking in retrieval augmented generation applications and learn how different chunking strategies impact the same piece of data.
- [Massive Text Embedding Benchmark (MTEB)](https://zilliz.com/glossary/massive-text-embedding-benchmark-(mteb)): A standardized way to evaluate text embedding models across a range of tasks and languages, leading to better text embedding models for your app
- [About Lance Martin | Zilliz](https://zilliz.com/authors/Lance_Martin): Software / ML at LangChain
- [The guide to gte-base-en-v1.5 | Alibaba](https://zilliz.com/ai-models/gte-base-en-v1.5): gte-base-en-v1.5: specialized for English text; Built upon the transformer++ encoder backbone (BERT + RoPE + GLU)
- [What Is Semantic Search?](https://zilliz.com/glossary/semantic-search): Semantic search is a search technique that uses natural language processing (NLP) and machine learning (ML) to understand the context and meaning behind a user's search query.
- [What are LLMs? Unlocking the Secrets of GPT-4.0 and Large Language Models - Zilliz blog](https://zilliz.com/learn/what-are-llms-unlock-secrets-of-gpt-4-and-llms): Unlocking the Secrets of GPT-4.0 and Large Language Models
- [Large Language Models (LLMs)](https://zilliz.com/glossary/large-language-models-(llms)): What Is a Large Language Model? A Developer's Reference