Dolly Generation

author: Jael

Description

A LLM operator generates answer given prompt in messages using a large language model or service. This operator uses a pretrained Dolly to generate response. It will download model from HuggingFace Models.

Code Example

Use the default model to continue the conversation from given messages.

Write a pipeline with explicit inputs/outputs name specifications:

from towhee import pipe, ops

p = (
    pipe.input('question', 'docs', 'history')
        .map(('question', 'docs', 'history'), 'prompt', ops.prompt.question_answer(llm_name='dolly'))
        .map('prompt', 'answer', ops.LLM.Dolly())
        .output('answer')
)

history=[('Who won the world series in 2020?', 'The Los Angeles Dodgers won the World Series in 2020.')]
question = 'Where was it played?'
answer = p(question, [], history).get()[0]

Write a retrieval-augmented generation pipeline with explicit inputs/outputs name specifications:

from towhee import pipe, ops


temp = '''{question}

Input:
{context}
'''


docs = ['You can install Towhee via the command `pip install towhee`.']
history = [
    ('What is Towhee?', 'Towhee is an open-source machine learning project that helps you encode your unstructured data into embeddings.')
]
question = 'How to install it?'

p = (
    pipe.input('question', 'docs', 'history')
        .map(('question', 'docs', 'history'),
             'prompt',
             ops.prompt.template(temp, ['question', 'context']))
        .map('prompt', 'answer',
             ops.LLM.Dolly())
        .output('answer')
)

answer = p(question, docs, history).get()[0]

Factory Constructor

Create the operator via the following factory method:

LLM.Dolly(model_name: str)

Parameters:

model_name: str

The model name in string, defaults to 'databricks/dolly-v2-12b'. Supported model names:

• databricks/dolly-v2-12b
• databricks/dolly-v2-7b
• databricks/dolly-v2-3b
• databricks/dolly-v1-6b

**kwargs

Other Dolly model parameters such as device_map.

Interface

The operator takes a piece of text in string as input. It returns answer in json.

__call__(txt)

Parameters:

messages: list

​ A list of messages to set up chat. Must be a list of dictionaries with key value from "system", "question", "answer". For example, [{"question": "a past question?", "answer": "a past answer."}, {"question": "current question?"}]

Returns:

answer: str

​ The answer generated.

More Resources

• Training Your Own Text Embedding Model - Zilliz blog: Explore how to train your text embedding model using the sentence-transformers library and generate our training data by leveraging a pre-trained LLM.
• Boost your LLM with Private Data Using LlamaIndex | Zilliz Webinar: Zilliz webinar covering how to boost your LLM with private data with LlamaIndex to generate accurate and meaningful responses that reflect unique data inputs.
• LLama2 vs ChatGPT: How They Perform in Question Answering - Zilliz blog: What is Llama 2, and how does it perform in question answering compared to ChatGPT?