Readme
Files and versions
Updated 6 months ago
LLM
Dolly Generation
author: Jael
Description
A LLM operator generates answer given prompt in messages using a large language model or service. This operator uses a pretrained Dolly to generate response. It will download model from HuggingFace Models.
Code Example
Use the default model to continue the conversation from given messages.
Write a pipeline with explicit inputs/outputs name specifications:
from towhee import pipe, ops
p = (
pipe.input('question', 'docs', 'history')
.map(('question', 'docs', 'history'), 'prompt', ops.prompt.question_answer(llm_name='dolly'))
.map('prompt', 'answer', ops.LLM.Dolly())
.output('answer')
)
history=[('Who won the world series in 2020?', 'The Los Angeles Dodgers won the World Series in 2020.')]
question = 'Where was it played?'
answer = p(question, [], history).get()[0]
Write a retrieval-augmented generation pipeline with explicit inputs/outputs name specifications:
from towhee import pipe, ops
temp = '''{question}
Input:
{context}
'''
docs = ['You can install Towhee via the command `pip install towhee`.']
history = [
('What is Towhee?', 'Towhee is an open-source machine learning project that helps you encode your unstructured data into embeddings.')
]
question = 'How to install it?'
p = (
pipe.input('question', 'docs', 'history')
.map(('question', 'docs', 'history'),
'prompt',
ops.prompt.template(temp, ['question', 'context']))
.map('prompt', 'answer',
ops.LLM.Dolly())
.output('answer')
)
answer = p(question, docs, history).get()[0]
Factory Constructor
Create the operator via the following factory method:
LLM.Dolly(model_name: str)
Parameters:
model_name: str
The model name in string, defaults to 'databricks/dolly-v2-12b'. Supported model names:
- databricks/dolly-v2-12b
- databricks/dolly-v2-7b
- databricks/dolly-v2-3b
- databricks/dolly-v1-6b
**kwargs
Other Dolly model parameters such as device_map.
Interface
The operator takes a piece of text in string as input. It returns answer in json.
__call__(txt)
Parameters:
messages: list
A list of messages to set up chat. Must be a list of dictionaries with key value from "system", "question", "answer". For example, [{"question": "a past question?", "answer": "a past answer."}, {"question": "current question?"}]
Returns:
answer: str
The answer generated.
More Resources
- Training Your Own Text Embedding Model - Zilliz blog: Explore how to train your text embedding model using the
sentence-transformers
library and generate our training data by leveraging a pre-trained LLM. - Boost your LLM with Private Data Using LlamaIndex | Zilliz Webinar: Zilliz webinar covering how to boost your LLM with private data with LlamaIndex to generate accurate and meaningful responses that reflect unique data inputs.
- LLama2 vs ChatGPT: How They Perform in Question Answering - Zilliz blog: What is Llama 2, and how does it perform in question answering compared to ChatGPT?
| 8 Commits | ||
---|---|---|---|
|
1.1 KiB
|
2 years ago | |
|
3.5 KiB
|
6 months ago | |
|
114 B
|
2 years ago | |
|
2.2 KiB
|
2 years ago | |
|
59 B
|
2 years ago |