logo
You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
Readme
Files and versions

76 lines
1.3 KiB

# Image Captioning with BLIP
*author: David Wang*
<br />
## Description
This operator generates the caption with [BLIP](https://arxiv.org/abs/2201.12086) which describes the content of the given image. This is an adaptation from [salesforce/BLIP](https://github.com/salesforce/BLIP).
<br />
## Code Example
Load an image from path './animals.jpg' to generate the caption.
*Write a pipeline with explicit inputs/outputs name specifications:*
```python
from towhee.dc2 import pipe, ops, DataCollection
p = (
pipe.input('url')
.map('url', 'img', ops.image_decode.cv2_rgb())
.map('img', 'text', ops.image_captioning.blip(model_name='blip_base'))
.output('img', 'text')
)
DataCollection(p('./animals.jpg')).show()
```
<img src="./tabular.png" alt="result2" style="height:60px;"/>
<br />
## Factory Constructor
Create the operator via the following factory method
***blip(model_name)***
**Parameters:**
***model_name:*** *str*
​ The model name of BLIP. Supported model names:
- blip_base
<br />
## Interface
An image captioning operator takes a [towhee image](link/to/towhee/image/api/doc) as input and generate the correspoing caption.
**Parameters:**
***img:*** *towhee.types.Image (a sub-class of numpy.ndarray)*
​ The image to generate caption.
**Returns:** *str*
2 years ago
​ The caption generated by model.