# Image Captioning with ClipCap
*author: David Wang*
< br / >
## Description
This operator generates the caption with [ClipCap ](https://arxiv.org/abs/2111.09734 ) which describes the content of the given image. ClipCap uses CLIP encoding as a prefix to the caption, by employing a simple mapping network, and then fine-tunes a language model to generate the image captions. This is an adaptation from [rmokady/CLIP_prefix_caption ](https://github.com/rmokady/CLIP_prefix_caption ).
< br / >
## Code Example
Load an image from path './hulk.jpg' to generate the caption.
*Write the pipeline in simplified style* :
```python
import towhee
towhee.glob('./hulk.jpg') \
.image_decode() \
.image_captioning.clipcap(model_name='clipcap_coco') \
.show()
```
< img src = "./cap.png" alt = "result1" style = "height:20px;" / >
*Write a same pipeline with explicit inputs/outputs name specifications:*
```python
import towhee
towhee.glob['path']('./hulk.jpg') \
.image_decode['path', 'img']() \
.image_captioning.clipcap['img', 'text'](model_name='clipcap_coco') \
.select['img', 'text']() \
.show()
```
< img src = "./tabular.png" alt = "result2" style = "height:60px;" / >
< br / >
## Factory Constructor
Create the operator via the following factory method
***clipcap(model_name)***
**Parameters:**
** *model_name:*** *str*
The model name of ClipCap. Supported model names:
- clipcap_coco
- clipcap_conceptual
< br / >
## Interface
An image-text embedding operator takes a [towhee image ](link/to/towhee/image/api/doc ) as input and generate the correspoing caption.
**Parameters:**
** *data:*** *towhee.types.Image (a sub-class of numpy.ndarray)*
The image to generate embedding.
**Returns:** *str*
The caption generated by model.
