logo
Browse Source

update the doc.

Signed-off-by: wxywb <xy.wang@zilliz.com>
main
wxywb 2 years ago
parent
commit
27c734e5ab
  1. 27
      README.md
  2. BIN
      cap.png
  3. 2
      capdec.py
  4. BIN
      tabular.png

27
README.md

@ -18,28 +18,19 @@ This operator generates the caption with [CapDec](https://arxiv.org/abs/2211.005
Load an image from path './image.jpg' to generate the caption. Load an image from path './image.jpg' to generate the caption.
*Write the pipeline in simplified style*:
*Write a pipeline with explicit inputs/outputs name specifications:*
```python ```python
import towhee
from towhee.dc2 import pipe, ops, DataCollection
towhee.glob('./image.jpg') \
.image_decode() \
.image_captioning.capdec(model_name='capdec_noise_0') \
.show()
```
<img src="./cap.png" alt="result1" style="height:20px;"/>
*Write a same pipeline with explicit inputs/outputs name specifications:*
```python
import towhee
p = (
pipe.input('url')
.map('url', 'img', ops.image_decode.cv2_rgb())
.map('img', 'text', ops.image_captioning.capdec(model_name='capdec_noise_0')
.output('img', 'text')
)
towhee.glob['path']('./image.jpg') \
.image_decode['path', 'img']() \
.image_captioning.capdec['img', 'text'](model_name='capdec_noise_0') \
.select['img', 'text']() \
.show()
DataCollection(p('./image.jpg')).show()
``` ```
<img src="./tabular.png" alt="result2" style="height:60px;"/> <img src="./tabular.png" alt="result2" style="height:60px;"/>

BIN
cap.png

Binary file not shown.

After

Width:  |  Height:  |  Size: 14 KiB

2
capdec.py

@ -55,7 +55,7 @@ class Capdec(NNOperator):
self.clip_model = clip.create_model(model_name='clip_resnet_r50x4', pretrained=True, jit=True) self.clip_model = clip.create_model(model_name='clip_resnet_r50x4', pretrained=True, jit=True)
self.clip_model.to(self.device) self.clip_model.to(self.device)
self.clip_tfms = clip.get_transforms(model_name='clip_resnet_r50x4') self.clip_tfms = clip.get_transforms(model_name='clip_resnet_r50x4')
self.tokenizer = GPT2Tokenizer.from_pretrained("gpt2").to(self.device)
self.tokenizer = GPT2Tokenizer.from_pretrained("gpt2")
self.generate_beam = generate_beam self.generate_beam = generate_beam
self.generate2 = generate2 self.generate2 = generate2

BIN
tabular.png

Binary file not shown.

After

Width:  |  Height:  |  Size: 183 KiB

Loading…
Cancel
Save