logo
Browse Source

update the readme.

Signed-off-by: wxywb <xy.wang@zilliz.com>
main
wxywb 1 year ago
parent
commit
db25aefe45
  1. 27
      README.md
  2. 1
      clip_caption_reward.py

27
README.md

@ -17,28 +17,19 @@ This operator generates the caption with [CLIPReward](https://arxiv.org/abs/2205
Load an image from path './animals.jpg' to generate the caption.
*Write the pipeline in simplified style*:
*Write a pipeline with explicit inputs/outputs name specifications:*
```python
import towhee
from towhee.dc2 import pipe, ops, DataCollection
towhee.glob('./animals.jpg') \
.image_decode() \
.image_captioning.clip_caption_reward(model_name='clipRN50_clips_grammar') \
.show()
```
<img src="./cap.png" alt="result1" style="height:20px;"/>
*Write a same pipeline with explicit inputs/outputs name specifications:*
```python
import towhee
p = (
pipe.input('url')
.map('url', 'img', ops.image_decode.cv2_rgb())
.map('img', 'text', ops.image_captioning.clip_caption_reward(model_name='clipRN50_clips_grammar'))
.output('img', 'text')
)
towhee.glob['path']('./animals.jpg') \
.image_decode['path', 'img']() \
.image_captioning.clip_caption_reward['img', 'text'](model_name='clipRN50_clips_grammar') \
.select['img', 'text']() \
.show()
DataCollection(p('./animals.jpg')).show()
```
<img src="./tabular.png" alt="result2" style="height:60px;"/>

1
clip_caption_reward.py

@ -80,6 +80,7 @@ class ClipCaptionReward(NNOperator):
self.model = TransformerModel(opt)
self.model.load_state_dict(raw_state_dict)
self.model.to(self.device)
self.image_mean = (
torch.Tensor([0.48145466, 0.4578275, 0.40821073])

Loading…
Cancel
Save