diff --git a/README.md b/README.md index dc22870..f459b25 100644 --- a/README.md +++ b/README.md @@ -17,15 +17,27 @@ This operator extracts features for video or text with [MDMMT: Multidomain Multi ## Code Example -Load a video embeddings extracted from different upstream expert networks, such as video, RGB, audio. - Read the text to generate a text embedding. - *Write the pipeline code*: +```python +from towhee.dc2 import pipe, ops, DataCollection + +p = ( + pipe.input('text') \ + .map('text', 'vec', ops.video_text_embedding.mdmmt(modality='text', device='cuda:0')) \ + .output('text', 'vec') +) + +DataCollection(p('Hello world.')).show() +``` +![](text_emb_result.png) + +Load a video embeddings extracted from different upstream expert networks, such as video, RGB, audio. ```python -import towhee import torch +from towhee.dc2 import pipe, ops, DataCollection + torch.manual_seed(42) @@ -52,14 +64,16 @@ features_ind = { video_input_dict = {"features": features, "features_t": features_t, "features_ind": features_ind} -towhee.dc([video_input_dict]).video_text_embedding.mdmmt(modality='video', device='cpu').show() +p = ( + pipe.input('video_input_dict') \ + .map('video_input_dict', 'vec', ops.video_text_embedding.mdmmt(modality='video', device='cuda:0')) \ + .output('video_input_dict', 'vec') +) -towhee.dc(['Hello world.']).video_text_embedding.mdmmt(modality='text', device='cpu').show() +DataCollection(p(video_input_dict)).show() ``` -![](vect_simplified_video.png) -![](vect_simplified_text.png) +![](video_emb_result.png) -*Write a same pipeline with explicit inputs/outputs name specifications:*
diff --git a/text_emb_result.png b/text_emb_result.png new file mode 100644 index 0000000..b7b2a21 Binary files /dev/null and b/text_emb_result.png differ diff --git a/vect_simplified_text.png b/vect_simplified_text.png deleted file mode 100644 index 91004b6..0000000 Binary files a/vect_simplified_text.png and /dev/null differ diff --git a/vect_simplified_video.png b/vect_simplified_video.png deleted file mode 100644 index 056d876..0000000 Binary files a/vect_simplified_video.png and /dev/null differ diff --git a/video_emb_result.png b/video_emb_result.png new file mode 100644 index 0000000..7bc5c68 Binary files /dev/null and b/video_emb_result.png differ