diff --git a/README.md b/README.md index 4367176..7c1ce36 100644 --- a/README.md +++ b/README.md @@ -18,18 +18,18 @@ This operator extracts features for image or text with [CLIP](https://arxiv.org/ ## Code Example Load an image from path './teddy.jpg' to generate an image embedding. -Read the text 'a dog' to generate an text embedding. +Read the text 'A teddybear on a skateboard in Times Square.' to generate an text embedding. *Write the pipeline in simplified style*: ```python import towhee -towhee.glob('./dog.jpg') \ +towhee.glob('./teddy.jpg') \ .image_decode.cv2() \ .towhee.clip(name='ViT-B/32', modality='image') \ .show() -towhee.dc(["a dog"]) \ +towhee.dc(["A teddybear on a skateboard in Times Square."]) \ .image_decode.cv2() \ .towhee.clip(name='ViT-B/32', modality='text') \ .show() @@ -42,19 +42,19 @@ towhee.dc(["a dog"]) \ ```python import towhee -towhee.glob['path']('./dog.jpg') \ +towhee.glob['path']('./teddy.jpg') \ .image_decode.cv2['path', 'img']() \ .towhee.clip['data', 'vec'](name='ViT-B/32', modality='image') \ .select['data', 'vec']() \ .show() -towhee.dc(["a dog"]) \ - .select['img', 'vec']() \ - .towhee.clip['data', 'vec'](name='ViT-B/32', modality='image') \ - .select['data', 'vec']() \ +towhee.dc['text'](["A teddybear on a skateboard in Times Square."]) \ + .towhee.clip['text','vec'](name='ViT-B/32', modality='text') \ + .select['text', 'vec']() \ .show() ``` result1 +result2
@@ -71,7 +71,16 @@ Create the operator via the following factory method ​ ***name:*** *str* -​ The model name of CLIP. +​ The model name of CLIP. avaliable options are: +- RN50 +- RN101 +- RN50x4 +- RN50x16 +- RN50x64 +- ViT-B/32 +- ViT-B/64 +- ViT-L/14 + ​ ***modality:*** *str* @@ -83,7 +92,7 @@ Create the operator via the following factory method ## Interface -An image-text embedding operator takes a [towhee image](link/to/towhee/image/api/doc) or string as input and generate an image embedding in ndarray. +An image-text embedding operator takes a [towhee image](link/to/towhee/image/api/doc) or string as input and generate an embedding in ndarray. **Parameters:** diff --git a/tabular2.png b/tabular2.png new file mode 100644 index 0000000..965a93a Binary files /dev/null and b/tabular2.png differ