|
@ -2,7 +2,7 @@ |
|
|
|
|
|
|
|
|
*author: Jael Gu, Filip* |
|
|
*author: Jael Gu, Filip* |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
</br> |
|
|
|
|
|
|
|
|
## Desription |
|
|
## Desription |
|
|
|
|
|
|
|
@ -11,6 +11,8 @@ This operator extracts features for image with pretrained models provided by [Ti |
|
|
Timm is a deep-learning library developed by [Ross Wightman](https://twitter.com/wightmanr), |
|
|
Timm is a deep-learning library developed by [Ross Wightman](https://twitter.com/wightmanr), |
|
|
which maintains SOTA deep-learning models and tools in computer vision. |
|
|
which maintains SOTA deep-learning models and tools in computer vision. |
|
|
|
|
|
|
|
|
|
|
|
</br> |
|
|
|
|
|
|
|
|
## Code Example |
|
|
## Code Example |
|
|
|
|
|
|
|
|
Load an image from path './towhee.jpg' |
|
|
Load an image from path './towhee.jpg' |
|
@ -26,7 +28,7 @@ towhee.glob('./towhee.jpg') \ |
|
|
.image_embedding.timm(model_name='resnet50') \ |
|
|
.image_embedding.timm(model_name='resnet50') \ |
|
|
.show() |
|
|
.show() |
|
|
``` |
|
|
``` |
|
|
<img src="https://towhee.io/image-embedding/timm/raw/branch/main/result1.png" alt="result1" style="height:20px;"/> |
|
|
|
|
|
|
|
|
<img src="./result1.png" height="80px"/> |
|
|
|
|
|
|
|
|
*Write a same pipeline with explicit inputs/outputs name specifications:* |
|
|
*Write a same pipeline with explicit inputs/outputs name specifications:* |
|
|
|
|
|
|
|
@ -39,7 +41,9 @@ towhee.glob['path']('./towhee.jpg') \ |
|
|
.select('img', 'vec') \ |
|
|
.select('img', 'vec') \ |
|
|
.show() |
|
|
.show() |
|
|
``` |
|
|
``` |
|
|
<img src="https://towhee.io/image-embedding/timm/raw/branch/main/result2.png" alt="result2" style="height:60px;"/> |
|
|
|
|
|
|
|
|
<img src="./result2.png" height="100px"/> |
|
|
|
|
|
|
|
|
|
|
|
</br> |
|
|
|
|
|
|
|
|
## Factory Constructor |
|
|
## Factory Constructor |
|
|
|
|
|
|
|
@ -49,44 +53,45 @@ Create the operator via the following factory method |
|
|
|
|
|
|
|
|
**Parameters:** |
|
|
**Parameters:** |
|
|
|
|
|
|
|
|
***model_name***: *str* |
|
|
|
|
|
|
|
|
  ***model_name***: *str* |
|
|
|
|
|
|
|
|
The model name in string. The default value is "resnet34". |
|
|
|
|
|
|
|
|
  The model name in string. The default value is "resnet34". |
|
|
Refer [Timm Docs](https://fastai.github.io/timmdocs/#List-Models-with-Pretrained-Weights) to get a full list of supported models. |
|
|
Refer [Timm Docs](https://fastai.github.io/timmdocs/#List-Models-with-Pretrained-Weights) to get a full list of supported models. |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
***num_classes***: *int* |
|
|
|
|
|
|
|
|
  ***num_classes***: *int* |
|
|
|
|
|
|
|
|
The number of classes. The default value is 1000. |
|
|
|
|
|
|
|
|
  The number of classes. The default value is 1000. |
|
|
It is related to model and dataset. |
|
|
It is related to model and dataset. |
|
|
|
|
|
|
|
|
***skip_preprocess***: *bool* |
|
|
|
|
|
|
|
|
  ***skip_preprocess***: *bool* |
|
|
|
|
|
|
|
|
The flag to control whether to skip image preprocess. |
|
|
|
|
|
|
|
|
  The flag to control whether to skip image preprocess. |
|
|
The default value is False. |
|
|
The default value is False. |
|
|
If set to True, it will skip image preprocessing steps (transforms). |
|
|
If set to True, it will skip image preprocessing steps (transforms). |
|
|
In this case, input image data must be prepared in advance in order to properly fit the model. |
|
|
In this case, input image data must be prepared in advance in order to properly fit the model. |
|
|
|
|
|
|
|
|
|
|
|
</br> |
|
|
|
|
|
|
|
|
## Interface |
|
|
## Interface |
|
|
|
|
|
|
|
|
An image embedding operator takes a [towhee image](link/to/towhee/image/api/doc) as input. |
|
|
|
|
|
|
|
|
An image embedding operator takes a towhee image as input. |
|
|
It uses the pre-trained model specified by model name to generate an image embedding in ndarray. |
|
|
It uses the pre-trained model specified by model name to generate an image embedding in ndarray. |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
**Parameters:** |
|
|
**Parameters:** |
|
|
|
|
|
|
|
|
***img***: *towhee.types.Image* |
|
|
|
|
|
|
|
|
  ***img***: *towhee.types.Image (a sub-class of numpy.ndarray)* |
|
|
|
|
|
|
|
|
The decoded image data in towhee.types.Image (numpy.ndarray). |
|
|
|
|
|
|
|
|
  The decoded image data in numpy.ndarray. |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
**Returns**: |
|
|
**Returns**: |
|
|
|
|
|
|
|
|
*numpy.ndarray* |
|
|
|
|
|
|
|
|
  *numpy.ndarray* |
|
|
|
|
|
|
|
|
The image embedding extracted by model. |
|
|
|
|
|
|
|
|
  The image embedding extracted by model. |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|