|
|
|
# Image Embedding with Timm
|
|
|
|
|
|
|
|
*author: Jael Gu, Filip*
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
## Desription
|
|
|
|
|
|
|
|
An image embedding operator implemented with pretrained models provided by [Timm](https://github.com/rwightman/pytorch-image-models).
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
```python
|
|
|
|
from towhee import ops
|
|
|
|
import numpy as np
|
|
|
|
|
|
|
|
img_encoder = ops.image_embedding.timm(model_name='resnet50')
|
|
|
|
fake_img = np.zeros((256, 256, 3))
|
|
|
|
image_embedding = img_encoder(fake_img)
|
|
|
|
```
|
|
|
|
|
|
|
|
## Factory Constructor
|
|
|
|
|
|
|
|
Create the operator via the following factory method
|
|
|
|
|
|
|
|
***ops.image_embedding.timm(model_name)***
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
## Interface
|
|
|
|
|
|
|
|
An image embedding operator takes an image in ndarray as input.
|
|
|
|
It uses the pre-trained model specified by model name to generate an image embedding in ndarray.
|
|
|
|
|
|
|
|
|
|
|
|
**Parameters:**
|
|
|
|
|
|
|
|
***img***: *numpy.ndarray*
|
|
|
|
|
|
|
|
The decoded image data in numpy.ndarray.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
**Returns**: *numpy.ndarray*
|
|
|
|
|
|
|
|
The image embedding extracted by model.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
## Code Example
|
|
|
|
|
|
|
|
Load an image from path './dog.jpg'
|
|
|
|
and use the pretrained ResNet50 model ('resnet50') to generate an image embedding.
|
|
|
|
|
|
|
|
*Write the pipeline in simplified style*:
|
|
|
|
|
|
|
|
```python
|
|
|
|
from towhee import dc
|
|
|
|
|
|
|
|
dc.glob('./dog.jpg')
|
|
|
|
.image_decode()
|
|
|
|
.image_embedding.timm('resnet50')
|
|
|
|
.show()
|
|
|
|
```
|
|
|
|
|
|
|
|
*Write a same pipeline with explicit inputs/outputs name specifications:*
|
|
|
|
|
|
|
|
```python
|
|
|
|
from towhee import dc
|
|
|
|
|
|
|
|
dc.glob['path']('./dog.jpg')
|
|
|
|
.image_decode['path', 'img']()
|
|
|
|
.image_embedding.timm['img', 'vec']('resnet50')
|
|
|
|
.select('img')
|
|
|
|
.show()
|
|
|
|
```
|
|
|
|
|
|
|
|
|