You can not select more than 25 topics
Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
|
|
|
# Image Embedding with Timm
|
|
|
|
|
|
|
|
*author: Jael Gu, Filip*
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
## Desription
|
|
|
|
|
|
|
|
An image embedding operator implemented with pretrained models provided by [Timm](https://github.com/rwightman/pytorch-image-models).
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
```python
|
|
|
|
from towhee import ops
|
|
|
|
import numpy as np
|
|
|
|
|
|
|
|
img_encoder = ops.image_embedding.timm('resnet50')
|
|
|
|
fake_img = np.zeros((256, 256, 3))
|
|
|
|
image_embedding = img_encoder(fake_img)
|
|
|
|
```
|
|
|
|
|
|
|
|
## Factory Constructor
|
|
|
|
|
|
|
|
Create the operator via the following factory method
|
|
|
|
|
|
|
|
***ops.image_embedding.timm(model_name)***
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
## Interface
|
|
|
|
|
|
|
|
An image decode operator takes an image path as input. It decodes the image back to ndarray.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
**Parameters:**
|
|
|
|
|
|
|
|
***img***: *numpy.ndarray*
|
|
|
|
|
|
|
|
The decoded image data in numpy.ndarray.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
**Returns**: *numpy.ndarray*
|
|
|
|
|
|
|
|
The image embedding extracted by model.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
## Code Example
|
|
|
|
|
|
|
|
Load an image from path './dog.jpg'
|
|
|
|
and use the pretrained ResNet50 model ('resnet50') to generate an image embedding.
|
|
|
|
|
|
|
|
*Write the pipeline in simplified style*:
|
|
|
|
|
|
|
|
```python
|
|
|
|
import towhee.DataCollection as dc
|
|
|
|
|
|
|
|
dc.glob(./dog.jpg)
|
|
|
|
.image_decode()
|
|
|
|
.image_embedding.timm('resnet50')
|
|
|
|
.show()
|
|
|
|
```
|
|
|
|
|
|
|
|
*Write a same pipeline with explicit inputs/outputs name specifications:*
|
|
|
|
|
|
|
|
```python
|
|
|
|
from towhee import DataCollection as dc
|
|
|
|
|
|
|
|
dc.glob['path'](./dog.jpg)
|
|
|
|
.image_decode['path', 'img']()
|
|
|
|
.image_embedding.timm['img', 'vec']('resnet50')
|
|
|
|
.select('img')
|
|
|
|
.show()
|
|
|
|
```
|
|
|
|
|
|
|
|
|