@ -3,11 +3,18 @@
*author: David Wang*
*author: David Wang*
< br / >
## Desription
## Description
This operator extracts features for image with [DOLG ](https://arxiv.org/abs/2108.02927 ) which has special design for image retrieval task. It integrates local and global information inside images into compact image representations. This operator is an adaptation from [dongkyuk/DOLG-pytorch ](https://github.com/dongkyuk/DOLG-pytorch ).
This operator extracts features for image with [DOLG ](https://arxiv.org/abs/2108.02927 ) which has special design for image retrieval task. It integrates local and global information inside images into compact image representations. This operator is an adaptation from [dongkyuk/DOLG-pytorch ](https://github.com/dongkyuk/DOLG-pytorch ).
< br / >
## Code Example
## Code Example
Load an image from path './towhee.jpg' to generate an image embedding.
Load an image from path './towhee.jpg' to generate an image embedding.
@ -33,11 +40,16 @@ import towhee
towhee.glob['path']('./towhee.jpg') \
towhee.glob['path']('./towhee.jpg') \
.image_decode.cv2['path', 'img']() \
.image_decode.cv2['path', 'img']() \
.image_embedding.dolg['img', 'vec'](img_size=512, input_dim=3, hidden_dim=1024, output_dim=2048) \
.image_embedding.dolg['img', 'vec'](img_size=512, input_dim=3, hidden_dim=1024, output_dim=2048) \
.select('img', 'vec' ) \
.select['img', 'vec']( ) \
.show()
.show()
```
```
< img src = "https://towhee.io/image-embedding/dolg/raw/branch/main/result2.png" alt = "result2" style = "height:60px;" / >
< img src = "https://towhee.io/image-embedding/dolg/raw/branch/main/result2.png" alt = "result2" style = "height:60px;" / >
< br / >
## Factory Constructor
## Factory Constructor
Create the operator via the following factory method
Create the operator via the following factory method
@ -46,22 +58,27 @@ Create the operator via the following factory method
**Parameters:**
**Parameters:**
** *img_size***: *int*
** *img_size: *** *int*
Scaled input image size to extract embedding. The higher resolution would generate the more discriminateive feature but cost more time to calculate.
Scaled input image size to extract embedding. The higher resolution would generate the more discriminateive feature but cost more time to calculate.
** *input_dim***: *int*
** *input_dim: *** *int*
The input dimension of DOLG module (equals pretrained cnn output dimension).
The input dimension of DOLG module (equals pretrained cnn output dimension).
** *hidden_dim***: *int*
** *hidden_dim: *** *int*
The hidden dimension size, local feature branch output dimension.
The hidden dimension size, local feature branch output dimension.
** *output_dim***: *int*
** *output_dim: *** *int*
The output dimsion size, same as embedding size.
The output dimsion size, same as embedding size.
< br / >
## Interface
## Interface
An image embedding operator takes a [towhee image ](link/to/towhee/image/api/doc ) as input.
An image embedding operator takes a [towhee image ](link/to/towhee/image/api/doc ) as input.
@ -70,13 +87,13 @@ It uses the pre-trained model specified by model name to generate an image embed
**Parameters:**
**Parameters:**
** *img***: *towhee.types.Image (a sub-class of numpy.ndarray)*
** *img: *** *towhee.types.Image (a sub-class of numpy.ndarray)*
The decoded image data in towhee.types.Image (numpy.ndarray).
The decoded image data in towhee.types.Image (numpy.ndarray).
**Returns**: *numpy.ndarray*
**Returns: ** *numpy.ndarray*
The image embedding extracted by model.
The image embedding extracted by model.