dolg/README.md

Image Embdding with DOLG

author: David Wang

Desription

This operator extracts features for image with DOLG which has special design for image retrieval task. It integrates local and global information inside images into compact image representations. This operator is an adaptation from dongkyuk/DOLG-pytorch.

Code Example

Load an image from path './towhee.jpg' to generate an image embedding.

Write the pipeline in simplified style:

import towhee

towhee.glob('./towhee.jpg') \
      .image_decode.cv2() \
      .image_embedding.dolg(img_size=512, input_dim=3, hidden_dim=1024, output_dim=2048) \
      .show()

Write a same pipeline with explicit inputs/outputs name specifications:

import towhee

towhee.glob['path']('./towhee.jpg') \
      .image_decode.cv2['path', 'img']() \
      .image_embedding.dolg['img', 'vec'](img_size=512, input_dim=3, hidden_dim=1024, output_dim=2048) \
      .select('img', 'vec') \
      .show()

Factory Constructor

Create the operator via the following factory method

image_embedding.dolg(img_size=512, input_dim=3, hidden_dim=1024, output_dim=2048)

Parameters:

​ img_size: int

​ Scaled input image size to extract embedding. The higher resolution would generate the more discriminateive feature but cost more time to calculate.

​ input_dim: int

​ The input dimension of DOLG module (equals pretrained cnn output dimension).

​ hidden_dim: int

​ The hidden dimension size, local feature branch output dimension.

​ output_dim: int

​ The output dimsion size, same as embedding size.

Interface

An image embedding operator takes a towhee image as input. It uses the pre-trained model specified by model name to generate an image embedding in ndarray.

Parameters:

​ img: towhee.types.Image (a sub-class of numpy.ndarray)

​ The decoded image data in towhee.types.Image (numpy.ndarray).

Returns: numpy.ndarray

​ The image embedding extracted by model.