logo
Browse Source

Update

Signed-off-by: shiyu22 <shiyu.chen@zilliz.com>
main
shiyu22 3 years ago
parent
commit
8185d07e61
  1. 20
      README.md
  2. 14
      image_embedding_resnet50.yaml

20
README.md

@ -4,21 +4,25 @@ Authors: derekdqc, shiyu22
## Overview ## Overview
This pipeline is used to **extract the feature vector of the image**. First step is to normalize the image, and then use resnet50 model to generate the vector.
This pipeline is used to **extract the feature vector of a given image**.
The pipeline performs two major steps. First, normalize the image, and then use resnet50 model to generate the vector.
## Interface ## Interface
**Args:**
**Pipeline Input Arguments:**
- img(`PIL.Image`):
​ img_tensor(`PIL.Image`):
The image to be encoded. `PIL.Image` is the currently supported image type.
​ The image to be encoded.
**Pipeline Returns:**
**Returns:**
- (`Tuple[('cnn', numpy.ndarray)]`)
​ (`Tuple[('cnn', numpy.ndarray)]`)
The pipeline returns a tuple.
​ The embedding of the image.
- The `feature_vector` field contains the embedding of the image.
## How to use ## How to use
@ -43,6 +47,6 @@ $ pip3 install towhee
## How it works ## How it works
You can learn the pipeline and operator in [Towhee architecture](https://github.com/towhee-io/towhee#towhee-architecture). This pipeline includes two main functions: [towhee/transform-image](https://hub.towhee.io/towhee/transform-image) and [towhee/resnet50-image-embedding](https://hub.towhee.io/towhee/resnet50-image-embedding). It is necessary to ensure that the input and output of the four Operators correspond to each other, and the input and output data types can be defined by DataFrame.
This pipeline includes two main operators: transform image (implemented as [towhee/transform-image](https://hub.towhee.io/towhee/transform-image)) and image embedding (implemented as [towhee/resnet50-image-embedding](https://hub.towhee.io/towhee/resnet50-image-embedding)). The transform image operator will first convert the original image into a normalized format, such as with 512x512 resolutions. Then, the normalized image will be encoded via image embedding operator, and finally we get a feature vector of the given image. (The basic concepts of pipeline and operator are introduced in [Towhee architecture](https://github.com/towhee-io/towhee#towhee-architecture).)
![img](./readme_res/pipeline.png) ![img](./readme_res/pipeline.png)

14
image_embedding_resnet50.yaml

@ -7,7 +7,7 @@ operators:
inputs: inputs:
- -
df: '_start_df' df: '_start_df'
name: 'img_tensor'
name: 'img'
col: 0 col: 0
outputs: outputs:
- -
@ -23,7 +23,7 @@ operators:
inputs: inputs:
- -
df: 'image' df: 'image'
name: 'img_tensor'
name: 'img'
col: 0 col: 0
outputs: outputs:
- -
@ -53,7 +53,7 @@ operators:
inputs: inputs:
- -
df: 'embedding' df: 'embedding'
name: 'cnn'
name: 'feature_vector'
col: 0 col: 0
outputs: outputs:
- -
@ -65,13 +65,13 @@ dataframes:
name: '_start_df' name: '_start_df'
columns: columns:
- -
name: 'img_tensor'
name: 'img'
vtype: 'PIL.Image' vtype: 'PIL.Image'
- -
name: 'image' name: 'image'
columns: columns:
- -
name: 'img_tensor'
name: 'img'
vtype: 'PIL.Image' vtype: 'PIL.Image'
- -
name: 'image_preproc' name: 'image_preproc'
@ -83,11 +83,11 @@ dataframes:
name: 'embedding' name: 'embedding'
columns: columns:
- -
name: 'cnn'
name: 'feature_vector'
vtype: 'numpy.ndarray' vtype: 'numpy.ndarray'
- -
name: '_end_df' name: '_end_df'
columns: columns:
- -
name: 'cnn'
name: 'feature_vector'
vtype: 'numpy.ndarray' vtype: 'numpy.ndarray'

Loading…
Cancel
Save