Image Embedding Pipeline with Resnet50

Authors: Kyle, shiyu22

Overview

This pipeline is used to extract the feature vector of the image, first to normalize the image , and then use resnet50 model to generate the vector.

In fact, the pipeline runs by parsing the yaml file, which declares some functions we call Operator, and the DataFrame required by each Operator. Next will introduce the interface, how to use it and how it works, have fun with it!

Interface

towhee.pipeline(task: str, fmc: FileManagerConfig = FileManagerConfig(), branch: str = 'main', force_download: bool = False) source

param:

• task(str), task name or pipeline repo name.
• fmc(FileManagerConfig), optional file manager config for the local instance, default is FileManagerConfig().
• branch(str), which branch to use for operators/pipelines on hub, defaults to 'main'.
• force_download(bool), whether to redownload pipeline and operators, default is False.

return:

• _PipelineWrapper, which is a wrapper class around Pipeline.

When we declare a pipeline object with a specific task, such as towhee/image-embedding-resnet50 in this repo, it will run according to the Yaml file, and the input and output are as follows:

inputs:

• img_tensor(PIL.Image), image to be embedded.

outputs:

• cnn(numpy.ndarray), the embedding of image.

How to use

1. Install Towhee

$ pip3 install towhee

You can refer to Getting Started with Towhee for more details. If you have questions, you can submit an issue to the towhee repository.

2. Run it with Towhee

>>> from towhee import pipeline
>>> from PIL import Image

>>> img = Image.open('./test_data/test.jpg')
>>> embedding_pipeline = pipeline('towhee/image-embedding-resnet50')
>>> embedding = embedding_pipeline(img)

How it works

First of all, you need to learn the pipeline and operator in Towhee architecture:

• Pipeline: A Pipeline is a single machine learning task that is composed of several operators. Operators are connected together internally via a directed acyclic graph.

• Operator: An Operator is a single node within a pipeline. It contains files (e.g. code, configs, models, etc...) and works for reusable operations (e.g., preprocessing an image, inference with a pretrained model).

This pipeline includes four functions: _start_op, towhee/transform-image, towhee/resnet50-image-embedding and _end_op. It is necessary to ensure that the input and output of the four Operators correspond to each other, and the input and output data types can be defined by DataFrame.

Among the four Operator,_start_op and _end_op are required in any Pipeline, and they are used to start and end the pipeline in the Towhee system. For the other two Operators, please refer to towhee/transform-image and towhee/resnet50-image-embedding.