image-embedding-resnet50/README.md

# Image Embedding Pipeline with Resnet50

Authors: derekdqc, shiyu22

## Overview

This pipeline is used to **extract the feature vector of the image**. First step is to normalize the image, and then use resnet50 model to generate the vector.

The pipeline parses [the yaml file](./image_embedding_resnet50.yaml), which declares some components we call **Operator** and **DataFrame**. Next, we will introduce the interface, show how to use it and how it works, have fun with it!

## Interface

`towhee.pipeline(task: str, fmc: FileManagerConfig = FileManagerConfig(), branch: str = 'main', force_download: bool = False)`  [\[source\]](https://github.com/towhee-io/towhee/blob/main/towhee/__init__.py)

**params:**

- **task**(str): task name or pipeline repo name.
- **fmc**(FileManagerConfig): optional, file manager config for the local instance, default is a default FileManagerConfig obejct.
- **branch**(str): optional, which branch to use for operators/pipelines on hub, defaults to 'main'.
- **force_download**(bool): optional, whether to redownload pipeline and operators, default is False.

**return:**

- **_PipelineWrapper**, an instance of the wrapper class around `Pipeline`.

When we declare a pipeline object with a specific task, such as `towhee/image-embedding-resnet50` in this repo, it will run according to the Yaml file, and the input and output are:

**inputs:**

- **img_tensor**(PIL.Image), the image to be encoded.

**outputs:**

- **cnn**(numpy.ndarray), the embedding of the image.

## How to use

1. Install [Towhee](https://github.com/towhee-io/towhee)

```bash
$ pip3 install towhee
```

> You can refer to [Getting Started with Towhee](https://towhee.io/) for more details. If you have any questions, you can [submit an issue to the towhee repository](https://github.com/towhee-io/towhee/issues).

2. Run it with Towhee

```python
>>> from towhee import pipeline
>>> from PIL import Image

>>> img = Image.open('path/to/your/image') # for example, './test_data/test.jpg'
>>> embedding_pipeline = pipeline('towhee/image-embedding-resnet50')
>>> embedding = embedding_pipeline(img)
```

## How it works

First of all, you need to learn the pipeline and operator in Towhee architecture:

- **Pipeline**: A `Pipeline` is a single machine learning task that is composed of several operators. Operators are connected together internally via a directed acyclic graph.

- **Operator**: An `Operator` is a single node within a pipeline. It contains files (e.g. code, configs, models, etc...) and works for reusable operations (e.g., preprocessing an image, inference with a pretrained model).

This pipeline includes four functions: `_start_op`, `towhee/transform-image`, `towhee/resnet50-image-embedding` and `_end_op`. It is necessary to ensure that the input and output of the four Operators correspond to each other, and the input and output data types can be defined by DataFrame.

![img](./pic/pipeline.png)

Among the four Operator, `_start_op` and `_end_op` are required in any Pipeline, and they are used to start and end the pipeline in the Towhee system. For the other two Operators, please refer to [towhee/transform-image](https://hub.towhee.io/towhee/transform-image) and [towhee/resnet50-image-embedding](https://hub.towhee.io/towhee/resnet50-image-embedding).