Image generation using Stable Diffusion

A text2image operator generates image given a text prompt. This operator is implemented with Huggingface Diffusers.

Code example

from towhee import pipe, ops

pipe = (
    pipe.input('prompt')
        .map('prompt', 'image', ops.text2image.stable_diffusion())
        .output('image')
)

image = pipe('an orange cat')
image.save('an_orange_cat.png')

Factory Constructor

Create the operator via the following factory method:

text2image.stable_diffusion(model_id='stabilityai/stable-diffusion-2-1', device=None)

Parameters:

model_id: str

The model id in string, defaults to 'stabilityai/stable-diffusion-2-1'.

Supported model names: pretrained diffuser models

device: str

The device to running model on, defaults to None. If None, it will automatically use cuda if gpu is available.

Interface

The operator takes a text prompt in string as input. It loads pretrained diffuser model and generates an image.

__call__(txt)

Parameters:

prompt: str

​ The text in string.

Returns:

PIL.Image

​ The generated image.

More Resources

• Scalar Quantization and Product Quantization - Zilliz blog: A hands-on dive into scalar quantization (integer quantization) and product quantization with Python.
• Supercharged Semantic Similarity Search in Production - Zilliz blog: Building a Blazing Fast, Highly Scalable Text-to-Image Search with CLIP embeddings and Milvus, the most advanced open-source vector database.
• Optimizing AI: A Guide to Stable Diffusion and Efficient Caching Strategies - Zilliz blog: This blog post will explore various caching strategies for optimizing Stable Diffusion models.
• An LLM Powered Text to Image Prompt Generation with Milvus - Zilliz blog: An interesting LLM project powered by the Milvus vector database for generating more efficient text-to-image prompts.
• Generative AI for Creative Applications using Storia Lab - Zilliz blog: This post discusses how Storia AI generates and edits images through simple text prompts or clicks and how we can leverage Storia AI and Milvus to build multimodal RAG.
• What Is a Diffusion Model? A Comprehensive Definition: A diffusion model applies Gaussian noise to an image and learns to remove the noise in a series of Markov steps. Learn more in this post.
• What is a Generative Adversarial Network? An Easy Guide: Just like we classify animal fossils into domains, kingdoms, and phyla, we classify AI networks, too. At the highest level, we classify AI networks as "discriminative" and "generative." A generative neural network is an AI that creates something new. This differs from a discriminative network, which classifies something that already exists into particular buckets. Kind of like we're doing right now, by bucketing generative adversarial networks (GANs) into appropriate classifications. So, if you were in a situation where you wanted to use textual tags to create a new visual image, like with Midjourney, you'd use a generative network. However, if you had a giant pile of data that you needed to classify and tag, you'd use a discriminative model.