towhee
copied
Readme
Files and versions
2.9 KiB
Audio Embedding
Description
The audio embedding pipeline converts an input audio into a dense vector which can be used to represent the audio clip's semantics. Each vector represents for an audio clip with a fixed length of around 0.9s. This operator is built on top of VGGish with Pytorch.
Code Example
- Create audio embedding pipeline with the default configuration.
from towhee import AutoPipes
p = AutoPipes.pipeline('audio-embedding')
res = p('test.wav')
res.get()
Interface
AudioEmbeddingConfig
You can find some parameters in audio_decode.ffmpeg and audio_embedding.vggish operators.
weights_path: str
The path to model weights. If None, it will load default model weights.
framework: str
The framework of model implementation. Default value is "pytorch" since the model is implemented in Pytorch.
device: int
The number of GPU device, defaults to -1, which means using CPU.
More Resources
- Exploring Multimodal Embeddings with FiftyOne and Milvus - Zilliz blog: This post explored how multimodal embeddings work with Voxel51 and Milvus.
- About Towhee team | Zilliz: Towhee is an open-source machine learning pipeline that helps you encode your unstructured data into embeddings.
- How to Get the Right Vector Embeddings - Zilliz blog: A comprehensive introduction to vector embeddings and how to generate them with popular open-source models.
- The guide to clip-vit-base-patch32 | OpenAI: clip-vit-base-patch32: a CLIP multimodal model variant by OpenAI for image and text embedding.
- Audio Retrieval Based on Milvus - Zilliz blog: Create an audio retrieval system using Milvus, an open-source vector database. Classify and analyze sound data in real time.
- Vector Database Use Case: Audio Similarity Search - Zilliz: Building agile and reliable audio similarity search with Zilliz vector database (fully managed Milvus).
- Making Machine Learning More Accessible for Developers - Zilliz blog: Learn how Towhee, an open-source embedding pipeline, supercharges the app development that requires embeddings and other ML tasks.
- An Introduction to Vector Embeddings: What They Are and How to Use Them - Zilliz blog: In this blog post, we will understand the concept of vector embeddings and explore its applications, best practices, and tools for working with embeddings."
2.9 KiB
Audio Embedding
Description
The audio embedding pipeline converts an input audio into a dense vector which can be used to represent the audio clip's semantics. Each vector represents for an audio clip with a fixed length of around 0.9s. This operator is built on top of VGGish with Pytorch.
Code Example
- Create audio embedding pipeline with the default configuration.
from towhee import AutoPipes
p = AutoPipes.pipeline('audio-embedding')
res = p('test.wav')
res.get()
Interface
AudioEmbeddingConfig
You can find some parameters in audio_decode.ffmpeg and audio_embedding.vggish operators.
weights_path: str
The path to model weights. If None, it will load default model weights.
framework: str
The framework of model implementation. Default value is "pytorch" since the model is implemented in Pytorch.
device: int
The number of GPU device, defaults to -1, which means using CPU.
More Resources
- Exploring Multimodal Embeddings with FiftyOne and Milvus - Zilliz blog: This post explored how multimodal embeddings work with Voxel51 and Milvus.
- About Towhee team | Zilliz: Towhee is an open-source machine learning pipeline that helps you encode your unstructured data into embeddings.
- How to Get the Right Vector Embeddings - Zilliz blog: A comprehensive introduction to vector embeddings and how to generate them with popular open-source models.
- The guide to clip-vit-base-patch32 | OpenAI: clip-vit-base-patch32: a CLIP multimodal model variant by OpenAI for image and text embedding.
- Audio Retrieval Based on Milvus - Zilliz blog: Create an audio retrieval system using Milvus, an open-source vector database. Classify and analyze sound data in real time.
- Vector Database Use Case: Audio Similarity Search - Zilliz: Building agile and reliable audio similarity search with Zilliz vector database (fully managed Milvus).
- Making Machine Learning More Accessible for Developers - Zilliz blog: Learn how Towhee, an open-source embedding pipeline, supercharges the app development that requires embeddings and other ML tasks.
- An Introduction to Vector Embeddings: What They Are and How to Use Them - Zilliz blog: In this blog post, we will understand the concept of vector embeddings and explore its applications, best practices, and tools for working with embeddings."