From f45471f59e88052f82d31fa6b6dc17966422b744 Mon Sep 17 00:00:00 2001 From: Jael Gu Date: Thu, 7 Apr 2022 17:39:49 +0800 Subject: [PATCH] Update README Signed-off-by: Jael Gu --- README.md | 49 ++++++++++++++++++++++++++++--------------------- 1 file changed, 28 insertions(+), 21 deletions(-) diff --git a/README.md b/README.md index 6ac0f91..8f2ab36 100644 --- a/README.md +++ b/README.md @@ -2,6 +2,7 @@ *Author: Jael Gu* +
## Desription @@ -10,34 +11,36 @@ Each vector represents for an audio clip with a fixed length of around 2s. This operator is built on top of the original implementation of [CLMR](https://github.com/Spijkervet/CLMR). The [default model weight](clmr_checkpoint_10000.pt) provided is pretrained on [Magnatagatune Dataset](https://paperswithcode.com/dataset/magnatagatune) with [SampleCNN](sample_cnn.py). +
+ ## Code Example Generate embeddings for the audio "test.wav". - *Write the pipeline in simplified style*: +*Write the pipeline in simplified style*: ```python -from towhee import dc +import towhee -dc.glob('test.wav') - .audio_decode() - .time_window(range=10) - .audio_embedding.clmr() - .show() +towhee.glob('test.wav') \ + .audio_decode() \ + .time_window(range=10) \ + .audio_embedding.clmr() \ + .show() ``` | [-2.1045141, 0.55381, 0.4537212, ...] shape=(6, 512) | *Write a same pipeline with explicit inputs/outputs name specifications:* ```python -from towhee import dc - -dc.glob['path']('test.wav') - .audio_decode['path', 'audio']() - .time_window['audio', 'frames'](range=10) - .audio_embedding.clmr['frames', 'vecs']() - .select('vecs') - .to_vec() +import towhee + +towhee.glob['path']('test.wav') \ + .audio_decode['path', 'audio']() \ + .time_window['audio', 'frames'](range=10) \ + .audio_embedding.clmr['frames', 'vecs']() \ + .select('vecs') \ + .to_vec() ``` [array([[-2.1045141 , 0.55381 , 0.4537212 , ..., 0.18805158, 0.3079657 , -1.216063 ], @@ -52,6 +55,8 @@ dc.glob['path']('test.wav') [-2.0771549 , 0.5641221 , 0.43814805, ..., 0.1822092 , 0.33022994, -1.2070588 ]], dtype=float32)] +
+ ## Factory Constructor Create the operator via the following factory method @@ -60,26 +65,28 @@ Create the operator via the following factory method **Parameters:** -​ *framework: str* +*framework: str* -​ The framework of model implementation. +The framework of model implementation. Default value is "pytorch" since the model is implemented in Pytorch. +
+ ## Interface An audio embedding operator generates vectors in numpy.ndarray given an audio file path or a [towhee audio](link/to/AudioFrame/api/doc). **Parameters:** -​ *Union[str, towhee.types.Audio]* +*Union[str, towhee.types.Audio (a sub-class of numpy.ndarray]* -​ The audio path or link in string. +The audio path or link in string. Or audio input data in towhee audio frames. The input data should represent for an audio longer than 2s. **Returns**: -​ *numpy.ndarray* +*numpy.ndarray* -​ Audio embeddings in shape (num_clips, 512). +Audio embeddings in shape (num_clips, 512). Each embedding stands for features of an audio clip with length of 2s. \ No newline at end of file