Browse Source
        
      
      Update
      
        Signed-off-by: Jael Gu <mengjia.gu@zilliz.com>
      
      
        main
      
      
     
    
    
    
	
		
			
				 2 changed files with 
10 additions and 
25 deletions
			 
			
		 
		
			
				- 
					
					
					 
					README.md
				
- 
					
						
							
								BIN
							
						
					 result.png
				
				
					
						
							
								
									
	
		
			
				|  | @ -18,37 +18,22 @@ This operator extracts features for audio with [data2vec](https://arxiv.org/abs/ | 
		
	
		
			
				|  |  | 
 |  |  | 
 | 
		
	
		
			
				|  |  | Generate embeddings for the audio "test.wav". |  |  | Generate embeddings for the audio "test.wav". | 
		
	
		
			
				|  |  | 
 |  |  | 
 | 
		
	
		
			
				|  |  | 
 |  |  |  | 
		
	
		
			
				|  |  |  *Write the pipeline in simplified style*: |  |  |  | 
		
	
		
			
				|  |  |  |  |  | *Write a pipeline with explicit inputs/outputs name specifications:* | 
		
	
		
			
				|  |  | 
 |  |  | 
 | 
		
	
		
			
				|  |  | ```python |  |  | ```python | 
		
	
		
			
				|  |  | import towhee |  |  |  | 
		
	
		
			
				|  |  | 
 |  |  |  | 
		
	
		
			
				|  |  | ( |  |  |  | 
		
	
		
			
				|  |  |     towhee.glob('test.wav') |  |  |  | 
		
	
		
			
				|  |  |           .audio_decode.ffmpeg() |  |  |  | 
		
	
		
			
				|  |  |           .runas_op(func=lambda x:[y[0] for y in x]) |  |  |  | 
		
	
		
			
				|  |  |           .audio_embedding.data2vec() |  |  |  | 
		
	
		
			
				|  |  |           .show() |  |  |  | 
		
	
		
			
				|  |  | ) |  |  |  | 
		
	
		
			
				|  |  | 
 |  |  |  | 
		
	
		
			
				|  |  | ``` |  |  |  | 
		
	
		
			
				|  |  |  |  |  | from towhee.dc2 import pipe, ops, DataCollection | 
		
	
		
			
				|  |  | 
 |  |  | 
 | 
		
	
		
			
				|  |  | *Write a same pipeline with explicit inputs/outputs name specifications:* |  |  |  | 
		
	
		
			
				|  |  | 
 |  |  |  | 
		
	
		
			
				|  |  | ```python |  |  |  | 
		
	
		
			
				|  |  | import towhee |  |  |  | 
		
	
		
			
				|  |  | 
 |  |  |  | 
		
	
		
			
				|  |  | ( |  |  |  | 
		
	
		
			
				|  |  |     towhee.glob['path']('test.wav') |  |  |  | 
		
	
		
			
				|  |  |           .audio_decode.ffmpeg['path', 'frames']() |  |  |  | 
		
	
		
			
				|  |  |           .runas_op['frames', 'frames'](func=lambda x:[y[0] for y in x]) |  |  |  | 
		
	
		
			
				|  |  |           .audio_embedding.data2vec['frames', 'vecs'](model_name="facebook/data2vec-audio-base-960h") |  |  |  | 
		
	
		
			
				|  |  |           .select['path', 'vecs']() |  |  |  | 
		
	
		
			
				|  |  |           .show() |  |  |  | 
		
	
		
			
				|  |  |  |  |  | p = ( | 
		
	
		
			
				|  |  |  |  |  |     pipe.input('path') | 
		
	
		
			
				|  |  |  |  |  |         .map('path', 'frame', ops.audio_decode.ffmpeg(sample_rate=16000)) | 
		
	
		
			
				|  |  |  |  |  |         .map('frame', 'vecs', ops.audio_embedding.data2vec(model_name='facebook/data2vec-audio-base-960h')) | 
		
	
		
			
				|  |  |  |  |  |         .output('path', 'vecs') | 
		
	
		
			
				|  |  | ) |  |  | ) | 
		
	
		
			
				|  |  |  |  |  | 
 | 
		
	
		
			
				|  |  |  |  |  | DataCollection(p('test.wav')).show() | 
		
	
		
			
				|  |  | ``` |  |  | ``` | 
		
	
		
			
				|  |  | 
 |  |  | 
 | 
		
	
		
			
				|  |  |  |  |  | <img src="./result.png" width="800px"/> | 
		
	
		
			
				|  |  | 
 |  |  | 
 | 
		
	
		
			
				|  |  | <br /> |  |  | <br /> | 
		
	
		
			
				|  |  | 
 |  |  | 
 | 
		
	
	
		
			
				|  | 
 | 
		
	
								
							
						
					 
					
				 
			 
		
			
			
			
			
			
			
				
				
					
					
						
							
								
									
	| 
			
			
				
					
						
						
						
							
							![]()  
								
									Width: 
									 | 
									Height: 
									 | 
								
								Size: 5.7 KiB
							 |