diff --git a/README.md b/README.md index 00496dd..2bb0c7f 100644 --- a/README.md +++ b/README.md @@ -9,14 +9,14 @@ Author: Jinling xu An action classification operator generates labels of human activities (with corresponding scores) and extracts features for the input video. It transforms the video into frames and loads pre-trained models by model names. -This operator has implemented pre-trained models from [TimeSformer](https://arxiv.org/abs/2102.05095) +This operator has implemented pre-trained models from [VideoSwinTransformer](https://arxiv.org/abs/2106.13230) and maps vectors with labels.
## Code Example -Use the pretrained TimeSformer model ('timesformer_k400_8x224') +Use the pretrained VideoSwinTransformer model ('swin_tiny_patch244_window877_kinetics400_1k') to classify and generate a vector for the given video path './archery.mp4' ([download](https://dl.fbaipublicfiles.com/pytorchvideo/projects/archery.mp4)). *Write the pipeline in simplified style*: @@ -25,9 +25,8 @@ to classify and generate a vector for the given video path './archery.mp4' ([dow import towhee ( - towhee.dc(['./demo_video.mp4']) - .video_decode.ffmpeg(sample_type='uniform_temporal_subsample', args={'num_samples': 4}) - .runas_op(func=lambda x: [y for y in x]) + towhee.glob('./archery.mp4') + .video_decode.ffmpeg() .action_classification.video_swin_transformer(model_name='swin_tiny_patch244_window877_kinetics400_1k') .show() ) @@ -42,9 +41,8 @@ import towhee import towhee ( - towhee.dc['path']('./demo_video.mp4') - .video_decode.ffmpeg['path', 'frames'](sample_type='uniform_temporal_subsample', args={'num_samples': 4}) - .runas_op['frames', 'frames'](func=lambda x: [y for y in x]) + towhee.glob['path']('./archery.mp4') + .video_decode.ffmpeg['path', 'frames']() .action_classification.video_swin_transformer['frames', ('labels', 'scores', 'features')](model_name='swin_tiny_patch244_window877_kinetics400_1k') .select['path', 'labels', 'scores', 'features']() .show(formatter={'path': 'video_path'}) diff --git a/result1.png b/result1.png index d019f6d..e231f97 100644 Binary files a/result1.png and b/result1.png differ diff --git a/result2.png b/result2.png index dc0935a..8749ed2 100644 Binary files a/result2.png and b/result2.png differ