logo
Browse Source

add6

Signed-off-by: xujinling <jinling.xu@zilliz.com>
main
xujinling 2 years ago
parent
commit
af30f8698c
  1. 14
      README.md
  2. BIN
      result1.png
  3. BIN
      result2.png

14
README.md

@ -9,14 +9,14 @@ Author: Jinling xu
An action classification operator generates labels of human activities (with corresponding scores) An action classification operator generates labels of human activities (with corresponding scores)
and extracts features for the input video. and extracts features for the input video.
It transforms the video into frames and loads pre-trained models by model names. It transforms the video into frames and loads pre-trained models by model names.
This operator has implemented pre-trained models from [TimeSformer](https://arxiv.org/abs/2102.05095)
This operator has implemented pre-trained models from [VideoSwinTransformer](https://arxiv.org/abs/2106.13230)
and maps vectors with labels. and maps vectors with labels.
<br /> <br />
## Code Example ## Code Example
Use the pretrained TimeSformer model ('timesformer_k400_8x224')
Use the pretrained VideoSwinTransformer model ('swin_tiny_patch244_window877_kinetics400_1k')
to classify and generate a vector for the given video path './archery.mp4' ([download](https://dl.fbaipublicfiles.com/pytorchvideo/projects/archery.mp4)). to classify and generate a vector for the given video path './archery.mp4' ([download](https://dl.fbaipublicfiles.com/pytorchvideo/projects/archery.mp4)).
*Write the pipeline in simplified style*: *Write the pipeline in simplified style*:
@ -25,9 +25,8 @@ to classify and generate a vector for the given video path './archery.mp4' ([dow
import towhee import towhee
( (
towhee.dc(['./demo_video.mp4'])
.video_decode.ffmpeg(sample_type='uniform_temporal_subsample', args={'num_samples': 4})
.runas_op(func=lambda x: [y for y in x])
towhee.glob('./archery.mp4')
.video_decode.ffmpeg()
.action_classification.video_swin_transformer(model_name='swin_tiny_patch244_window877_kinetics400_1k') .action_classification.video_swin_transformer(model_name='swin_tiny_patch244_window877_kinetics400_1k')
.show() .show()
) )
@ -42,9 +41,8 @@ import towhee
import towhee import towhee
( (
towhee.dc['path']('./demo_video.mp4')
.video_decode.ffmpeg['path', 'frames'](sample_type='uniform_temporal_subsample', args={'num_samples': 4})
.runas_op['frames', 'frames'](func=lambda x: [y for y in x])
towhee.glob['path']('./archery.mp4')
.video_decode.ffmpeg['path', 'frames']()
.action_classification.video_swin_transformer['frames', ('labels', 'scores', 'features')](model_name='swin_tiny_patch244_window877_kinetics400_1k') .action_classification.video_swin_transformer['frames', ('labels', 'scores', 'features')](model_name='swin_tiny_patch244_window877_kinetics400_1k')
.select['path', 'labels', 'scores', 'features']() .select['path', 'labels', 'scores', 'features']()
.show(formatter={'path': 'video_path'}) .show(formatter={'path': 'video_path'})

BIN
result1.png

Binary file not shown.

Before

Width:  |  Height:  |  Size: 50 KiB

After

Width:  |  Height:  |  Size: 13 KiB

BIN
result2.png

Binary file not shown.

Before

Width:  |  Height:  |  Size: 153 KiB

After

Width:  |  Height:  |  Size: 89 KiB

Loading…
Cancel
Save