logo
Browse Source

add6

Signed-off-by: xujinling <jinling.xu@zilliz.com>
main
xujinling 2 years ago
parent
commit
af30f8698c
  1. 14
      README.md
  2. BIN
      result1.png
  3. BIN
      result2.png

14
README.md

@ -9,14 +9,14 @@ Author: Jinling xu
An action classification operator generates labels of human activities (with corresponding scores)
and extracts features for the input video.
It transforms the video into frames and loads pre-trained models by model names.
This operator has implemented pre-trained models from [TimeSformer](https://arxiv.org/abs/2102.05095)
This operator has implemented pre-trained models from [VideoSwinTransformer](https://arxiv.org/abs/2106.13230)
and maps vectors with labels.
<br />
## Code Example
Use the pretrained TimeSformer model ('timesformer_k400_8x224')
Use the pretrained VideoSwinTransformer model ('swin_tiny_patch244_window877_kinetics400_1k')
to classify and generate a vector for the given video path './archery.mp4' ([download](https://dl.fbaipublicfiles.com/pytorchvideo/projects/archery.mp4)).
*Write the pipeline in simplified style*:
@ -25,9 +25,8 @@ to classify and generate a vector for the given video path './archery.mp4' ([dow
import towhee
(
towhee.dc(['./demo_video.mp4'])
.video_decode.ffmpeg(sample_type='uniform_temporal_subsample', args={'num_samples': 4})
.runas_op(func=lambda x: [y for y in x])
towhee.glob('./archery.mp4')
.video_decode.ffmpeg()
.action_classification.video_swin_transformer(model_name='swin_tiny_patch244_window877_kinetics400_1k')
.show()
)
@ -42,9 +41,8 @@ import towhee
import towhee
(
towhee.dc['path']('./demo_video.mp4')
.video_decode.ffmpeg['path', 'frames'](sample_type='uniform_temporal_subsample', args={'num_samples': 4})
.runas_op['frames', 'frames'](func=lambda x: [y for y in x])
towhee.glob['path']('./archery.mp4')
.video_decode.ffmpeg['path', 'frames']()
.action_classification.video_swin_transformer['frames', ('labels', 'scores', 'features')](model_name='swin_tiny_patch244_window877_kinetics400_1k')
.select['path', 'labels', 'scores', 'features']()
.show(formatter={'path': 'video_path'})

BIN
result1.png

Binary file not shown.

Before

Width:  |  Height:  |  Size: 50 KiB

After

Width:  |  Height:  |  Size: 13 KiB

BIN
result2.png

Binary file not shown.

Before

Width:  |  Height:  |  Size: 153 KiB

After

Width:  |  Height:  |  Size: 89 KiB

Loading…
Cancel
Save