logo
You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
Readme
Files and versions

106 lines
2.1 KiB

# Video Alignment with Temporal Network
*author: David Wang*
<br />
## Description
This operator can compare two ordered sequences, then detect the range which features from each sequence are computationally similar in order.
<br />
## Code Example
```python
from towhee.dc2 import pipe, ops, DataCollection
import numpy as np
# simulate a video feature by 10 frames of 512d vectors.
videos_embeddings = np.random.randn(10,512)
videos_embeddings = videos_embeddings / np.linalg.norm(videos_embeddings,axis=1).reshape(10,-1)
p = (
pipe.input('src', 'dest') \
.map(('src', 'dest'), ('range', 'range_score'), ops.video_copy_detection.temporal_network()) \
.output('src', 'dest', 'range', 'range_score')
)
DataCollection(p(videos_embeddings, videos_embeddings)).show()
```
<img src="./tabular.png" alt="result" style="height:60px;"/>
<br />
## Factory Constructor
Create the operator via the following factory method
***clip(model_name, modality)***
***temporal_network(tn_max_step, tn_top_k, max_path, min_sim, min_length, max_iou)***
**Parameters:**
***tn_max_step:*** *str*
​ Max step range in TN.
***tn_top_k:*** *str*
​ Top k frame similarity selection in TN.
***max_path:*** *str*
​ Max loop for multiply segments detection.
***min_sim:*** *str*
​ Min average similarity score for each aligned segment.
***min_length:*** *str*
​ Min segment length.
***max_iout:*** *str*
​ Max iou for filtering overlap segments (bbox).
<br />
## Interface
A Temporal Network operator takes two numpy.ndarray(shape(N,D) N: number of features. D: dimension of features) and get the duplicated ranges and scores.
**Parameters:**
***src_video_vec*** *numpy.ndarray*
​ Source video feature vectors.
***dst_video_vec:*** *numpy.ndarray*
​ Destination video feature vectors.
**Returns:**
***aligned_ranges:*** *List[List[Int]]*
​ The returned aligned range.
***aligned_scores:*** *List[float]*
​ The returned similarity scores(length same as aligned_ranges).
2 years ago