transformers
copied
You can not select more than 25 topics
Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
Readme
Files and versions
Updated 2 years ago
sentence-embedding
Evaluation
Model performance in sentence similarity
- Download SentEval & test data
git clone https://github.com/facebookresearch/SentEval.git
cd SentEval/data/downstream
./get_transfer_data.bash
- Run test script
python transformers_test.py MODEL_NAME
QPS Test
Please note that qps_test.py
uses:
localhost:8000
: to connect triton client'Hello, world.''
: as test sentence
python qps_test --model paraphrase-albert-small-v2 --pipe --onnx --triton --num 100
Args:
--model
: mandatory, string, model name--pipe
: optional, on/off flag to enable qps test for pipe--onnx
: optional, on/off flag to enable qps test for onnx--triton
: optional, on/off flag to enable qps for triton (please make sure that triton client is ready)--num
: optional, integer, defaults to 100, batch size in each loop (10 loops in total)--device
: optional, int, defaults to -1, cuda index or use cpu when -1
Jael Gu
ea8020947c
| 31 Commits | ||
---|---|---|---|
.. | |||
README.md |
970 B
|
2 years ago | |
evaluate.py |
2.0 KiB
|
2 years ago | |
qps_test.py |
4.4 KiB
|
2 years ago | |
test_client.py |
387 B
|
2 years ago |