logo
client
repo-copy-icon

copied

Browse Source

Add triton client

Signed-off-by: shiyu22 <shiyu.chen@zilliz.com>
main
shiyu22 3 years ago
parent
commit
9bffc0af10
  1. 81
      README.md
  2. 5
      __init__.py
  3. 34
      client.py
  4. 2
      requirements.txt

81
README.md

@ -1,3 +1,80 @@
# client
# Remote Operator
2
*author: shiyu*
<br />
## Desription
Remote triton server.
<br />
## Code Example
*Run with ops:*
```python
from towhee.dc2 import ops
c = ops.triton.client('<your-ip>:<your-port>')
res = c('<your-data>')
```
*Run with pipeline:*
```python
from towhee.dc2 import ops, pipe
p = (pipe.input('data')
.map('data', 'res', ops.triton.client('<your-ip>:<your-port>'))
.output('res'))
p('<your-data>').get()
```
<br />
## Factory Constructor
Create the operator via the following factory method:
***towhee.remote(uri, mode='infer', model_name='pipeline', protocol='grpc')***
**Parameters:**
***url:*** *str*
IP address and port for the triton server, such as '<host>:<port>' and '127.0.0.1:8001'.
***model_name:*** *str*
The name of the model to run inference, defaults to 'pipline'.
<br />
## Interface
**Parameters:**
***data:***
The data to your triton server.
**Returns:**
Return the results in triton.

5
__init__.py

@ -0,0 +1,5 @@
from .client import Client
def remote(*args, **kwargs):
return Client(*args, **kwargs)

34
client.py

@ -0,0 +1,34 @@
import logging
from towhee.operator import PyOperator
from towhee import triton_client
log = logging.getLogger()
class Client(PyOperator):
"""
Triton client class.
Args:
url: str
IP address and https port for the triton server, such as '<host>:<port>', '127.0.0.1:8001'.
model_name: str
The name of the model to run inference, defaults to 'pipline'.
"""
def __init__(self, url, model_name='pipeline'):
super().__init__()
self._client = triton_client(url, model_name)
def __call__(self, *data):
"""
A one line summary of this function.
Args:
data:
This data to your triton server.
"""
return self._client(*data)
def batch(self, data_list):
return self._client.batch(data_list)

2
requirements.txt

@ -0,0 +1,2 @@
towhee
tritonclient[all]
Loading…
Cancel
Save