Remote Operator

author: shiyu

Desription

Remote triton server.

Code Example

Run with ops:

from towhee.dc2 import ops

c = ops.triton.client('<your-ip>:<your-port>')
res = c('<your-data>')

Run with pipeline:

from towhee.dc2 import ops, pipe

p = (pipe.input('data')
        .map('data', 'res', ops.triton.client('<your-ip>:<your-port>'))
        .output('res'))
        
p('<your-data>').get()

Factory Constructor

Create the operator via the following factory method:

towhee.remote(uri, mode='infer', model_name='pipeline', protocol='grpc')

Parameters:

url: str

IP address and port for the triton server, such as ':' and '127.0.0.1:8001'.

model_name: str

The name of the model to run inference, defaults to 'pipline'.

Interface

Parameters:

data:

The data to your triton server.

Returns:

Return the results in triton.

More Resources

• Multimodal RAG with Milvus and GPT-4o: Join us for a webinar for a demo of multimodal RAG with Milvus and GPT-4o
• The Journey to Optimizing Billion-scale Image Search (2/2) - Zilliz blog: A case study with UPYUN, part II
• Multimodal RAG with Milvus and GPT-4o: Join us for a webinar for a demo of multimodal RAG with Milvus and GPT-4o
• The GUI for Milvus - Attu: Attu is an all-in-one Milvus administration tool, enabling you to dramatically reduce the DevOps cost of managing Milvus.