Text Loader

author: shiyu22

Description

Text loader is used to load text file. It supports loading data from url or file path(file format as .md or .txt).

Code Example

from towhee import pipe, ops, DataCollection

p = (
    pipe.input('url')
        .map('url', 'text', ops.text_loader())
        .output('url', 'text')
    )

res = p('https://github.com/towhee-io/towhee/blob/main/README.md')
DataCollection(res).show()

Factory Constructor

Create the operator via the following factory method

towhee.text_loader()

Interface

The operator load the documentation, then split incoming the text and return chunks.

Parameters:

​ data_src: str

​ Path or url of the document to be loaded.

Return: str

String data with the text.