logo
You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
Readme
Files and versions

3.6 KiB

Text Loader

author: shiyu22


Description

Text loader is used to load text file. It supports loading data from url or file path(file format as .md or .txt).


Code Example

from towhee import pipe, ops, DataCollection

p = (
    pipe.input('url')
        .map('url', 'text', ops.text_loader())
        .output('url', 'text')
    )

res = p('https://github.com/towhee-io/towhee/blob/main/README.md')
DataCollection(res).show()
result


Factory Constructor

Create the operator via the following factory method

towhee.text_loader()


Interface

The operator load the documentation, then split incoming the text and return chunks.

Parameters:

data_src: str

​ Path or url of the document to be loaded.

Return: str

String data with the text.

More Resources

3.6 KiB

Text Loader

author: shiyu22


Description

Text loader is used to load text file. It supports loading data from url or file path(file format as .md or .txt).


Code Example

from towhee import pipe, ops, DataCollection

p = (
    pipe.input('url')
        .map('url', 'text', ops.text_loader())
        .output('url', 'text')
    )

res = p('https://github.com/towhee-io/towhee/blob/main/README.md')
DataCollection(res).show()
result


Factory Constructor

Create the operator via the following factory method

towhee.text_loader()


Interface

The operator load the documentation, then split incoming the text and return chunks.

Parameters:

data_src: str

​ Path or url of the document to be loaded.

Return: str

String data with the text.

More Resources