logo
Browse Source

Add README

Signed-off-by: shiyu22 <shiyu.chen@zilliz.com>
main
shiyu22 2 years ago
parent
commit
ac3789523c
  1. 22
      README.md

22
README.md

@ -8,8 +8,6 @@
### Description
**Text Loader** is used to load the documents and split it to a list of text.
**Text loader** is used to load files and split them into text lists. It supports loading local files (with file path), or web links (with url).
> Refer to [Recursive Characters](https://python.langchain.com/en/latest/modules/indexes/text_splitters/examples/recursive_text_splitter.html) for the operation of splitting text.
@ -20,8 +18,6 @@
### Code Example
Writing the pipeline in the simplified way
```Python
from towhee import pipe, ops, DataCollection
@ -47,6 +43,16 @@ Create the operator via the following factory method
***towhee.text_loader(chunk_size=300, source_type='file')***
**Parameters:**
***chunk_size***: int
​ The size of each chunk, defaults to 300.
***source_type***: str
​ The type of the soure, defaults to 'file', you can also set to 'url' for you url of your documentation.
<br />
@ -57,13 +63,9 @@ The operator load the documentation, then split incoming the text and return chu
**Parameters:**
***chunk_size***: int
***data_src***: str
​ The size of each chunk, defaults to 300.
***source_type***: str
​ The type of the soure, defaults to 'file', you can also set to 'url' for you url of your documentation.
​ Path or url of the document to be loaded.

Loading…
Cancel
Save