From ac3789523ceaae4ed5925c24eee4d3649196b896 Mon Sep 17 00:00:00 2001 From: shiyu22 Date: Mon, 29 May 2023 17:11:18 +0800 Subject: [PATCH] Add README Signed-off-by: shiyu22 --- README.md | 22 ++++++++++++---------- 1 file changed, 12 insertions(+), 10 deletions(-) diff --git a/README.md b/README.md index a57d747..e7d5b08 100644 --- a/README.md +++ b/README.md @@ -8,8 +8,6 @@ ### Description -**Text Loader** is used to load the documents and split it to a list of text. - **Text loader** is used to load files and split them into text lists. It supports loading local files (with file path), or web links (with url). > Refer to [Recursive Characters](https://python.langchain.com/en/latest/modules/indexes/text_splitters/examples/recursive_text_splitter.html) for the operation of splitting text. @@ -20,8 +18,6 @@ ### Code Example -Writing the pipeline in the simplified way - ```Python from towhee import pipe, ops, DataCollection @@ -47,6 +43,16 @@ Create the operator via the following factory method ***towhee.text_loader(chunk_size=300, source_type='file')*** +**Parameters:** + +​ ***chunk_size***: int + +​ The size of each chunk, defaults to 300. + +​ ***source_type***: str + +​ The type of the soure, defaults to 'file', you can also set to 'url' for you url of your documentation. +
@@ -57,13 +63,9 @@ The operator load the documentation, then split incoming the text and return chu **Parameters:** -​ ***chunk_size***: int +​ ***data_src***: str -​ The size of each chunk, defaults to 300. - -​ ***source_type***: str - -​ The type of the soure, defaults to 'file', you can also set to 'url' for you url of your documentation. +​ Path or url of the document to be loaded.