Enter to send·Shift+Enter new line
KonlpyTextSplitter( self, separator: str = '\n\n', kwargs: Any = {} )
TextSplitter
Create a list of Document objects from a list of texts.
Document
Split documents.
Text splitter that uses Hugging Face tokenizer to count length.
Text splitter that uses tiktoken encoder to count length.
tiktoken
Transform sequence of documents by splitting them.
separator
str
'\n\n'
Splitting text using Konlpy package.
It is good for splitting Korean text.
The separator to use when combining splits.