Text splitter that uses tiktoken encoder to count length.
| Name | Type | Description |
|---|---|---|
separator | str | Default: '\n\n' |
| Name | Type |
|---|---|
| separator | str |
Splitting text using Konlpy package.
It is good for splitting Korean text.
The separator to use when combining splits.