# SentenceTransformersTokenTextSplitter

> **Class** in `langchain_text_splitters`

📖 [View in docs](https://reference.langchain.com/python/langchain-text-splitters/sentence_transformers/SentenceTransformersTokenTextSplitter)

Splitting text to tokens using sentence model tokenizer.

## Signature

```python
SentenceTransformersTokenTextSplitter(
    self,
    chunk_overlap: int = 50,
    model_name: str = 'sentence-transformers/all-mpnet-base-v2',
    tokens_per_chunk: int | None = None,
    model_kwargs: dict[str, Any] | None = None,
    **kwargs: Any = {},
)
```

## Parameters

| Name | Type | Required | Description |
|------|------|----------|-------------|
| `chunk_overlap` | `int` | No | The number of tokens to overlap between chunks. (default: `50`) |
| `model_name` | `str` | No | The name of the sentence transformer model to use. (default: `'sentence-transformers/all-mpnet-base-v2'`) |
| `tokens_per_chunk` | `int \| None` | No | The number of tokens per chunk.  If `None`, uses the maximum tokens allowed by the model. (default: `None`) |
| `model_kwargs` | `dict[str, Any] \| None` | No | Additional parameters for model initialization. Parameters of sentence_transformers.SentenceTransformer can be used. (default: `None`) |

## Extends

- `TextSplitter`

## Constructors

```python
__init__(
    self,
    chunk_overlap: int = 50,
    model_name: str = 'sentence-transformers/all-mpnet-base-v2',
    tokens_per_chunk: int | None = None,
    model_kwargs: dict[str, Any] | None = None,
    **kwargs: Any = {},
) -> None
```

| Name | Type |
|------|------|
| `chunk_overlap` | `int` |
| `model_name` | `str` |
| `tokens_per_chunk` | `int \| None` |
| `model_kwargs` | `dict[str, Any] \| None` |


## Properties

- `model_name`
- `tokenizer`

## Methods

- [`split_text()`](https://reference.langchain.com/python/langchain-text-splitters/sentence_transformers/SentenceTransformersTokenTextSplitter/split_text)
- [`count_tokens()`](https://reference.langchain.com/python/langchain-text-splitters/sentence_transformers/SentenceTransformersTokenTextSplitter/count_tokens)

---

[View source on GitHub](https://github.com/langchain-ai/langchain/blob/9f232caa7a8fe1ca042a401942d5d90d54ceb1a6/libs/text-splitters/langchain_text_splitters/sentence_transformers.py#L20)