Ask a question to get started
Enter to sendā¢Shift+Enter new line
get_tokenizer() -> Any
Get a GPT-2 tokenizer instance.
This function is cached to avoid re-loading the tokenizer every time it is called.