TextGen

The full URL to the textgen webui including http[s]://host:port

The preset to use in the textgen webui

The maximum number of tokens to generate.

Do sample

Primary factor to control randomness of outputs. 0 = deterministic (only the most likely token is used). Higher value = more randomness.

If not set to 1, select tokens with probabilities adding up to less than this number. Higher value = higher range of possible random results.

If not set to 1, select only tokens that are at least this much more likely to appear than random tokens, given the prior text.

Epsilon cutoff

ETA cutoff

Exponential penalty factor for repeating prior tokens. 1 means no penalty, higher value = less repetition, lower value = more repetition.

Similar to top_p, but select instead only the top_k most likely tokens. Higher value = higher range of possible random results.

Minimum generation length in tokens.

If not set to 0, specifies the length of token sets that are completely blocked from repeating at all. Higher values = blocks larger phrases, lower values = blocks words or letters from repeating. Only 0 or high values are a good idea in most cases.

Number of beams

Penalty Alpha

Length Penalty

Early stopping

Seed (-1 for random)

Add the bos_token to the beginning of prompts. Disabling this can make the replies more creative.

Truncate the prompt up to this length. The leftmost tokens are removed if the prompt exceeds this length. Most models require this to be at most 2048.

Ban the eos_token. Forces the model to never end the generation prematurely.

Skip special tokens. Some specific models need this unset.

A list of strings to stop generation when encountered.

Whether to stream the results, token by token.