CTranslate2

Path to the CTranslate2 model directory.

Name of the original Hugging Face model needed to load the proper tokenizer.

Device to use (possible values are: cpu, cuda, auto).

Device IDs where to place this generator on.

Model computation type or a dictionary mapping a device name to the computation type (possible values are: default, auto, int8, int8_float32, int8_float16, int8_bfloat16, int16, float16, bfloat16, float32).

Maximum generation length.

Randomly sample predictions from the top K candidates.

Keep the most probable tokens whose cumulative probability exceeds this value.

Sampling temperature to generate more random samples.

Holds any model parameters valid for ctranslate2.Generator call not explicitly specified.

Validate that python package exists in environment.