| Name | Type | Description |
|---|---|---|
model_id* | str | Path for the huggingface repo id to be downloaded or the huggingface checkpoint folder. |
tokenizer_id | Optional[str] | Default: NonePath for the huggingface repo id to be downloaded or the huggingface checkpoint folder which contains the tokenizer. |
load_in_4bit | bool | Default: True |
load_in_low_bit | Optional[str] | Default: None |
model_kwargs | Optional[dict] | Default: None |
kwargs | Any | Default: {} |
Construct object from model_id
"Whether to load model in 4bit.
Unused if load_in_low_bit is not None.
Which low bit precisions to use when loading model.
Example values: 'sym_int4', 'asym_int4', 'fp4', 'nf4', 'fp8', etc.
Overrides load_in_4bit if specified.
Keyword arguments to pass to the model and tokenizer.
Extra arguments to pass to the model and tokenizer.
