Construct object from model_id
from_model_id(
cls,
model_id: str,
model_kwargs: Optional[dict] = None,
*,
tokenizer_id: Optional[str] = None,
load_in_4bit: bool = True,
load_in_low_bit: Optional[str] = None,
**kwargs: Any = {}
) -> LLM| Name | Type | Description |
|---|---|---|
model_id* | str | Path for the huggingface repo id to be downloaded or the huggingface checkpoint folder. |
tokenizer_id | Optional[str] | Default: NonePath for the huggingface repo id to be downloaded or the huggingface checkpoint folder which contains the tokenizer. |
load_in_4bit | bool | Default: True"Whether to load model in 4bit.
Unused if |
load_in_low_bit | Optional[str] | Default: NoneWhich low bit precisions to use when loading model.
Example values: 'sym_int4', 'asym_int4', 'fp4', 'nf4', 'fp8', etc.
Overrides |
model_kwargs | Optional[dict] | Default: NoneKeyword arguments to pass to the model and tokenizer. |
kwargs | Any | Default: {}Extra arguments to pass to the model and tokenizer. |