ChatDeepInfra

Model name to use.

URL to use for the API call.

Run inference with this temperature. Must be in the closed interval [0.0, 1.0].

Holds any model parameters valid for API call not explicitly specified.

Decode using nucleus sampling: consider the smallest set of tokens whose probability sum is at least top_p. Must be in the closed interval [0.0, 1.0].

Decode using top-k sampling: consider the set of top_k most probable tokens. Must be positive.

Number of chat completions to generate for each prompt. Note that the API may not return the full n completions if duplicates are generated.

Use tenacity to retry the completion call.

Use tenacity to retry the async completion call.

Validate api key, python package exists, temperature, top_p, and top_k.

Bind tool-like objects to this chat model.

Assumes model is compatible with OpenAI tool-calling API.