| Name | Type | Description |
|---|---|---|
base_url | str | Default: 'http://localhost'The base URL where the Takeoff
Inference Server is listening. Defaults to |
port | int | Default: 3000What port is Takeoff Inference API listening on. Defaults to 3000. |
mgmt_port | int | Default: 3001What port is Takeoff Management API listening on. Defaults to 3001. |
streaming | bool | Default: False |
models | List[ReaderConfig] | Default: [] |
Titan Takeoff API LLMs.
Titan Takeoff is a wrapper to interface with Takeoff Inference API for generative text to text language models.
You can use this wrapper to send requests to a generative language model and to deploy readers with Takeoff.
Whether you want to by default use the generate_stream endpoint over generate to stream responses. Defaults to False. In reality, this is not significantly different as the streamed response is buffered and returned similar to the non-streamed response, but the run manager is applied per token generated.
Any readers you'd like to spin up on. Defaults to [].
