Configuration for the reader to be deployed in Titan Takeoff API.
ReaderConfig()The name of the model to use
The device to use for inference, cuda or cpu
The consumer group to place the reader into
The number of gpus you would like your model to be split across
The maximum sequence length to use for inference, defaults to 512
The max batch size for continuous batching of requests