Enter to send·Shift+Enter new line
max_batch_size: int = 4
The max batch size for continuous batching of requests
