Module chat_completions 

Define APIs for chat completion.

Structs

ModelLockGuard

RAII guard for model lock. Automatically releases the lock when dropped. Used by non-stream mode to ensure lock release even on early return or panic.

ModelStreamLock

Per-model stream lock state. Each model has its own lock to allow parallel inference across different models.

Functions

chat

Processes a chat-completion request and returns either a stream of ChatCompletionChunk instances or a ChatCompletionObject instance.

get_default_model_lock

Get the lock for the default model (first available model). Used when the request does not specify a model name.

get_or_create_model_lock

Get or create a stream lock for the specified model.