Struct ModelStreamLock 

pub struct ModelStreamLock {
    pub active: AtomicBool,
    pub waker_queue: Mutex<VecDeque<Waker>>,
}

Per-model stream lock state. Each model has its own lock to allow parallel inference across different models.

Fields

active: AtomicBool

Whether this model currently has an active stream

waker_queue: Mutex<VecDeque<Waker>>

Waker queue for requests waiting on this model

Implementations

impl ModelStreamLock

pub fn new() -> Self

Create a new model stream lock

pub fn try_acquire(&self) -> bool

Try to acquire the lock. Returns true if successful.

pub fn release(&self)

Release the lock and wake the next waiting request.

pub fn register_waker(&self, waker: &Waker)

Register a waker to be notified when the lock is released.

Trait Implementations

impl Default for ModelStreamLock

fn default() -> Self

Returns the “default value” for a type.