Struct Graph

pub struct Graph<M: BaseMetadata + Serialize + Clone + Default> {
    pub created: Duration,
    pub metadata: M,
    /* private fields */
}

Wrapper of the wasmedge_wasi_nn::Graph struct

Fields

created: Duration

metadata: M

Implementations

impl<M: BaseMetadata + Serialize + Clone + Default> Graph<M>

pub fn new(metadata: M) -> Result<Self, LlamaCoreError>

Create a new computation graph from the given metadata.

pub fn name(&self) -> &str

Get the name of the model

pub fn alias(&self) -> &str

Get the alias of the model

pub fn update_metadata(&mut self) -> Result<(), LlamaCoreError>

Update metadata

pub fn set_input<T: Sized>( &mut self, index: usize, tensor_type: TensorType, dimensions: &[usize], data: impl AsRef<[T]>, ) -> Result<(), WasiNnError>

Set input uses the data, not only u8, but also f32, i32, etc.

pub fn compute(&mut self) -> Result<(), WasiNnError>

Compute the inference on the given inputs.

pub fn compute_single(&mut self) -> Result<(), WasiNnError>

Compute the inference on the given inputs.

Note that this method is used for the stream mode. It generates one token at a time.

pub fn get_output<T: Sized>( &self, index: usize, out_buffer: &mut [T], ) -> Result<usize, WasiNnError>

Copy output tensor to out_buffer, return the output’s size in bytes.

pub fn get_output_single<T: Sized>( &self, index: usize, out_buffer: &mut [T], ) -> Result<usize, WasiNnError>

Copy output tensor to out_buffer, return the output’s size in bytes.

Note that this method is used for the stream mode. It returns one token at a time.

pub fn finish_single(&mut self) -> Result<(), WasiNnError>

Clear the computation context.

Note that this method is used for the stream mode. It clears the context after the stream mode is finished.

Trait Implementations

impl<M: Debug + BaseMetadata + Serialize + Clone + Default> Debug for Graph<M>

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl<M: BaseMetadata + Serialize + Clone + Default> Drop for Graph<M>

fn drop(&mut self)

Executes the destructor for this type.