View Source OnnxInterp (onnx_interp v0.1.2)

Onnx runtime intepreter for Elixir. Deep Learning inference framework.

basic-usage

Basic Usage

You get the trained onnx model and save it in a directory that your application can read. "your-app/priv" may be good choice.

$ cp your-trained-model.onnx ./priv

Next, you will create a module that interfaces with the deep learning model. The module will need pre-processing and post-processing in addition to inference processing, as in the example following. OnnxInterp provides inference processing only.

You put use OnnxInterp at the beginning of your module, specify the model path as an optional argument. In the inference section, you will put data input to the model (OnnxInterp.set_input_tensor/3), inference execution (OnnxInterp.invoke/1), and inference result retrieval (OnnxInterp.get_output_tensor/2).

defmodule YourApp.YourModel do
  use OnnxInterp, model: "priv/your-trained-model.onnx"

  def predict(data) do
    # preprocess
    #  to convert the data to be inferred to the input format of the model.
    input_bin = convert-float32-binaries(data)

    # inference
    #  typical I/O data for Onnx models is a serialized 32-bit float tensor.
    output_bin =
      __MODULE__
      |> OnnxInterp.set_input_tensor(0, input_bin)
      |> OnnxInterp.invoke()
      |> OnnxInterp.get_output_tensor(0)

    # postprocess
    #  add your post-processing here.
    #  you may need to reshape output_bin to tensor at first.
    tensor = output_bin
      |> Nx.from_binary({:f, 32})
      |> Nx.reshape({size-x, size-y, :auto})

    * your-postprocessing *
    ...
  end
end

Link to this section Summary

Functions

Get the flat binary from the output tensor on the interpreter.

Get the propaty of the tflite model.

Invoke prediction.

Execute the inference session. In session mode, data input/execution of inference/output of results to the DL model is done all at once.

Put a flat binary to the input tensor on the interpreter.

Stop the tflite interpreter.

Link to this section Functions

Link to this function

get_output_tensor(mod, index)

View Source

Get the flat binary from the output tensor on the interpreter.

parameters

Parameters

  • mod - modules' names or session.
  • index - index of output tensor in the model

Get the propaty of the tflite model.

parameters

Parameters

  • mod - modules' names

Invoke prediction.

parameters

Parameters

  • mod - modules' names
Link to this function

non_max_suppression_multi_class( mod, arg, boxes, scores, iou_threshold \\ 0.5, score_threshold \\ 0.25, sigma \\ 0.0 )

View Source

Execute post processing: nms.

parameters

Parameters

  • mod - modules' names
  • num_boxes - number of candidate boxes
  • num_class - number of category class
  • boxes - binaries, serialized boxes tensor[num_boxes][4]; dtype: float32
  • scores - binaries, serialized score tensor[num_boxes][num_class]; dtype: float32
  • iou_threshold - IOU threshold
  • score_threshold - score cutoff threshold
  • sigma - soft IOU parameter

Execute the inference session. In session mode, data input/execution of inference/output of results to the DL model is done all at once.

parameters

Parameters

  • session - session.

examples

Examples.

    output_bin =
      session()
      |> OnnxInterp.set_input_tensor(0, input_bin)
      |> OnnxInterp.run()
      |> OnnxInterp.get_output_tensor(0)
Link to this function

set_input_tensor(mod, index, bin, opts \\ [])

View Source

Put a flat binary to the input tensor on the interpreter.

parameters

Parameters

  • mod - modules' names or session.
  • index - index of input tensor in the model
  • bin - input data - flat binary, cf. serialized tensor
  • opts - data conversion

Stop the tflite interpreter.

parameters

Parameters

  • mod - modules' names