Call TensorFlow Lite models from Elixir / BEAM, with full vendor accelerator access on phones — Apple Neural Engine on iOS, MediaTek / Qualcomm GPU+NPU HALs on Android.

This is NOT an Nx.Backend

NxTfliteMob does not replace Nx.BinaryBackend, EMLX.Backend, NxVulkan.Backend, etc. There is no NxTfliteMob.Backend module to set via Nx.global_default_backend/1.

TFLite executes pre-compiled model graphs (.tflite files) end-to-end through vendor-optimised delegates. The whole graph stays opaque so the delegate can fuse + schedule it for ANE / GPU / NPU. You don't compose your own ops here — you call a pre-trained model.

Use NxTfliteMob when you have a pre-trained model to run. Use Nx backends when you're writing arbitrary tensor math in Elixir.

Both can coexist in the same app.

API surface

Three functions: load_module/2, call/2, release_module/1.

iex> tflite = File.read!("priv/yolov8n_float16.tflite")
iex> {:ok, handle} = NxTfliteMob.load_module(tflite,
...>   delegate: "coreml", coreml_ane_only: false)
iex> {:ok, [output_bytes]} = NxTfliteMob.call(handle, [input_bytes])
iex> NxTfliteMob.release_module(handle)
:ok

See the YOLO walkthrough for a complete end-to-end example with input prep, inference, and output decode.

Delegate options

The delegate opt selects how the model graph runs. Per-platform recommendations (see also the delegates guide):

Android — delegate: "nnapi"

NNAPI is Android's neural-net dispatch API. It picks a vendor HAL driver based on the accelerator name:

Discover available accelerators on a connected device with adb shell + the standalone bench CLI's list-nnapi mode (see the package's scripts/bench_android/).

Other Android opts:

• num_threads: — XNNPACK CPU thread count (default 6)
• allow_fp16: — let NNAPI run FP32 ops in FP16 (default true)

iOS — delegate: "coreml"

Core ML routes the delegated portion through Apple's Core ML framework, which internally schedules to the Apple Neural Engine when ops are supported. For YOLOv8n FP16, ~56% of nodes delegate to the ANE on an iPhone SE 3rd gen A15 (the rest fall to CPU via XNNPACK), hitting 24 ms per inference.

Caveats:

• INT8 + Core ML doesn't work. Core ML's tooling doesn't understand the Ultralytics INT8 quant flavour — 0/256 nodes delegate. Use the FP16 model variant for Core ML.
• coreml_ane_only: (default false) — when true, the delegate returns nil instead of falling back to CPU on devices without an ANE. Useful for "ANE-only or skip" logic; irrelevant on A11+ devices where the ANE is always present.

iOS — delegate: "metal" (planned)

TFLite ships TensorFlowLiteCMetal.xcframework for Metal GPU inference but the current NIF doesn't expose it as a delegate: option yet. PR welcome. Core ML is usually faster anyway on Apple Silicon devices (Core ML can pick GPU when ANE ops are unsupported).

XNNPACK CPU — delegate: "xnnpack" (default)

Bundled into TFLite. Highly-optimised CPU+SIMD path. Default when no other delegate is set. Surprisingly competitive on modern phones — ~77 ms on the Moto G Power 5G (tied with the GPU path) and 27-37 ms on iPhone SE 3rd gen A15. Use this when:

• You're on a device without GPU/NPU acceleration
• The vendor delegate fails to delegate (e.g. INT8 + Core ML)
• You want deterministic, reproducible numbers (CPU paths don't thermal-throttle as aggressively as GPUs)

Input + output byte layout

call/2 is raw-bytes-in, raw-bytes-out. The byte layout is model-specific — you have to match what the .tflite model expects.

Inspect a model's expected shape/dtype via mix Python helpers or TFLite's flatc tool. Or :erlang.load_nif/2 an inspector NIF built against TFLite's TfLiteInterpreterGetInputTensor — exposing this in the Elixir API is on the roadmap.

Common shapes:

See the YOLO walkthrough for the layout-aware decoder we use in production (pure-BEAM, 13 ms for the full INT8 NMS pass).

Where Nx fits in (optionally)

You CAN use Nx tensors on either side of call/2. It's optional — bytes-in/bytes-out is the canonical interface.

Input prep with Nx:

input_bytes =
  camera_frame_f32_binary
  |> Nx.from_binary(:f32)
  |> Nx.reshape({1, 640, 640, 3})
  |> Nx.as_type(:s8)      # quantize for INT8 model
  |> Nx.to_binary()

{:ok, [out]} = NxTfliteMob.call(handle, [input_bytes])

Output decode with Nx:

detections =
  out
  |> Nx.from_binary(:s8)
  |> Nx.reshape({1, 84, 8400})
  |> Nx.as_type(:f32)
  |> Nx.multiply(scale)
  |> Nx.subtract(zero_point)
  |> extract_detections()

In practice we bypass Nx for performance-critical decoding — Nx.BinaryBackend for an argmax across {80, 8400} is 1700 ms; a pure-BEAM :binary.at/2 loop is 13 ms (130× faster). See NxeigenProbe.LiveYoloScreen for the pure-BEAM decoder pattern.

Using with Mob

If you're building a Mob app, the easiest path is mob_dev's Igniter task:

mix mob.enable tflite

This adds the dep + generates a per-platform default-opts helper and registers the NIF in mob_dev's static-NIF table. Requires mob_dev >= 0.5.9. See mob_dev's mob.enable docs for details.

After mix mob.enable tflite, the auto-generated helper picks delegate opts per platform:

{:ok, h} = NxTfliteMob.load_module(model_bytes,
             MyApp.TfliteInit.default_opts())

Building from source (non-Mob)

See the package's Makefile — targets android, ios_device, ios_sim, mac. Each requires platform-appropriate TFLite distribution (cached at ~/.mob/cache/ by mob_dev's downloader, or per-target overrides for standalone builds).

Mac builds require building libtensorflowlite_c.dylib from TF source first — TFLite has no Mac arm64 prebuilt. See docs/build_mac_tflite.md in the repo.