Nasty.Statistics.Neural.Quantization.INT8 (Nasty v0.3.0)

View Source

INT8 post-training quantization for neural models.

Converts Float32 model weights to INT8 representation for:

  • 4x smaller model files
  • 2-3x faster inference on CPU
  • 40-60% lower memory usage
  • <1% accuracy degradation (with proper calibration)

Process

  1. Calibration: Run representative data through model to collect activation statistics
  2. Quantization: Convert Float32 weights to INT8 using calibration data
  3. Validation: Verify accuracy degradation is within acceptable bounds

Example

alias Nasty.Statistics.Neural.Quantization.INT8

# Load a trained model
{:ok, model} = NeuralTagger.load("pos_tagger.axon")

# Quantize with calibration data
{:ok, quantized} = INT8.quantize(model,
  calibration_data: calibration_samples,
  target_accuracy_loss: 0.01  # Max 1% accuracy loss
)

# Save quantized model
INT8.save(quantized, "pos_tagger_int8.axon")

Summary

Types

calibration_data()
model()
quantization_config()

Functions

estimate_size_reduction(model)

Estimates size reduction from quantization.

load(path)

Loads a quantized model from disk.

quantize(model, opts \\ [])

Quantizes a model to INT8 precision using post-training quantization.

save(quantized_model, path)

Saves a quantized model to disk.

Types

calibration_data()

@type calibration_data() :: [map()]

model()

@type model() :: map()

quantization_config()

@type quantization_config() :: %{
  calibration_method: :minmax | :percentile | :entropy,
  percentile: float(),
  per_channel: boolean(),
  symmetric: boolean()
}

Functions

estimate_size_reduction(model)

@spec estimate_size_reduction(model()) :: map()

Estimates size reduction from quantization.

Examples

INT8.estimate_size_reduction(model)
# => %{original_mb: 400, quantized_mb: 100, reduction: 4.0}

load(path)

@spec load(String.t()) :: {:ok, model()} | {:error, term()}

Loads a quantized model from disk.

Examples

{:ok, model} = INT8.load("model_int8.axon")

quantize(model, opts \\ [])

@spec quantize(
  model(),
  keyword()
) :: {:ok, model()} | {:error, term()}

Quantizes a model to INT8 precision using post-training quantization.

Parameters

  • model - Trained model to quantize
  • opts - Quantization options

Options

  • :calibration_data - Representative data for calibration (required)
  • :calibration_method - Method for determining quantization ranges
    • :minmax - Use min/max values (default)
    • :percentile - Use percentile ranges (more robust to outliers)
    • :entropy - Minimize KL divergence
  • :per_channel - Quantize per-channel vs per-tensor (default: true)
  • :symmetric - Use symmetric quantization (default: true)
  • :target_accuracy_loss - Max acceptable accuracy loss (default: 0.01)

Returns

  • {:ok, quantized_model} - Successfully quantized model
  • {:error, reason} - Quantization failed

save(quantized_model, path)

@spec save(model(), String.t()) :: :ok | {:error, term()}

Saves a quantized model to disk.

Examples

INT8.save(quantized_model, "model_int8.axon")