Enhanced telemetry and metrics collection for LlmGuard.

Provides comprehensive metrics tracking including:

• Detection latency percentiles (P50, P95, P99)
• Detection outcome rates (safe/detected/error)
• Detector-specific performance metrics
• Cache hit rates and efficiency
• Error categorization and tracking

Metrics Emitted

Counter Metrics

• llm_guard.requests.total - Total requests processed
• llm_guard.detections.total - Total detections by category
• llm_guard.errors.total - Total errors by type
• llm_guard.cache.hits - Cache hits by type
• llm_guard.cache.misses - Cache misses by type

Distribution Metrics

• llm_guard.request.duration - Request duration histogram
• llm_guard.detector.duration - Per-detector duration
• llm_guard.detection.confidence - Confidence score distribution

Usage

# Setup telemetry handlers
LlmGuard.Telemetry.Metrics.setup()

# Metrics are automatically emitted during detection

# Get current metrics snapshot
metrics = LlmGuard.Telemetry.Metrics.snapshot()

# Export Prometheus-format metrics
prometheus = LlmGuard.Telemetry.Metrics.prometheus_metrics()

Integration

Telemetry.Metrics

import Telemetry.Metrics

defp metrics do
  [
    # Counter metrics
    counter("llm_guard.requests.total"),

    # Distribution metrics with percentiles
    distribution("llm_guard.request.duration",
      unit: {:native, :millisecond},
      reporter_options: [
        buckets: [10, 50, 100, 500, 1000, 5000]
      ]
    )
  ]
end

Prometheus Integration

# In your supervision tree
children = [
  {TelemetryMetricsPrometheus, metrics: LlmGuard.Telemetry.Metrics.metrics()}
]