Runtime Filters

Runtime filtering allows you to control which models are visible at load-time and during runtime without rebuilding the packaged snapshot.

Overview

LLMDB separates build-time and load-time concerns:

• Build-time (Engine): Produces complete, unfiltered snapshots from sources
• Load-time (LLMDB.load): Applies filters and builds indexes
• Runtime (Runtime.apply): Updates filters dynamically without reload

The base_models list stored in the snapshot enables widening filters later without rebuilding from sources.

Configuration Model

Keys

Configure filters using the :filter key in your application config:

config :llm_db,
  filter: %{
    allow: :all | %{provider_atom => [pattern_strings]},
    deny: %{provider_atom => [pattern_strings]}
  }

Semantics

Provider Keys:

• Accept atoms (:openai) or strings ("openai")
• Unknown providers are warned and ignored
• Must correspond to existing provider atoms

Pattern Types:

• Glob: "gpt-4*", "*-preview", "claude-3-haiku-*"
• Regex: ~r/gpt-4.*/, ~r/claude-3-\w+-20240307/
• Globs compile to anchored regex for matching

Common Recipes

Allow only Haiku models across providers

config :llm_db,
  filter: %{
    allow: %{
      anthropic: ["claude-3-haiku-*"],
      openrouter: ["anthropic/claude-3-haiku-*"]
    },
    deny: %{}
  }

Allow all except preview/beta models

config :llm_db,
  filter: %{
    allow: :all,
    deny: %{
      openai: ["*-preview", "*-beta"],
      google: ["*-experimental"]
    }
  }

Block a provider entirely

config :llm_db,
  filter: %{
    allow: %{
      openai: [],  # Empty list blocks provider
      anthropic: ["claude-3-*"]
    },
    deny: %{}
  }

Alternatively, simply omit the provider from a non-empty allow map:

config :llm_db,
  filter: %{
    allow: %{
      anthropic: ["claude-3-*"]
      # openai implicitly blocked (not in allow map)
    },
    deny: %{}
  }

Carve out exceptions with deny

config :llm_db,
  filter: %{
    allow: %{anthropic: ["claude-3-haiku-*"]},
    deny: %{anthropic: ["*-legacy"]}
  }

This allows all Haiku models except legacy versions.

Runtime override to switch families

Start with one configuration:

# config/runtime.exs
config :llm_db,
  filter: %{
    allow: %{anthropic: ["claude-3-haiku-*"]},
    deny: %{}
  }

Then override at runtime to switch to different models:

{:ok, _snapshot} = LLMDB.load(
  runtime_overrides: %{
    filter: %{
      allow: %{
        anthropic: ["claude-3.5-sonnet-*", "claude-3-opus-*"]
      },
      deny: %{}
    }
  }
)

Programmatic update without reload

For hot updates during runtime without calling load/1:

# Get current snapshot
snapshot = LLMDB.Store.snapshot()

# Apply new filters
{:ok, updated_snapshot} = LLMDB.Runtime.apply(snapshot, %{
  filter: %{
    allow: %{openai: ["gpt-4o-*"]},
    deny: %{openai: ["*-preview"]}
  }
})

# Update store (atomic swap)
LLMDB.Store.put!(updated_snapshot)

The Runtime.apply/2 function:

• Recompiles filters and reapplies to base_models
• Rebuilds indexes (models_by_key, aliases_by_key, models_by_provider)
• Enables filter widening because it uses the full base_models list
• Returns {:ok, snapshot} or {:error, reason}

Runtime Overrides

Via LLMDB.load/1

Pass runtime_overrides to override config at load time:

{:ok, _snapshot} = LLMDB.load(
  runtime_overrides: %{
    filter: %{allow: %{...}, deny: %{...}},
    prefer: [:openai, :anthropic]
  }
)

Runtime overrides take precedence over application config.

Via LLMDB.Runtime.apply/2

For updates without reloading:

snapshot = LLMDB.Store.snapshot()

{:ok, updated_snapshot} = LLMDB.Runtime.apply(snapshot, %{
  filter: %{...}
})

LLMDB.Store.put!(updated_snapshot)

Error Handling and Troubleshooting

"Filters eliminated all models"

Error:

{:error, "llm_db: filters eliminated all models (allow: ..., deny: ...). 
 Use allow: :all to widen filters or remove deny patterns."}

Causes:

• Allow patterns match no models
• All models are denied
• Unknown providers in allow map (silently ignored, leaving no valid providers)

Solutions:

1. Check provider names for typos
2. Verify patterns match actual model IDs
3. Use allow: :all to widen filters
4. Remove or adjust deny patterns

"Unknown providers in filter"

Warning:

llm_db: unknown provider(s) in filter: [:unknwon_provider]. 
Known providers: [:openai, :anthropic, ...]. 
Check spelling or remove unknown providers from configuration.

Causes:

• Typo in provider name
• Provider doesn't exist in snapshot
• String not converted to existing atom

Solutions:

1. Check spelling: :openai not :open_ai
2. Verify provider exists: LLMDB.providers() or LLMDB.list_providers()
3. Remove unknown provider from config

Nothing shows for provider X

With a non-empty allow map, providers must be explicitly listed:

# This BLOCKS openai (not in allow map)
config :llm_db,
  filter: %{
    allow: %{anthropic: ["claude-3-*"]},  # Only anthropic allowed
    deny: %{}
  }

# To include openai, add it to allow:
config :llm_db,
  filter: %{
    allow: %{
      anthropic: ["claude-3-*"],
      openai: ["gpt-4*"]
    },
    deny: %{}
  }

Check what's allowed

LLMDB.allowed?("openai:gpt-4o-mini")  
#=> true or false

LLMDB.allowed?({:openai, "gpt-4o-preview"})
#=> false (if denied by pattern)

{:ok, model} = LLMDB.model("openai:gpt-4o-mini")
LLMDB.allowed?(model)
#=> true

Safety and Performance

Provider Key Safety

• Provider keys use String.to_existing_atom/1 to prevent atom leaks
• Unknown string keys are ignored with warnings
• Atom keys are validated against known providers
• Safe for use in production with untrusted config sources

Pattern Performance

• Patterns compile once at load/override time
• Runtime matching is O(patterns-for-provider) per allowed?/1 call
• Regex patterns are compiled and cached
• Typical pattern counts (<100 per provider) have negligible overhead

Recommendations:

• Prefer glob patterns ("gpt-4*") over complex regex for readability
• Avoid untrusted user input for Regex patterns (config should be trusted)
• For thousands of patterns, consider pre-filtering at source level

Filter Widening

Runtime filter updates can widen or narrow because they operate on base_models:

# Start narrow
LLMDB.load(runtime_overrides: %{
  filter: %{allow: %{anthropic: ["claude-3-haiku-*"]}, deny: %{}}
})
#=> Only Haiku models visible

# Widen later
snapshot = LLMDB.Store.snapshot()
{:ok, snapshot} = LLMDB.Runtime.apply(snapshot, %{
  filter: %{allow: %{anthropic: ["claude-3-*"]}, deny: %{}}
})
LLMDB.Store.put!(snapshot)
#=> All Claude 3 models now visible (pulled from base_models)

Edge Cases

Empty allow map

config :llm_db,
  filter: %{allow: %{}, deny: %{}}

This behaves like allow: :all (map size is 0).

Allow with only unknown providers

config :llm_db,
  filter: %{
    allow: %{nonexistent_provider: ["model-*"]},
    deny: %{}
  }

After filtering out unknown providers, allow becomes %{} (empty map), which acts like :all. You'll see a warning but load will succeed.

To actually restrict to specific providers, use known provider names:

config :llm_db,
  filter: %{
    allow: %{anthropic: ["claude-3-haiku-*"]},
    deny: %{}
  }

Regex safety

Regex from application config is trusted. Avoid accepting user-supplied Regex patterns:

# Safe: config file
config :llm_db,
  filter: %{allow: %{openai: [~r/gpt-4.*/]}, deny: %{}}

# Unsafe: user input (don't do this)
user_pattern = ~r/#{user_input}.*/  # Potential ReDoS attack

Stick to glob patterns for user-facing configuration.

Migration from Old Format

If you previously used top-level :allow and :deny keys:

Old:

config :llm_db,
  allow: %{openai: ["gpt-4*"]},
  deny: %{openai: ["*-preview"]}

New:

config :llm_db,
  filter: %{
    allow: %{openai: ["gpt-4*"]},
    deny: %{openai: ["*-preview"]}
  }

The old format is no longer supported as of version 2025.11.7+. Update your configuration to use the singular :filter key.