Release Process

View Source

Snapshot-Based Model Data

LLM Models packages a pre-built snapshot at priv/llm_db/snapshot.json.

Snapshot Structure

V2 schema uses nested providers with embedded models:

{
  "version": 2,
  "generated_at": "2025-11-06T14:23:45.123456Z",
  "providers": {
    "openai": {
      "id": "openai",
      "name": "OpenAI",
      "base_url": "https://api.openai.com/v1",
      "models": {
        "gpt-4": {
          "id": "gpt-4",
          "provider": "openai",
          "name": "GPT-4",
          "capabilities": {...},
          "limits": {...},
          "cost": {...}
        }
      }
    }
  }
}

Runtime Loading

# Reads from priv/llm_db/snapshot.json
{:ok, _} = LLMDB.load()

Compile-time embed (zero runtime IO):

# config/config.exs
config :llm_db, compile_embed: true

Snapshot is inlined into BEAM bytecode during compilation. Requires recompilation on snapshot changes.

Updating Model Data

1. Pull Remote Sources

$ mix llm_db.pull
Pulling from configured sources...
✓ ModelsDev: 450 models cached at priv/llm_db/cache/models_dev.json

2. Build Snapshot

$ mix llm_db.build
Running ETL pipeline...
✓ Ingested 3 sources
✓ Validated 450 models (3 invalid dropped)
✓ Merged with precedence (last wins)
✓ Filtered to 400 models (50 excluded by filters)
✓ Enriched and indexed
✓ Snapshot written to priv/llm_db/snapshot.json
✓ Generated lib/llm_db/generated/valid_providers.ex

Summary:
  Providers: 12
  Models: 400
  Snapshot size: 1.2 MB

Outputs:

  1. priv/llm_db/snapshot.json - V2 snapshot
  2. lib/llm_db/generated/valid_providers.ex - Pre-existing provider atoms (prevents atom leaks)

3. Commit Changes

$ git add priv/llm_db/snapshot.json lib/llm_db/generated/valid_providers.ex
$ git commit -m "Update model snapshot"

Versioning and Tagging

Date-based versioning: YYYY.M.D

Update Version

$ mix llm_db.version
Updated version to 2025.11.6 in mix.exs

Updates @version in mix.exs.

Generate Changelog and Tag

$ mix git_ops.release
Analyzing commits since last release...
Updating CHANGELOG.md...
Creating git tag v2025.11.6...
  1. Parses conventional commits since last tag
  2. Updates CHANGELOG.md
  3. Creates git tag v2025.11.6

Push Tag

$ git push && git push --tags

CI triggers on tag push: Hex.pm publish, GitHub release, HexDocs publish.

Snapshot Format

V2 Schema

{
  "version": 2,
  "generated_at": "ISO8601 timestamp",
  "providers": {
    "provider_atom_as_string": {
      "id": "provider_atom_as_string",
      "name": "Provider Name",
      "base_url": "https://...",
      "env": {...},
      "doc": "https://...",
      "extra": {...},
      "models": {
        "model_id": {
          "id": "model_id",
          "provider": "provider_atom_as_string",
          "name": "Model Name",
          "capabilities": {...},
          "limits": {...},
          "cost": {...},
          "modalities": {...},
          "deprecated": false,
          "aliases": [...],
          "extra": {...}
        }
      }
    }
  }
}

Runtime-only indexes (rebuilt during LLMDB.load/1):

  • providers_by_id
  • models_by_key
  • models_by_provider
  • aliases_by_key

Release Triggers

  1. Upstream data changes (new models, pricing, capabilities)
  2. Schema changes (new fields, modality types)
  3. Provider additions
  4. Taxonomy updates
  5. Metadata corrections

Release Checklist

Automated Release

$ mix llm_db.pull && \
  mix llm_db.build && \
  mix test && \
  mix llm_db.version && \
  mix git_ops.release && \
  git push && \
  git push --tags

Or via mix alias:

defp aliases do
  [
    release: [
      "llm_db.pull",
      "llm_db.build",
      "test",
      "llm_db.version",
      "git_ops.release"
    ]
  ]
end
$ mix release
$ git push && git push --tags

Snapshot Versions

  • v1: Flat lists (deprecated)
  • v2: Nested providers with models (current)

Snapshot Size

Current: ~1.2 MB (400 models × 12 providers)

Optimization options:

  1. Filter deprecated models at build time
  2. gzip compression (transparent in BEAM)
  3. Split by provider (on-demand load)
  4. Remove :extra fields

Schema Evolution Signals

  • New capability types (e.g., "video generation")
  • New cost models (e.g., "per-character")
  • New modality types (e.g., "3d", "code")
  • Provider-specific features becoming common

Schema changes require updates to:

  1. Source adapters (ModelsDev transform)
  2. Validation schemas (Zoi definitions)
  3. Documentation (capability lists)