SnmpKit.SnmpLib.Cache (snmpkit v0.6.3)

Intelligent caching system for SNMP operations with adaptive strategies.

This module provides sophisticated caching capabilities designed to optimize SNMP polling performance in high-throughput environments. Based on patterns proven in the DDumb project for managing thousands of concurrent device polls.

Features

  • Multi-Level Caching: L1 (in-memory), L2 (ETS), L3 (persistent storage)
  • Adaptive TTL: Dynamic cache expiration based on data volatility
  • Smart Invalidation: Automatic cache invalidation based on data patterns
  • Compression: Efficient storage of large SNMP responses
  • Hot/Cold Data Management: Automatic promotion of frequently accessed data
  • Cache Warming: Proactive loading of expected data

Caching Strategies

Time-Based Caching

Standard TTL-based caching for static or slowly changing data.

Volatility-Based Caching

Dynamic TTL adjustment based on observed change frequency.

Dependency-Based Caching

Cache invalidation based on related data changes.

Predictive Caching

Pre-loading data based on access patterns and time of day.

Performance Benefits

  • 50-80% reduction in redundant SNMP queries
  • Improved response times for frequently accessed data
  • Reduced network load on monitored devices
  • Better scalability for large device inventories

Usage Patterns

# Cache SNMP response data
SnmpKit.SnmpLib.Cache.put("device_123:sysDescr", response_data, ttl: 300_000)

# Retrieve cached data
case SnmpKit.SnmpLib.Cache.get("device_123:sysDescr") do
  {:ok, data} -> data
  :miss -> perform_snmp_query()
end

# Cache with adaptive TTL
SnmpKit.SnmpLib.Cache.put_adaptive("device_123:ifTable", interface_data,
  base_ttl: 60_000,
  volatility: :medium
)

# Warm cache for predictable access
SnmpKit.SnmpLib.Cache.warm_cache("device_123", [:sysDescr, :sysUpTime, :ifTable])

# Invalidate related caches
SnmpKit.SnmpLib.Cache.invalidate_pattern("device_123:*")

Cache Key Patterns

  • device_id:oid - Single OID values
  • device_id:table:index - Table row data
  • device_id:walk:base_oid - Walk results
  • device_id:bulk:oids - Bulk query results
  • global:topology - Cross-device topology data

Summary

Functions

Returns a specification to start this module under a supervisor.

Clears all cached data.

Removes a specific key from the cache.

Retrieves a value from the cache.

Gets comprehensive cache performance statistics.

Invalidates cache entries by tag.

Invalidates multiple cache entries matching a pattern.

Stores a value in the cache with specified options.

Stores a value with adaptive TTL based on observed volatility.

Starts the cache manager with specified configuration.

Pre-loads cache with expected data to improve response times.

Types

cache_key()

@type cache_key() :: binary()

cache_opts()

@type cache_opts() :: [
  ttl: cache_ttl(),
  strategy: cache_strategy(),
  volatility: volatility(),
  compress: boolean(),
  dependencies: [cache_key()],
  tags: [atom()]
]

cache_stats()

@type cache_stats() :: %{
  total_entries: non_neg_integer(),
  hit_rate: float(),
  miss_rate: float(),
  eviction_count: non_neg_integer(),
  memory_usage_mb: float(),
  compression_ratio: float()
}

cache_strategy()

@type cache_strategy() ::
  :time_based | :volatility_based | :dependency_based | :predictive

cache_ttl()

@type cache_ttl() :: pos_integer()

cache_value()

@type cache_value() :: any()

volatility()

@type volatility() :: :low | :medium | :high | :extreme

Functions

child_spec(init_arg)

Returns a specification to start this module under a supervisor.

See Supervisor.

clear()

@spec clear() :: :ok

Clears all cached data.

Examples

:ok = SnmpKit.SnmpLib.Cache.clear()

delete(key)

@spec delete(cache_key()) :: :ok

Removes a specific key from the cache.

Examples

:ok = SnmpKit.SnmpLib.Cache.delete("device_1:sysDescr")

get(key)

@spec get(cache_key()) :: {:ok, cache_value()} | :miss

Retrieves a value from the cache.

Returns

  • {:ok, value}: Cache hit with the stored value
  • :miss: Cache miss, value not found or expired

Examples

case SnmpKit.SnmpLib.Cache.get("device_1:sysDescr") do
  {:ok, description} ->
    Logger.debug("Cache hit for system description")
    description
  :miss ->
    Logger.debug("Cache miss, performing SNMP query")
    perform_snmp_get(device, [1,3,6,1,2,1,1,1,0])
end

get_stats()

@spec get_stats() :: cache_stats()

Gets comprehensive cache performance statistics.

Returns

Statistics including hit rates, memory usage, and performance metrics.

Examples

cache_stats = SnmpKit.SnmpLib.Cache.get_stats()
IO.puts "Cache hit rate: " <> Float.to_string(Float.round(cache_stats.hit_rate * 100, 2)) <> "%"
IO.puts "Memory usage: " <> Float.to_string(Float.round(cache_stats.memory_usage_mb, 2)) <> " MB"

invalidate_by_tag(tag)

@spec invalidate_by_tag(atom()) :: :ok

Invalidates cache entries by tag.

Examples

SnmpKit.SnmpLib.Cache.invalidate_by_tag(:routing_data)

invalidate_pattern(pattern)

@spec invalidate_pattern(binary()) :: :ok

Invalidates multiple cache entries matching a pattern.

Supports wildcards (*) for pattern matching.

Examples

# Invalidate all data for a device
SnmpKit.SnmpLib.Cache.invalidate_pattern("device_1:*")

# Invalidate all interface data
SnmpKit.SnmpLib.Cache.invalidate_pattern("*:ifTable")

# Invalidate by tag
SnmpKit.SnmpLib.Cache.invalidate_by_tag(:interface_data)

put(key, value, opts \\ [])

@spec put(cache_key(), cache_value(), cache_opts()) :: :ok

Stores a value in the cache with specified options.

Parameters

  • key: Unique cache key
  • value: Data to cache
  • opts: Caching options

Options

  • ttl: Time-to-live in milliseconds (default: 300,000)
  • strategy: Caching strategy (default: :time_based)
  • volatility: Data change frequency (default: :medium)
  • compress: Force compression for this entry (default: auto)
  • dependencies: Keys that invalidate this entry when changed
  • tags: Metadata tags for grouping and invalidation

Examples

# Simple time-based caching
:ok = SnmpKit.SnmpLib.Cache.put("device_1:sysDescr", "Cisco Router", ttl: 600_000)

# Adaptive caching based on volatility
:ok = SnmpKit.SnmpLib.Cache.put("device_1:ifTable", interface_data,
  strategy: :volatility_based,
  volatility: :high,
  tags: [:interface_data]
)

# Dependency-based caching
:ok = SnmpKit.SnmpLib.Cache.put("device_1:route_summary", summary_data,
  dependencies: ["device_1:routeTable", "device_1:arpTable"]
)

put_adaptive(key, value, base_ttl, volatility)

@spec put_adaptive(cache_key(), cache_value(), cache_ttl(), volatility()) :: :ok

Stores a value with adaptive TTL based on observed volatility.

The cache automatically adjusts TTL based on how frequently the data changes.

Parameters

  • key: Cache key
  • value: Data to cache
  • base_ttl: Starting TTL value
  • volatility: Expected change frequency

Examples

# Interface counters change frequently
SnmpKit.SnmpLib.Cache.put_adaptive("device_1:ifInOctets", counter_data,
  base_ttl: 30_000,
  volatility: :high
)

# System description rarely changes
SnmpKit.SnmpLib.Cache.put_adaptive("device_1:sysDescr", description,
  base_ttl: 3_600_000,
  volatility: :low
)

start_link(opts \\ [])

@spec start_link(keyword()) :: {:ok, pid()} | {:error, any()}

Starts the cache manager with specified configuration.

Options

  • max_size: Maximum number of cache entries (default: 100,000)
  • cleanup_interval: Cleanup frequency in milliseconds (default: 60,000)
  • compression_enabled: Enable compression for large values (default: true)
  • adaptive_ttl_enabled: Enable adaptive TTL based on volatility (default: true)
  • predictive_enabled: Enable predictive caching (default: false)

Examples

# Start with defaults
{:ok, _pid} = SnmpKit.SnmpLib.Cache.start_link()

# Start with custom configuration
{:ok, _pid} = SnmpKit.SnmpLib.Cache.start_link(
  max_size: 50_000,
  compression_enabled: true,
  predictive_enabled: true
)

warm_cache(device_id, oids, opts \\ [])

@spec warm_cache(binary(), [binary()] | :auto, keyword()) :: :ok

Pre-loads cache with expected data to improve response times.

Parameters

  • device_id: Target device identifier
  • oids: List of OIDs to pre-load
  • strategy: Warming strategy (:immediate, :scheduled, :predictive)

Examples

# Immediate cache warming
SnmpKit.SnmpLib.Cache.warm_cache("device_1",
  ["1.3.6.1.2.1.1.1.0", "1.3.6.1.2.1.1.3.0"],
  strategy: :immediate
)

# Predictive warming based on historical access
SnmpKit.SnmpLib.Cache.warm_cache("device_1", :auto,
  strategy: :predictive
)