SnmpKit.SnmpLib.Cache (snmpkit v0.6.4)
Intelligent caching system for SNMP operations with adaptive strategies.
This module provides sophisticated caching capabilities designed to optimize SNMP polling performance in high-throughput environments. Based on patterns proven in the DDumb project for managing thousands of concurrent device polls.
Features
- Multi-Level Caching: L1 (in-memory), L2 (ETS), L3 (persistent storage)
- Adaptive TTL: Dynamic cache expiration based on data volatility
- Smart Invalidation: Automatic cache invalidation based on data patterns
- Compression: Efficient storage of large SNMP responses
- Hot/Cold Data Management: Automatic promotion of frequently accessed data
- Cache Warming: Proactive loading of expected data
Caching Strategies
Time-Based Caching
Standard TTL-based caching for static or slowly changing data.
Volatility-Based Caching
Dynamic TTL adjustment based on observed change frequency.
Dependency-Based Caching
Cache invalidation based on related data changes.
Predictive Caching
Pre-loading data based on access patterns and time of day.
Performance Benefits
- 50-80% reduction in redundant SNMP queries
- Improved response times for frequently accessed data
- Reduced network load on monitored devices
- Better scalability for large device inventories
Usage Patterns
# Cache SNMP response data
SnmpKit.SnmpLib.Cache.put("device_123:sysDescr", response_data, ttl: 300_000)
# Retrieve cached data
case SnmpKit.SnmpLib.Cache.get("device_123:sysDescr") do
{:ok, data} -> data
:miss -> perform_snmp_query()
end
# Cache with adaptive TTL
SnmpKit.SnmpLib.Cache.put_adaptive("device_123:ifTable", interface_data,
base_ttl: 60_000,
volatility: :medium
)
# Warm cache for predictable access
SnmpKit.SnmpLib.Cache.warm_cache("device_123", [:sysDescr, :sysUpTime, :ifTable])
# Invalidate related caches
SnmpKit.SnmpLib.Cache.invalidate_pattern("device_123:*")
Cache Key Patterns
device_id:oid
- Single OID valuesdevice_id:table:index
- Table row datadevice_id:walk:base_oid
- Walk resultsdevice_id:bulk:oids
- Bulk query resultsglobal:topology
- Cross-device topology data
Summary
Functions
Returns a specification to start this module under a supervisor.
Clears all cached data.
Removes a specific key from the cache.
Retrieves a value from the cache.
Gets comprehensive cache performance statistics.
Invalidates cache entries by tag.
Invalidates multiple cache entries matching a pattern.
Stores a value in the cache with specified options.
Stores a value with adaptive TTL based on observed volatility.
Starts the cache manager with specified configuration.
Pre-loads cache with expected data to improve response times.
Types
@type cache_key() :: binary()
@type cache_opts() :: [ ttl: cache_ttl(), strategy: cache_strategy(), volatility: volatility(), compress: boolean(), dependencies: [cache_key()], tags: [atom()] ]
@type cache_stats() :: %{ total_entries: non_neg_integer(), hit_rate: float(), miss_rate: float(), eviction_count: non_neg_integer(), memory_usage_mb: float(), compression_ratio: float() }
@type cache_strategy() ::
:time_based | :volatility_based | :dependency_based | :predictive
@type cache_ttl() :: pos_integer()
@type cache_value() :: any()
@type volatility() :: :low | :medium | :high | :extreme
Functions
Returns a specification to start this module under a supervisor.
See Supervisor
.
@spec clear() :: :ok
Clears all cached data.
Examples
:ok = SnmpKit.SnmpLib.Cache.clear()
@spec delete(cache_key()) :: :ok
Removes a specific key from the cache.
Examples
:ok = SnmpKit.SnmpLib.Cache.delete("device_1:sysDescr")
@spec get(cache_key()) :: {:ok, cache_value()} | :miss
Retrieves a value from the cache.
Returns
{:ok, value}
: Cache hit with the stored value:miss
: Cache miss, value not found or expired
Examples
case SnmpKit.SnmpLib.Cache.get("device_1:sysDescr") do
{:ok, description} ->
Logger.debug("Cache hit for system description")
description
:miss ->
Logger.debug("Cache miss, performing SNMP query")
perform_snmp_get(device, [1,3,6,1,2,1,1,1,0])
end
@spec get_stats() :: cache_stats()
Gets comprehensive cache performance statistics.
Returns
Statistics including hit rates, memory usage, and performance metrics.
Examples
cache_stats = SnmpKit.SnmpLib.Cache.get_stats()
IO.puts "Cache hit rate: " <> Float.to_string(Float.round(cache_stats.hit_rate * 100, 2)) <> "%"
IO.puts "Memory usage: " <> Float.to_string(Float.round(cache_stats.memory_usage_mb, 2)) <> " MB"
@spec invalidate_by_tag(atom()) :: :ok
Invalidates cache entries by tag.
Examples
SnmpKit.SnmpLib.Cache.invalidate_by_tag(:routing_data)
@spec invalidate_pattern(binary()) :: :ok
Invalidates multiple cache entries matching a pattern.
Supports wildcards (*) for pattern matching.
Examples
# Invalidate all data for a device
SnmpKit.SnmpLib.Cache.invalidate_pattern("device_1:*")
# Invalidate all interface data
SnmpKit.SnmpLib.Cache.invalidate_pattern("*:ifTable")
# Invalidate by tag
SnmpKit.SnmpLib.Cache.invalidate_by_tag(:interface_data)
@spec put(cache_key(), cache_value(), cache_opts()) :: :ok
Stores a value in the cache with specified options.
Parameters
key
: Unique cache keyvalue
: Data to cacheopts
: Caching options
Options
ttl
: Time-to-live in milliseconds (default: 300,000)strategy
: Caching strategy (default: :time_based)volatility
: Data change frequency (default: :medium)compress
: Force compression for this entry (default: auto)dependencies
: Keys that invalidate this entry when changedtags
: Metadata tags for grouping and invalidation
Examples
# Simple time-based caching
:ok = SnmpKit.SnmpLib.Cache.put("device_1:sysDescr", "Cisco Router", ttl: 600_000)
# Adaptive caching based on volatility
:ok = SnmpKit.SnmpLib.Cache.put("device_1:ifTable", interface_data,
strategy: :volatility_based,
volatility: :high,
tags: [:interface_data]
)
# Dependency-based caching
:ok = SnmpKit.SnmpLib.Cache.put("device_1:route_summary", summary_data,
dependencies: ["device_1:routeTable", "device_1:arpTable"]
)
@spec put_adaptive(cache_key(), cache_value(), cache_ttl(), volatility()) :: :ok
Stores a value with adaptive TTL based on observed volatility.
The cache automatically adjusts TTL based on how frequently the data changes.
Parameters
key
: Cache keyvalue
: Data to cachebase_ttl
: Starting TTL valuevolatility
: Expected change frequency
Examples
# Interface counters change frequently
SnmpKit.SnmpLib.Cache.put_adaptive("device_1:ifInOctets", counter_data,
base_ttl: 30_000,
volatility: :high
)
# System description rarely changes
SnmpKit.SnmpLib.Cache.put_adaptive("device_1:sysDescr", description,
base_ttl: 3_600_000,
volatility: :low
)
Starts the cache manager with specified configuration.
Options
max_size
: Maximum number of cache entries (default: 100,000)cleanup_interval
: Cleanup frequency in milliseconds (default: 60,000)compression_enabled
: Enable compression for large values (default: true)adaptive_ttl_enabled
: Enable adaptive TTL based on volatility (default: true)predictive_enabled
: Enable predictive caching (default: false)
Examples
# Start with defaults
{:ok, _pid} = SnmpKit.SnmpLib.Cache.start_link()
# Start with custom configuration
{:ok, _pid} = SnmpKit.SnmpLib.Cache.start_link(
max_size: 50_000,
compression_enabled: true,
predictive_enabled: true
)
Pre-loads cache with expected data to improve response times.
Parameters
device_id
: Target device identifieroids
: List of OIDs to pre-loadstrategy
: Warming strategy (:immediate, :scheduled, :predictive)
Examples
# Immediate cache warming
SnmpKit.SnmpLib.Cache.warm_cache("device_1",
["1.3.6.1.2.1.1.1.0", "1.3.6.1.2.1.1.3.0"],
strategy: :immediate
)
# Predictive warming based on historical access
SnmpKit.SnmpLib.Cache.warm_cache("device_1", :auto,
strategy: :predictive
)