Caches publishing group listing metadata to avoid expensive filesystem scans on every request.
Instead of scanning 50+ files per request, the listing page reads a single
.listing_cache.json file containing all post metadata.
How It Works
- When a post is created/updated/published,
regenerate/1is called - This scans all posts and writes metadata to
.listing_cache.json render_blog_listingreads from cache instead of scanning filesystem- Cache includes: title, slug, date, status, languages, versions (no content)
Cache File Location
priv/publishing/{group-slug}/.listing_cache.json(With legacy fallback to priv/blogging/{group-slug}/.listing_cache.json)
Performance
- Before: ~500ms (50+ file operations)
- After: ~20ms (1 file read + JSON parse)
Cache Invalidation
Cache is regenerated when:
- Post is created
- Post is updated (metadata or content)
- Post status changes (draft/published/archived)
- Translation is added
- Version is created
In-Memory Caching with :persistent_term
For sub-millisecond performance, parsed cache data is stored in :persistent_term.
- First read after restart: loads from file, parses JSON, stores in :persistent_term (~2ms)
- Subsequent reads: direct memory access (~0.1μs, no variance)
- On regenerate: updates both file and :persistent_term
- On invalidate: clears :persistent_term entry
The JSON file provides persistence across restarts. :persistent_term provides zero-copy, sub-microsecond reads during runtime.
Summary
Functions
Returns the cache file path for a publishing group.
Counts posts by version structure status for a group.
Counts posts by primary_language status in a group.
Checks if a cache exists for a blog (in :persistent_term or file).
Returns whether file caching is enabled. Uses cached settings to avoid database queries on every call. Checks new key first, falls back to legacy key.
Returns the :persistent_term key for tracking the file's generated_at when loaded into memory.
Finds a post by a previous URL slug for 301 redirects.
Finds a post by URL slug for a specific language.
Finds a post by slug in the cache.
Finds a post by path pattern in the cache (for timestamp mode).
Invalidates (deletes) the cache for a blog.
Loads the cache from file into :persistent_term without regenerating the file.
Returns the :persistent_term key for tracking when the memory cache was loaded.
Returns whether memory caching (:persistent_term) is enabled. Uses cached settings to avoid database queries on every call. Checks new key first, falls back to legacy key.
Returns the file's generated_at timestamp that was stored when the memory cache was loaded. This tells us what version of the file data is currently in memory.
Returns when the memory cache was loaded (ISO 8601 string), or nil if not loaded.
Returns the :persistent_term key for a publishing group's cache.
Returns a list of posts that need primary_language migration.
Returns list of posts that need version structure migration.
Reads the cached listing for a publishing group.
Regenerates the listing cache for a blog.
Regenerates only the file cache without loading into memory.
Regenerates the cache if no other process is already regenerating it.
Functions
Returns the cache file path for a publishing group.
Counts posts by version structure status for a group.
Returns %{versioned: n, legacy: n} where:
versioned- posts with v1/, v2/, etc. structurelegacy- posts with flat file structure (need migration)
Counts posts by primary_language status in a group.
Returns %{current: n, needs_migration: n, needs_backfill: n} where:
current- posts with primary_language matching global settingneeds_migration- posts with different primary_language (were created under old setting)needs_backfill- posts with no primary_language stored (legacy posts)
Checks if a cache exists for a blog (in :persistent_term or file).
@spec file_cache_enabled?() :: boolean()
Returns whether file caching is enabled. Uses cached settings to avoid database queries on every call. Checks new key first, falls back to legacy key.
Returns the :persistent_term key for tracking the file's generated_at when loaded into memory.
@spec find_by_previous_url_slug(String.t(), String.t(), String.t()) :: {:ok, map()} | {:error, :not_found | :cache_miss}
Finds a post by a previous URL slug for 301 redirects.
When a URL slug changes, the old slug is stored in previous_url_slugs.
This function finds posts that previously used the given URL slug.
Returns
{:ok, cached_post}- Found post that previously used this slug{:error, :not_found}- No post with this previous slug{:error, :cache_miss}- Cache not available
@spec find_by_url_slug(String.t(), String.t(), String.t()) :: {:ok, map()} | {:error, :not_found | :cache_miss}
Finds a post by URL slug for a specific language.
This enables O(1) lookup from URL slug to internal identifier, supporting per-language URL slugs for SEO-friendly localized URLs.
Parameters
group_slug- The publishing grouplanguage- The language code to search inurl_slug- The URL slug to find
Returns
{:ok, cached_post}- Found post (includes internalslugfor file lookup){:error, :not_found}- No post with this URL slug for this language{:error, :cache_miss}- Cache not available
Finds a post by slug in the cache.
This is useful for single post views where we need metadata (language_statuses, version_statuses, allow_version_access) without reading multiple files.
Returns {:ok, cached_post} if found, {:error, :not_found} otherwise.
@spec find_post_by_path(String.t(), String.t(), String.t()) :: {:ok, map()} | {:error, :not_found | :cache_miss}
Finds a post by path pattern in the cache (for timestamp mode).
Matches posts where the path contains the date/time pattern.
Returns {:ok, cached_post} if found, {:error, :not_found} otherwise.
@spec invalidate(String.t()) :: :ok
Invalidates (deletes) the cache for a blog.
Clears both the :persistent_term entry and the JSON file.
The next read will return :cache_miss, triggering a fallback to
the filesystem scan.
Loads the cache from file into :persistent_term without regenerating the file.
Returns :ok if successful, {:error, :no_file} if file doesn't exist,
or {:error, reason} for other failures.
Returns the :persistent_term key for tracking when the memory cache was loaded.
@spec memory_cache_enabled?() :: boolean()
Returns whether memory caching (:persistent_term) is enabled. Uses cached settings to avoid database queries on every call. Checks new key first, falls back to legacy key.
Returns the file's generated_at timestamp that was stored when the memory cache was loaded. This tells us what version of the file data is currently in memory.
Returns when the memory cache was loaded (ISO 8601 string), or nil if not loaded.
Returns the :persistent_term key for a publishing group's cache.
Returns a list of posts that need primary_language migration.
This checks all posts in a group and returns those that either:
- Have no
primary_languagestored (need backfill) - Have
primary_languagedifferent from global setting (need migration decision)
Returns list of posts that need version structure migration.
Reads the cached listing for a publishing group.
Returns {:ok, posts} if cache exists and is valid.
Returns {:error, :cache_miss} if cache doesn't exist, is corrupt, or caching is disabled.
Respects the publishing_file_cache_enabled and publishing_memory_cache_enabled settings
(with fallback to legacy blogging_* keys).
Regenerates the listing cache for a blog.
Scans all posts using the standard list_posts function and writes
the metadata to .listing_cache.json.
This should be called after any post operation that changes the listing:
- create_post
- update_post
- add_language_to_post
- create_new_version
Returns :ok on success or {:error, reason} on failure.
Regenerates only the file cache without loading into memory.
This scans all posts and writes to .listing_cache.json but does not
update :persistent_term. Use load_into_memory/1 separately if needed.
Regenerates the cache if no other process is already regenerating it.
This prevents the "thundering herd" problem where multiple concurrent requests all trigger cache regeneration simultaneously after a server restart.
Uses ETS with insert_new/2 for atomic lock acquisition - only one process
can acquire the lock at a time. The lock includes a timestamp and will be
considered stale after 30000ms to prevent permanent lockout
if a process dies mid-regeneration.
Returns:
:okif regeneration was performed successfully:already_in_progressif another process is currently regenerating{:error, reason}if regeneration failed
Usage
On cache miss in read paths, use this instead of regenerate/1:
case ListingCache.regenerate_if_not_in_progress(blog_slug) do
:ok -> # Cache is ready, read from it
:already_in_progress -> # Fall back to filesystem scan
{:error, _} -> # Fall back to filesystem scan
end