PhoenixKit.Modules.Storage (phoenix_kit v1.7.71)

Copy Markdown View Source

Storage context for managing files, buckets, and dimensions.

Provides a distributed file storage system with support for multiple storage providers (local filesystem, AWS S3, Backblaze B2, Cloudflare R2) with automatic redundancy and failover capabilities.

Features

  • Multi-location storage with configurable redundancy (1-5 copies)
  • Support for local, S3, B2, and R2 storage providers
  • Automatic variant generation for images and videos
  • Priority-based storage selection
  • Built-in usage tracking and statistics
  • PostgreSQL-backed file registry

Module Status

This module is always enabled and cannot be disabled. It provides core functionality for file management across PhoenixKit.

Summary

Functions

Calculates free space for a bucket.

Calculates storage usage for a bucket in MB.

Calculates user-specific file checksum (salted with user_uuid).

Returns an %Ecto.Changeset{} for tracking bucket changes.

Returns an %Ecto.Changeset{} for tracking dimension changes.

Returns an %Ecto.Changeset{} for tracking file changes.

Returns an %Ecto.Changeset{} for tracking file instance changes.

Returns the count of orphaned files.

Creates a new bucket.

Creates a new dimension.

Creates a directory if it doesn't exist.

Creates a new file record.

Creates a new file instance.

Creates file locations for a file instance across specified buckets.

Deletes a bucket.

Deletes a dimension.

Deletes a file.

Deletes a file completely - physical data from all storage buckets and database record.

Deletes file data from all storage buckets for all variants.

Deletes a file instance.

Ensures at least one default bucket exists.

Checks if a file exists in storage.

Returns true if the given file UUID is not referenced by any known entity.

Returns a list of orphaned files (files not referenced by any known entity).

Gets the absolute path for local storage.

Gets a single bucket by ID.

Gets a bucket by name.

Gets the current storage configuration.

Gets the default storage path for local file uploads (relative path).

Gets a single dimension by ID.

Gets a dimension by name.

Gets a single file by ID.

Gets a file by its original content checksum (file_checksum).

Gets a file by its user-specific checksum.

Gets a single file instance by ID.

Gets the bucket UUIDs where a file instance is stored.

Gets a file instance by file UUID and variant name.

Gets a public URL for a file.

Gets a public URL for a file by file ID.

Gets a public URL for a specific file variant by file ID.

Gets a public URL for a specific file variant.

Returns a list of all storage buckets, ordered by priority.

Returns a list of all dimensions, ordered by size (width x height).

Returns enabled dimensions for a specific file type.

Gets enabled buckets, ordered by priority.

Returns a list of file instances for a given file.

Returns a list of files, optionally filtered by bucket.

Checks if the Storage module is enabled.

Queues a list of file UUIDs for orphan cleanup via Oban.

Repairs the storage module by resetting configuration to defaults.

Resets all dimensions to default seeded values. Deletes all current dimensions and recreates the 8 default ones.

Resets storage settings to their default values.

Retrieves a file from storage by file UUID.

Retrieves a file by its hash.

Stores a file in the storage system.

Updates a bucket.

Updates the default storage path.

Updates a dimension.

Updates a file.

Updates a file instance.

Updates a file instance's processing status.

Updates a file instance with file information after processing.

Validates and normalizes a storage path.

Functions

calculate_bucket_free_space(bucket)

Calculates free space for a bucket.

For local storage, checks actual disk space. For cloud storage, returns the configured max_size_mb minus usage.

calculate_bucket_usage(bucket_uuid)

Calculates storage usage for a bucket in MB.

Returns total size of all files stored in this bucket by summing up all file instances that have locations in this bucket.

calculate_user_file_checksum(user_uuid, file_checksum)

Calculates user-specific file checksum (salted with user_uuid).

This creates a unique checksum per user+file combination for duplicate detection, while preserving the original file checksum for popularity queries.

Parameters

  • user_uuid: The user UUID
  • file_checksum: The SHA256 checksum of the file content

Returns

String representing the SHA256 checksum of "user_uuid + file_checksum"

change_bucket(bucket, attrs \\ %{})

Returns an %Ecto.Changeset{} for tracking bucket changes.

change_dimension(dimension, attrs \\ %{})

Returns an %Ecto.Changeset{} for tracking dimension changes.

change_file(file, attrs \\ %{})

Returns an %Ecto.Changeset{} for tracking file changes.

change_file_instance(instance, attrs \\ %{})

Returns an %Ecto.Changeset{} for tracking file instance changes.

count_orphaned_files()

Returns the count of orphaned files.

create_bucket(attrs \\ %{})

Creates a new bucket.

Examples

iex> create_bucket(%{name: "Local Storage", provider: "local"})
{:ok, %Bucket{}}

iex> create_bucket(%{name: nil})
{:error, %Ecto.Changeset{}}

create_dimension(attrs \\ %{})

Creates a new dimension.

Examples

iex> create_dimension(%{name: "thumbnail", width: 150, height: 150})
{:ok, %Dimension{}}

iex> create_dimension(%{name: nil})
{:error, %Ecto.Changeset{}}

create_directory(path)

Creates a directory if it doesn't exist.

create_file(attrs \\ %{})

Creates a new file record.

This only creates the database record. Use store_file/4 to actually store the file data in storage buckets.

create_file_instance(attrs \\ %{})

Creates a new file instance.

create_file_locations_for_instance(file_instance_uuid, bucket_uuids, file_path)

Creates file locations for a file instance across specified buckets.

Returns {:ok, locations} on success or {:error, :file_locations_failed, errors} if any insertions fail.

Parameters

  • file_instance_uuid - The UUID of the file instance
  • bucket_uuids - List of bucket UUIDs to create locations for
  • file_path - The storage path for the file

Examples

iex> create_file_locations_for_instance(instance_uuid, [bucket_uuid], "path/to/file")
{:ok, [%FileLocation{}]}

iex> create_file_locations_for_instance(instance_uuid, [invalid_bucket], "path")
{:error, :file_locations_failed, [{bucket_uuid, changeset}]}

delete_bucket(bucket)

Deletes a bucket.

Examples

iex> delete_bucket(bucket)
{:ok, %Bucket{}}

iex> delete_bucket(bucket)
{:error, %Ecto.Changeset{}}

delete_dimension(dimension)

Deletes a dimension.

Examples

iex> delete_dimension(dimension)
{:ok, %Dimension{}}

iex> delete_dimension(dimension)
{:error, %Ecto.Changeset{}}

delete_file(file)

Deletes a file.

This only removes the database record. Use delete_file_data/1 to remove the actual file data from storage buckets.

delete_file_completely(file)

Deletes a file completely - physical data from all storage buckets and database record.

Examples

iex> delete_file_completely(file)
{:ok, %File{}}

delete_file_data(file)

Deletes file data from all storage buckets for all variants.

delete_file_instance(instance)

Deletes a file instance.

ensure_default_bucket_exists()

Ensures at least one default bucket exists.

If no buckets exist, creates a default local storage bucket.

Returns

  • {:created, bucket} - If a new bucket was created
  • :exists - If buckets already exist

Examples

iex> ensure_default_bucket_exists()
{:created, %Bucket{name: "Local Storage"}}

iex> ensure_default_bucket_exists()
:exists

file_exists?(file)

Checks if a file exists in storage.

file_orphaned?(file_uuid)

Returns true if the given file UUID is not referenced by any known entity.

find_orphaned_files(opts \\ [])

Returns a list of orphaned files (files not referenced by any known entity).

Options

  • :limit - Maximum number of results
  • :offset - Number of results to skip

get_absolute_path()

Gets the absolute path for local storage.

get_auto_generate_variants()

get_bucket(id)

Gets a single bucket by ID.

Returns nil if bucket does not exist.

get_bucket_by_name(name)

Gets a bucket by name.

get_config()

Gets the current storage configuration.

get_default_path()

Gets the default storage path for local file uploads (relative path).

Returns the configured relative path or the default "priv/uploads" if not set.

get_dimension(id)

Gets a single dimension by ID.

get_dimension_by_name(name)

Gets a dimension by name.

get_file(id)

Gets a single file by ID.

get_file_by_checksum(file_checksum)

Gets a file by its original content checksum (file_checksum).

This can find files uploaded by any user with the same content. Useful for popularity queries.

get_file_by_user_checksum(user_file_checksum)

Gets a file by its user-specific checksum.

This checks for duplicates for a specific user.

get_file_instance(id)

Gets a single file instance by ID.

get_file_instance_bucket_uuids(file_instance_uuid)

Gets the bucket UUIDs where a file instance is stored.

Returns a list of bucket UUIDs from the file_locations for the given file instance.

get_file_instance_by_name(file_uuid, variant_name)

Gets a file instance by file UUID and variant name.

get_public_url(file)

Gets a public URL for a file.

get_public_url_by_uuid(file_uuid)

Gets a public URL for a file by file ID.

Convenience function that fetches the file and returns its URL.

Examples

iex> get_public_url_by_uuid("018e3c4a-9f6b-7890-abcd-ef1234567890")
"https://cdn.example.com/12/a1/a1b2c3d4e5f6/a1b2c3d4e5f6_original.jpg"

iex> get_public_url_by_uuid("invalid-uuid")
nil

get_public_url_by_uuid(file_uuid, variant_name)

Gets a public URL for a specific file variant by file ID.

Examples

iex> get_public_url_by_uuid("018e3c4a-9f6b-7890-abcd-ef1234567890", "thumbnail")
"https://cdn.example.com/12/a1/a1b2c3d4e5f6/a1b2c3d4e5f6_thumbnail.jpg"

get_public_url_by_variant(file, variant_name)

Gets a public URL for a specific file variant.

Variants

For images: "original", "thumbnail", "small", "medium", "large" For videos: "original", "360p", "720p", "1080p", "video_thumbnail"

Examples

iex> get_public_url_by_variant(file, "thumbnail")
"https://cdn.example.com/12/a1/a1b2c3d4e5f6/a1b2c3d4e5f6_thumbnail.jpg"

iex> get_public_url_by_variant(file, "medium")
"https://cdn.example.com/12/a1/a1b2c3d4e5f6/a1b2c3d4e5f6_medium.jpg"

list_buckets()

Returns a list of all storage buckets, ordered by priority.

list_dimensions()

Returns a list of all dimensions, ordered by size (width x height).

list_dimensions_for_type(file_type)

Returns enabled dimensions for a specific file type.

list_enabled_buckets()

Gets enabled buckets, ordered by priority.

list_file_instances(file_uuid)

Returns a list of file instances for a given file.

list_files(opts \\ [])

Returns a list of files, optionally filtered by bucket.

Options

  • :bucket_uuid - Filter by bucket UUID
  • :limit - Maximum number of results
  • :offset - Number of results to skip
  • :order_by - Ordering (default: [desc: :inserted_at])

module_enabled?()

Checks if the Storage module is enabled.

This module is always enabled and cannot be disabled.

Examples

iex> PhoenixKit.Modules.Storage.module_enabled?()
true

queue_file_cleanup(file_uuids)

Queues a list of file UUIDs for orphan cleanup via Oban.

Each file is scheduled for deletion after a 60-second delay to protect against race conditions (another entity may reference the file). Only files that are still orphaned at job execution time will be deleted.

repair_storage_module()

Repairs the storage module by resetting configuration to defaults.

This is a safe, non-destructive operation that:

  1. Creates a default local bucket if no buckets exist
  2. Resets dimensions to 8 defaults (4 image + 4 video)
  3. Resets storage settings to recommended defaults

All existing files are preserved.

Returns

  • {:ok, repairs} - List of repairs performed
  • {:error, reason} - If repair failed

Examples

iex> repair_storage_module()
{:ok, [{:bucket_created, "Local Storage"}, {:dimensions_reset, 8}, {:settings_reset, 3}]}

reset_dimensions_to_defaults()

Resets all dimensions to default seeded values. Deletes all current dimensions and recreates the 8 default ones.

reset_settings_to_defaults()

Resets storage settings to their default values.

Resets:

  • storage_redundancy_copies to "1"
  • storage_auto_generate_variants to "true"
  • storage_default_bucket_uuid to nil

Returns

  • :ok

retrieve_file(file_uuid)

Retrieves a file from storage by file UUID.

Will try buckets in priority order until the file is found.

retrieve_file_by_hash(hash)

Retrieves a file by its hash.

store_file(source_path, opts \\ [])

Stores a file in the storage system.

This will:

  1. Store the file in multiple buckets based on redundancy settings
  2. Generate variants if enabled
  3. Create database records for the file and its variants

Options

  • :filename - Original filename (required)
  • :content_type - MIME type (required)
  • :size_bytes - File size in bytes (required)
  • :user_uuid - User UUID who owns the file
  • :metadata - Additional metadata map

store_file_in_buckets(source_path, file_type, user_uuid, file_checksum, ext, original_filename \\ nil)

Stores a file in buckets with hierarchical path structure.

Path Structure

Files are stored using the pattern: {user_uuid[0..1]}/{hash[0..1]}/{full_hash}/{full_hash}_{variant}.{format}

Examples

User ID: "12345678" File hash: "a1b2c3d4e5f6..." Original: "12/a1/a1b2c3d4e5f6/a1b2c3d4e5f6_original.jpg" Thumbnail: "12/a1/a1b2c3d4e5f6/a1b2c3d4e5f6_thumbnail.jpg"

update_bucket(bucket, attrs)

Updates a bucket.

Examples

iex> update_bucket(bucket, %{name: "New Name"})
{:ok, %Bucket{}}

iex> update_bucket(bucket, %{name: nil})
{:error, %Ecto.Changeset{}}

update_default_path(relative_path)

Updates the default storage path.

update_dimension(dimension, attrs)

Updates a dimension.

Examples

iex> update_dimension(dimension, %{name: "New Name"})
{:ok, %Dimension{}}

iex> update_dimension(dimension, %{name: nil})
{:error, %Ecto.Changeset{}}

update_file(file, attrs)

Updates a file.

update_file_instance(instance, attrs)

Updates a file instance.

update_instance_status(instance, status)

Updates a file instance's processing status.

update_instance_with_file_info(instance, file_path, dimensions \\ nil)

Updates a file instance with file information after processing.

validate_and_normalize_path(path)

Validates and normalizes a storage path.

Returns {:ok, relative_path} if valid, or error tuple if invalid.