Conjure.Storage behaviour (Conjure v0.1.1-alpha)

View Source

Behaviour for session storage backends.

Storage backends manage the working directory and file operations for sessions. All backends must provide a local filesystem path for Docker mounting.

Available Backends

BackendModuleDescription
LocalConjure.Storage.LocalEphemeral temp directories (default)
S3Conjure.Storage.S3AWS S3 with local cache
TigrisConjure.Storage.TigrisFly.io Tigris storage

Usage

# Default local storage
{:ok, session} = Conjure.Session.new_docker(skills)

# S3 storage
{:ok, session} = Conjure.Session.new_docker(skills,
  storage: {Conjure.Storage.S3, bucket: "my-bucket"}
)

# Tigris (Fly.io) - zero config on Fly Machines
{:ok, session} = Conjure.Session.new_docker(skills,
  storage: Conjure.Storage.Tigris
)

Implementing a Custom Storage Backend

Any module can implement this behaviour for custom storage needs:

defmodule MyApp.Storage.Azure do
  @behaviour Conjure.Storage

  @impl true
  def init(session_id, opts) do
    # Initialize Azure Blob storage
    {:ok, state}
  end

  # ... implement all callbacks
end

# Use it:
{:ok, session} = Conjure.Session.new_docker(skills,
  storage: {MyApp.Storage.Azure, container: "sessions"}
)

File Callbacks

Sessions can be configured with callbacks for file events:

{:ok, session} = Conjure.Session.new_docker(skills,
  on_file_created: fn file_ref, session_id ->
    MyApp.Repo.insert!(%SessionFile{
      session_id: session_id,
      path: file_ref.path
    })
  end
)

See Also

Summary

Types

callback_event()
content()
file_callback()
file_ref()
path()
session_id()
state()

Callbacks

cleanup(state)

Cleanup storage for a session.

delete(state, path)

Delete a file from storage.

exists?(state, path)

Check if a file exists in storage.

init(session_id, opts)

Initialize storage for a session.

list(state)

List all files in storage.

local_path(state)

Get the local filesystem path for Docker mounting.

read(state, path)

Read a file from storage.

sync_from_remote(state, remote_files)

Sync files from a remote source.

sync_to_remote(state)

Sync local files to remote storage.

write(state, path, content)

Write a file to storage.

Functions

build_file_ref(path, content, opts \\ [])

Build a file reference from path and content.

compute_checksum(content)

Compute SHA256 checksum of content.

generate_session_id()

Generate a unique session ID.

guess_content_type(path)

Guess MIME content type from file extension.

Types

callback_event()

@type callback_event() :: :created | :modified | :deleted | :synced

content()

@type content() :: binary()

file_callback()

@type file_callback() :: (file_ref(), session_id() -> :ok | {:error, term()})

file_ref()

@type file_ref() :: %{
  path: path(),
  size: non_neg_integer(),
  content_type: String.t() | nil,
  checksum: String.t() | nil,
  storage_url: String.t() | nil,
  created_at: DateTime.t()
}

path()

@type path() :: String.t()

session_id()

@type session_id() :: String.t()

state()

@type state() :: term()

Callbacks

cleanup(state)

@callback cleanup(state()) :: :ok | {:error, term()}

Cleanup storage for a session.

Removes all files and releases resources. For remote storage, this also deletes objects from the remote store.

Parameters

  • state - Storage state from init/2

Returns

  • :ok - Cleanup successful
  • {:error, term()} - Cleanup failed

delete(state, path)

@callback delete(state(), path()) :: :ok | {:error, term()}

Delete a file from storage.

Parameters

  • state - Storage state
  • path - Relative path within the session

Returns

  • :ok - File deleted (or didn't exist)
  • {:error, term()} - Delete failed

exists?(state, path)

@callback exists?(state(), path()) :: boolean()

Check if a file exists in storage.

Parameters

  • state - Storage state
  • path - Relative path within the session

Returns

  • true - File exists
  • false - File does not exist

init(session_id, opts)

@callback init(session_id(), opts :: keyword()) :: {:ok, state()} | {:error, term()}

Initialize storage for a session.

Creates the storage state for the given session ID. For local storage, this creates the working directory. For remote storage, this sets up credentials and local cache.

Parameters

  • session_id - Unique identifier for the session
  • opts - Backend-specific options

Returns

  • {:ok, state} - Successfully initialized storage
  • {:error, term()} - Initialization failed

list(state)

@callback list(state()) :: {:ok, [file_ref()]} | {:error, term()}

List all files in storage.

Parameters

  • state - Storage state

Returns

  • {:ok, [file_ref]} - List of file references
  • {:error, term()} - List failed

local_path(state)

@callback local_path(state()) :: {:ok, path()} | {:error, :not_supported}

Get the local filesystem path for Docker mounting.

Docker containers require a local path to mount as a volume. For local storage, this is the storage directory. For remote storage, this is the local cache directory.

Parameters

  • state - Storage state

Returns

  • {:ok, path} - Local filesystem path
  • {:error, :not_supported} - This storage doesn't support local paths

read(state, path)

@callback read(state(), path()) :: {:ok, content()} | {:error, term()}

Read a file from storage.

Parameters

  • state - Storage state
  • path - Relative path within the session

Returns

  • {:ok, content} - File content
  • {:error, {:file_not_found, path}} - File does not exist
  • {:error, term()} - Read failed

sync_from_remote(state, remote_files)

(optional) 
@callback sync_from_remote(state(), remote_files :: [map()]) ::
  {:ok, [file_ref()]} | {:error, term()}

Sync files from a remote source.

Used by the Anthropic backend to download files created server-side.

Parameters

  • state - Storage state
  • remote_files - List of remote file descriptors

Returns

  • {:ok, [file_ref]} - Synced file references
  • {:error, term()} - Sync failed

sync_to_remote(state)

(optional) 
@callback sync_to_remote(state()) :: {:ok, [file_ref()]} | {:error, term()}

Sync local files to remote storage.

Forces upload of any pending local files to remote storage.

Parameters

  • state - Storage state

Returns

  • {:ok, [file_ref]} - Synced file references
  • {:error, term()} - Sync failed

write(state, path, content)

@callback write(state(), path(), content()) :: {:ok, file_ref()} | {:error, term()}

Write a file to storage.

Parameters

  • state - Storage state
  • path - Relative path within the session
  • content - File content as binary

Returns

  • {:ok, file_ref} - File written successfully
  • {:error, term()} - Write failed

Functions

build_file_ref(path, content, opts \\ [])

@spec build_file_ref(path(), content(), keyword()) :: file_ref()

Build a file reference from path and content.

Parameters

  • path - Relative file path
  • content - File content as binary
  • opts - Optional overrides:
    • :content_type - MIME type (auto-detected if not provided)
    • :checksum - SHA256 checksum (computed if not provided)
    • :storage_url - Remote storage URL

Example

iex> Conjure.Storage.build_file_ref("output.xlsx", content)
%{
  path: "output.xlsx",
  size: 1234,
  content_type: "application/vnd.openxmlformats-officedocument.spreadsheetml.sheet",
  checksum: "abc123...",
  storage_url: nil,
  created_at: ~U[2024-01-15 10:30:00Z]
}

compute_checksum(content)

@spec compute_checksum(content()) :: String.t()

Compute SHA256 checksum of content.

Example

iex> Conjure.Storage.compute_checksum("hello")
"2cf24dba5fb0a30e26e83b2ac5b9e29e1b161e5c1fa7425e73043362938b9824"

generate_session_id()

@spec generate_session_id() :: session_id()

Generate a unique session ID.

Uses cryptographically secure random bytes encoded as URL-safe base64.

Example

iex> session_id = Conjure.Storage.generate_session_id()
"Xt7kB9mP2Qw5nL3j"

guess_content_type(path)

@spec guess_content_type(path()) :: String.t()

Guess MIME content type from file extension.

Example

iex> Conjure.Storage.guess_content_type("report.pdf")
"application/pdf"