Store.Project.ConversationIndex (fnord v0.9.29)

View Source

Manages semantic index data for conversations within a project.

Index data is stored under the project's store path in a parallel directory tree to conversations themselves:

<store_path>/conversations/index/<conversation-id>/
  embeddings.json
  metadata.json

This module is responsible for tracking which conversations are indexed, determining which ones are new or stale, and reading/writing embeddings and associated metadata.

Summary

Types

metadata()
status()

Functions

all_embeddings(project)

Enumerates all indexed conversations, yielding {id, embedding_vector, metadata}.

delete(project, conversation_id)

Deletes the index entry for the given conversation id.

index_status(project)

Returns the status of the conversation index for the given project.

path_for(project, conversation_id)
read_embeddings(project, conversation_id)

Reads embeddings and metadata for a conversation.

read_metadata(project, conversation_id)

Reads only the metadata for a conversation index entry.

root(project)
stale?(project, convo)

Returns true when the given conversation still needs indexing in this project. Stale when any of

write_embeddings(project, conversation_id, embeddings, metadata)

Writes embeddings and metadata for a conversation.

Types

metadata()

@type metadata() :: %{optional(String.t()) => any()}

status()

@type status() :: %{
  new: [Store.Project.Conversation.t()],
  stale: [Store.Project.Conversation.t()],
  deleted: [String.t()]
}

Functions

all_embeddings(project)

@spec all_embeddings(Store.Project.t()) :: Enumerable.t()

Enumerates all indexed conversations, yielding {id, embedding_vector, metadata}.

delete(project, conversation_id)

@spec delete(Store.Project.t(), String.t()) :: :ok

Deletes the index entry for the given conversation id.

index_status(project)

@spec index_status(Store.Project.t()) :: status()

Returns the status of the conversation index for the given project.

It classifies conversations into:

  • :deleted - indexed conversations whose source conversation no longer exists
  • :stale - conversations whose index metadata is stale compared to the
              on-disk conversation timestamp or embedding model
  • :new - conversations that exist but have no index entry

path_for(project, conversation_id)

@spec path_for(Store.Project.t(), String.t()) :: String.t()

read_embeddings(project, conversation_id)

@spec read_embeddings(Store.Project.t(), String.t()) ::
  {:ok, %{embeddings: any(), metadata: metadata()}} | {:error, term()}

Reads embeddings and metadata for a conversation.

Returns {:ok, %{embeddings: embeddings, metadata: metadata}} on success or an error tuple if either file cannot be read/decoded.

read_metadata(project, conversation_id)

@spec read_metadata(Store.Project.t(), String.t()) ::
  {:ok, metadata()} | {:error, term()}

Reads only the metadata for a conversation index entry.

root(project)

@spec root(Store.Project.t()) :: String.t()

stale?(project, convo)

@spec stale?(Store.Project.t(), Store.Project.Conversation.t()) :: boolean()

Returns true when the given conversation still needs indexing in this project. Stale when any of:

  • No index entry on disk.
  • The entry's last_indexed_ts is older than the conversation's timestamp.
  • The stored embedding was produced by a different model (vector dimension doesn't match the current embedder).

Used by foreground indexers to re-check freshness inside a per-conversation lock: if a parallel indexer already refreshed the entry, skip instead of re-running the summarizer + embedder.

write_embeddings(project, conversation_id, embeddings, metadata)

@spec write_embeddings(Store.Project.t(), String.t(), any(), metadata()) ::
  :ok | {:error, term()}

Writes embeddings and metadata for a conversation.

The embeddings are stored in embeddings.json and the metadata in metadata.json under the conversation's index directory.