Store.Project.ConversationIndex (fnord v0.8.83)

View Source

Manages semantic index data for conversations within a project.

Index data is stored under the project's store path in a parallel directory tree to conversations themselves:

<store_path>/conversations/index/<conversation-id>/
  embeddings.json
  metadata.json

This module is responsible for tracking which conversations are indexed, determining which ones are new or stale, and reading/writing embeddings and associated metadata.

Summary

Types

metadata()
status()

Functions

all_embeddings(project)

Enumerates all indexed conversations, yielding {id, embedding_vector, metadata}.

delete(project, conversation_id)

Deletes the index entry for the given conversation id.

index_status(project)

Returns the status of the conversation index for the given project.

path_for(project, conversation_id)
read_embeddings(project, conversation_id)

Reads embeddings and metadata for a conversation.

read_metadata(project, conversation_id)

Reads only the metadata for a conversation index entry.

root(project)
write_embeddings(project, conversation_id, embeddings, metadata)

Writes embeddings and metadata for a conversation.

Types

metadata()

@type metadata() :: %{optional(String.t()) => any()}

status()

@type status() :: %{
  new: [Store.Project.Conversation.t()],
  stale: [Store.Project.Conversation.t()],
  deleted: [String.t()]
}

Functions

all_embeddings(project)

@spec all_embeddings(Store.Project.t()) :: Enumerable.t()

Enumerates all indexed conversations, yielding {id, embedding_vector, metadata}.

delete(project, conversation_id)

@spec delete(Store.Project.t(), String.t()) :: :ok

Deletes the index entry for the given conversation id.

index_status(project)

@spec index_status(Store.Project.t()) :: status()

Returns the status of the conversation index for the given project.

It classifies conversations into:

  • :deleted - indexed conversations whose source conversation no longer exists
  • :stale - conversations whose index metadata is stale compared to the
              on-disk conversation timestamp or embedding model
  • :new - conversations that exist but have no index entry

path_for(project, conversation_id)

@spec path_for(Store.Project.t(), String.t()) :: String.t()

read_embeddings(project, conversation_id)

@spec read_embeddings(Store.Project.t(), String.t()) ::
  {:ok, %{embeddings: any(), metadata: metadata()}} | {:error, term()}

Reads embeddings and metadata for a conversation.

Returns {:ok, %{embeddings: embeddings, metadata: metadata}} on success or an error tuple if either file cannot be read/decoded.

read_metadata(project, conversation_id)

@spec read_metadata(Store.Project.t(), String.t()) ::
  {:ok, metadata()} | {:error, term()}

Reads only the metadata for a conversation index entry.

root(project)

@spec root(Store.Project.t()) :: String.t()

write_embeddings(project, conversation_id, embeddings, metadata)

@spec write_embeddings(Store.Project.t(), String.t(), any(), metadata()) ::
  :ok | {:error, term()}

Writes embeddings and metadata for a conversation.

The embeddings are stored in embeddings.json and the metadata in metadata.json under the conversation's index directory.