Milvex.Data (milvex v0.10.2)

Copy Markdown

Builder for Milvus insert data.

Converts row-oriented Elixir data to column-oriented FieldData format required by Milvus for insert operations.

Examples

# Create data from row format
schema = Schema.build!(
  name: "movies",
  fields: [
    Field.primary_key("id", :int64),
    Field.varchar("title", 256),
    Field.vector("embedding", 128)
  ]
)

{:ok, data} = Data.from_rows([
  %{id: 1, title: "Movie 1", embedding: [0.1, 0.2, ...]},
  %{id: 2, title: "Movie 2", embedding: [0.3, 0.4, ...]}
], schema)

# Or from column format
{:ok, data} = Data.from_columns(%{
  id: [1, 2],
  title: ["Movie 1", "Movie 2"],
  embedding: [[0.1, 0.2, ...], [0.3, 0.4, ...]]
}, schema)

# Convert to proto for insert
fields_data = Data.to_proto(data)

Summary

Types

t()

Functions

field_names(data)

Returns the field names in the data.

from_columns(columns, schema)

Creates Data from column format.

from_columns!(columns, schema)

Creates Data from column format. Raises on error.

from_rows(rows, schema)

Creates Data from a list of row maps.

from_rows!(rows, schema)

Creates Data from a list of row maps. Raises on error.

get_field(data, name)

Gets the values for a specific field.

num_rows(data)

Returns the number of rows in the data.

to_proto(data)

Converts the Data to a list of FieldData proto structs.

Types

t()

@type t() :: %Milvex.Data{
  fields: %{required(String.t()) => list()},
  num_rows: non_neg_integer(),
  schema: Milvex.Schema.t()
}

Functions

field_names(data)

@spec field_names(t()) :: [String.t()]

Returns the field names in the data.

from_columns(columns, schema)

@spec from_columns(map(), Milvex.Schema.t()) ::
  {:ok, t()} | {:error, Milvex.Error.t()}

Creates Data from column format.

Columns should be a map where keys are field names and values are lists of values for that column. All columns must have the same length.

Parameters

  • columns - Map of field_name => [values]
  • schema - The Schema for type information

Returns

  • {:ok, data} on success
  • {:error, error} on validation failure

from_columns!(columns, schema)

@spec from_columns!(map(), Milvex.Schema.t()) :: t()

Creates Data from column format. Raises on error.

from_rows(rows, schema)

@spec from_rows([map()], Milvex.Schema.t()) :: {:ok, t()} | {:error, Milvex.Error.t()}

Creates Data from a list of row maps.

Each row should be a map with field names as keys. All rows must have consistent field names.

Parameters

  • rows - List of maps representing rows
  • schema - The Schema for type information

Returns

  • {:ok, data} on success
  • {:error, error} on validation failure

from_rows!(rows, schema)

@spec from_rows!([map()], Milvex.Schema.t()) :: t()

Creates Data from a list of row maps. Raises on error.

get_field(data, name)

@spec get_field(t(), String.t() | atom()) :: list() | nil

Gets the values for a specific field.

num_rows(data)

@spec num_rows(t()) :: non_neg_integer()

Returns the number of rows in the data.

to_proto(data)

@spec to_proto(t()) :: [Milvex.Milvus.Proto.Schema.FieldData.t()]

Converts the Data to a list of FieldData proto structs.