Milvex.Data.FieldData (milvex v0.10.2)

Copy Markdown

Converts between Elixir values and Milvus FieldData protobuf structures.

This module handles the conversion of column data to/from the FieldData proto format used by Milvus for insert operations and result parsing.

Summary

Functions

Builds a ScalarField proto for the given data type and values.

Builds a VectorField proto for the given data type and values.

Extracts values from a ScalarField proto.

Extracts values from a VectorField proto.

Extracts values from a FieldData proto struct.

Converts a column of values to a FieldData proto struct.

Converts dynamic field values to a FieldData proto struct with is_dynamic flag.

Functions

build_scalar_field(data_type, values)

Builds a ScalarField proto for the given data type and values.

build_vector_field(data_type, values, dimension)

Builds a VectorField proto for the given data type and values.

For dense vectors, values should be a list of lists (each inner list is a vector). For sparse vectors, values should be a list of tuple lists: [[{idx, val}, ...], ...].

extract_scalar_values(scalar_field)

@spec extract_scalar_values(Milvex.Milvus.Proto.Schema.ScalarField.t()) :: list()

Extracts values from a ScalarField proto.

extract_vector_values(vector_field)

@spec extract_vector_values(Milvex.Milvus.Proto.Schema.VectorField.t()) :: list()

Extracts values from a VectorField proto.

Returns vectors as lists for float vectors, or as-is for binary types. Sparse vectors are returned as tuple lists.

from_proto(field_data)

@spec from_proto(Milvex.Milvus.Proto.Schema.FieldData.t()) :: {String.t(), list()}

Extracts values from a FieldData proto struct.

Returns a tuple of {field_name, values} where values is a list.

Examples

iex> field_data = %FieldData{field_name: "age", scalars: %ScalarField{data: {:int_data, %IntArray{data: [25, 30]}}}}
iex> FieldData.from_proto(field_data)
{"age", [25, 30]}

to_proto(field_name, values, field_schema)

Converts a column of values to a FieldData proto struct.

Parameters

  • field_name - Name of the field
  • values - List of values for this column
  • field_schema - The Field struct with type information

Examples

iex> field = Field.scalar("age", :int32)
iex> FieldData.to_proto("age", [25, 30, 35], field)
%FieldData{field_name: "age", type: :Int32, scalars: %ScalarField{...}}

to_proto_dynamic(field_name, values)

@spec to_proto_dynamic(String.t(), [map()]) ::
  Milvex.Milvus.Proto.Schema.FieldData.t()

Converts dynamic field values to a FieldData proto struct with is_dynamic flag.

Used for dynamic fields when enable_dynamic_field is true on the schema. Each value should be a map representing the dynamic fields for that row.

Parameters

  • field_name - Name of the dynamic field (typically "$meta")
  • values - List of maps, one per row, containing the dynamic field values