WeaviateEx.Query.NearMedia (WeaviateEx v0.7.4)

View Source

Media-based vector search for multimodal collections.

Supports audio, video, thermal, depth, and IMU data types for multi2vec-bind and similar multimodal vectorizers.

Supported Media Types

  • :audio - Audio files (wav, mp3, etc.)
  • :video - Video files (mp4, avi, etc.)
  • :thermal - Thermal imaging data
  • :depth - Depth sensor data
  • :imu - Inertial measurement unit data

Examples

# Search by audio
NearMedia.new(:audio, media: base64_audio)
|> Query.execute(client, "MediaCollection")

# Search by video file
NearMedia.new(:video, media_file: "/path/to/video.mp4", certainty: 0.8)

# Using the Query builder
Query.get("MediaCollection")
|> Query.near_media(:audio, media: base64_audio, certainty: 0.7)
|> Query.fields(["name", "description"])
|> Query.execute(client)

Summary

Types

media_type()
t()

Functions

encode_media_file(path)

Encode a media file to base64.

get_encoded_media(near_media)

Get the base64-encoded media data, reading from file if necessary.

media_types()

Returns the list of supported media types.

new(type, opts)

Create a new near_media search configuration.

to_graphql(near_media)

Convert to GraphQL query format.

to_grpc(near_media)

Convert to gRPC NearMediaSearch format.

Types

media_type()

@type media_type() :: :audio | :video | :thermal | :depth | :imu

t()

@type t() :: %WeaviateEx.Query.NearMedia{
  certainty: float() | nil,
  distance: float() | nil,
  media: String.t() | nil,
  media_file: String.t() | nil,
  target_vectors: [String.t()] | nil,
  type: media_type()
}

Functions

encode_media_file(path)

@spec encode_media_file(String.t()) :: String.t()

Encode a media file to base64.

Examples

NearMedia.encode_media_file("/path/to/audio.wav")
# => "UklGRi4A..." (WAV header as base64)

get_encoded_media(near_media)

@spec get_encoded_media(t()) :: String.t()

Get the base64-encoded media data, reading from file if necessary.

If the struct was created with :media, returns it as-is. If the struct was created with :media_file, reads and encodes the file.

Examples

near_media = NearMedia.new(:audio, media: "base64data")
NearMedia.get_encoded_media(near_media)
# => "base64data"

near_media = NearMedia.new(:audio, media_file: "/path/to/audio.wav")
NearMedia.get_encoded_media(near_media)
# => "UklGRi4A..." (file contents as base64)

media_types()

@spec media_types() :: [media_type()]

Returns the list of supported media types.

Examples

NearMedia.media_types()
# => [:audio, :video, :thermal, :depth, :imu]

new(type, opts)

@spec new(
  media_type(),
  keyword()
) :: t()

Create a new near_media search configuration.

Arguments

  • type - Media type: :audio, :video, :thermal, :depth, or :imu
  • opts - Keyword options

Options

  • :media - Base64-encoded media data
  • :media_file - Path to media file (will be read and base64 encoded)
  • :certainty - Minimum certainty threshold (0.0 to 1.0)
  • :distance - Maximum distance threshold
  • :target_vectors - List of named vectors to target

Either :media or :media_file must be provided, but not both.

Examples

NearMedia.new(:audio, media: base64_audio)
NearMedia.new(:video, media_file: "/path/to/video.mp4", certainty: 0.8)
NearMedia.new(:thermal, media: data, target_vectors: ["thermal_vector"])

to_graphql(near_media)

@spec to_graphql(t()) :: map()

Convert to GraphQL query format.

Returns a map with string keys suitable for GraphQL nearMedia queries. The media type is converted to a lowercase string (e.g., "audio"). Nil values are excluded from the result.

Examples

near_media = NearMedia.new(:depth, media: "data", distance: 0.2)
NearMedia.to_graphql(near_media)
# => %{"media" => "data", "type" => "depth", "distance" => 0.2}

to_grpc(near_media)

@spec to_grpc(t()) :: map()

Convert to gRPC NearMediaSearch format.

Returns a map suitable for use with the gRPC search API. The media type is converted to the gRPC enum format (e.g., :MEDIA_TYPE_AUDIO). Nil values are excluded from the result.

Examples

near_media = NearMedia.new(:thermal, media: "data", certainty: 0.8)
NearMedia.to_grpc(near_media)
# => %{media: "data", type: :MEDIA_TYPE_THERMAL, certainty: 0.8}