Image.Video (image v0.66.0)

Copy Markdown View Source

Functions to extract frames from a video file or device as images using Xav, an Elixir wrapper around FFmpeg.

Frames can be extracted by frame number or by millisecond offset with Image.Video.image_from_video/2. Streams of frames can be produced with Image.Video.stream!/2.

A video must first be opened with Image.Video.open/2. The underlying Xav reader is garbage-collected, so explicit close/1 is no longer required, but the function is provided as a no-op for source compatibility.

The pattern can be wrapped by Image.Video.with_video/2 which opens a video, executes a function with the video reference, and (since closing is no longer required) simply discards the reference at the end.

Note

This module is only available if the optional dependency Xav is in your mix.exs. Xav in turn requires FFmpeg ≥ 6.0 to be installed on the system.

Migration from the eVision-backed implementation

Earlier releases of Image used :evision (OpenCV) for video frame extraction. From version 0.66.0 the implementation is FFmpeg-based via :xav. The public function shapes are the same with these intentional differences:

  • The opaque video struct is %Image.Video{} rather than %Evision.VideoCapture{}. Pattern-match on the new struct module if your code does so.

  • Backend selection (the :backend option to open/2) has been removed. FFmpeg is the only backend.

  • Camera input is now opened with a device path string rather than an integer index. :default_camera still works on Linux (resolves to /dev/video0) and on macOS (resolves to AVFoundation device 0). Other camera indices need an explicit device string.

  • Frame-based seeking (seek(video, frame: n) and image_from_video(video, frame: n)) is now implemented as a time-based seek to n / fps followed by zero or more next_frame calls to land on the exact frame. For keyframe-only files this is exact; for inter-frame compressed files (the common case) the behaviour is the same since FFmpeg seeks to the nearest keyframe and decodes forward.

Summary

Files and streams

close(video)

Closes a video.

close!(video)

Closes a video, raising on error.

open(source, options \\ [])

Opens a video for frame extraction.

open!(source)

Opens a video for frame extraction, raising on error.

stream!(video, options \\ [])

Returns a Stream of images from a video.

with_video(source, fun)

Opens a video, calls the given function with the video reference, and discards the reference when the function returns.

Operations

image_from_video(video, options \\ [])

Reads a single frame from a video as an Vix.Vips.Image.t/0.

image_from_video!(video, options \\ [])

Reads a single frame from a video as an Vix.Vips.Image.t/0, raising on error. See image_from_video/2.

scrub(video, frames)

Advances the video head by frames frames without decoding them as images.

seek(video, options)

Seeks the video head to a frame or millisecond offset.

seek!(video, options)

Seeks the video head to a frame or millisecond offset, raising on error. See seek/2.

Guards

is_frame(frame, frame_count)

Guards that a frame offset is valid for a video

is_stream(stream_id)

Guards that a stream identifier is valid for a video device

is_valid_millis(millis, duration_seconds)

Guards that a millisecond count is valid for a video

Types

open_options()

Options for Image.Video.open/2. Currently empty — the :backend option supported by the previous eVision-backed implementation has been removed.

seek_options()

The valid options for Image.Video.seek/2 and Image.Video.image_from_video/2.

source()

A video source. Either a file path / URL accepted by FFmpeg, :default_camera for the system's first webcam, or an explicit device path / integer index.

stream_options()

Options for Image.Video.stream!/2.

t()

The representation of an open video.

Files and streams

close(video)

@spec close(t()) :: {:ok, t()}

Closes a video.

Xav's reader is garbage-collected so explicit close is not required. This function is provided for source compatibility with the previous implementation: it returns {:ok, %Image.Video{reader: nil}} so subsequent operations against the same struct will fail with a clear error.

Arguments

Returns

  • {:ok, video} where video.reader is now nil.

close!(video)

@spec close!(t()) :: t()

Closes a video, raising on error.

See close/1.

open(source, options \\ [])

@spec open(source(), open_options()) :: {:ok, t()} | {:error, Image.error()}

Opens a video for frame extraction.

Arguments

  • source is one of:

    • a file path to a video file;
    • a URL accepted by FFmpeg (http://, https://, rtmp://, rtsp://, …);
    • the atom :default_camera for the system's first webcam. Resolves to /dev/video0 on Linux. On macOS this is passed to FFmpeg's AVFoundation input;
    • a non-negative integer camera index. Resolves to /dev/videoN on Linux. Use a device path string on other platforms;
    • a device path string interpreted by FFmpeg directly.

  • options is a keyword list. Currently no options are defined; the :backend option supported by previous releases has been removed.

Returns

  • {:ok, %Image.Video{}} on success or

  • {:error, %Image.Error{}}.

Example

iex> {:ok, video} = Image.Video.open("./test/support/video/video_sample.mp4")
iex> video.fps
30.0

open!(source)

@spec open!(source()) :: t() | no_return()

Opens a video for frame extraction, raising on error.

See open/2.

stream!(video, options \\ [])

@spec stream!(t(), stream_options()) :: Enumerable.t()

Returns a Stream of images from a video.

Arguments

Options

  • :frame — start frame offset (default 0).

  • :millisecond — start millisecond offset.

  • :start — same as :frame (kept for back-compat).

  • :finish — last frame offset, inclusive. Default -1 (meaning to the end of the video).

  • :step — number of frames to advance between yielded frames. Default 1.

Only one of :frame / :millisecond may be supplied.

Returns

Example

iex> video = Image.Video.open!("./test/support/video/video_sample.mp4")
iex> video |> Image.Video.stream!(start: 0, finish: 2) |> Enum.count()
3

with_video(source, fun)

@spec with_video(source(), (t() -> any())) :: any()

Opens a video, calls the given function with the video reference, and discards the reference when the function returns.

Arguments

  • source is the filename of a video file, a URL accepted by FFmpeg, or a device specifier — see open/2.

  • fun is a 1-arity function called with the open %Image.Video{} struct.

Returns

  • The result of fun.(video) or

  • {:error, reason} if the video could not be opened.

Example

iex> result = Image.Video.with_video("./test/support/video/video_sample.mp4", &Image.Video.image_from_video/1)
iex> match?({:ok, %Vix.Vips.Image{}}, result)
true

Operations

image_from_video(video, options \\ [])

@spec image_from_video(t(), seek_options()) ::
  {:ok, Vix.Vips.Image.t()} | {:error, Image.error()}

Reads a single frame from a video as an Vix.Vips.Image.t/0.

Arguments

Returns

  • {:ok, image} or

  • {:error, %Image.Error{}}.

Example

iex> {:ok, video} = Image.Video.open("./test/support/video/video_sample.mp4")
iex> {:ok, _image} = Image.Video.image_from_video(video)
iex> {:ok, _image} = Image.Video.image_from_video(video, frame: 0)
iex> {:ok, _image} = Image.Video.image_from_video(video, millisecond: 1_000)
iex> {:error, %Image.Error{reason: :negative_offset}} = Image.Video.image_from_video(video, frame: -1)
iex> {:error, %Image.Error{reason: :frame_out_of_range}} = Image.Video.image_from_video(video, frame: 500)
iex> :ok
:ok

image_from_video!(video, options \\ [])

@spec image_from_video!(t(), seek_options()) :: Vix.Vips.Image.t() | no_return()

Reads a single frame from a video as an Vix.Vips.Image.t/0, raising on error. See image_from_video/2.

scrub(video, frames)

@spec scrub(t(), pos_integer()) :: {:ok, t()} | {:error, Image.error()}

Advances the video head by frames frames without decoding them as images.

Arguments

Returns

  • {:ok, video} after advancing or

  • {:error, %Image.Error{}}.

seek(video, options)

@spec seek(t(), seek_options()) :: {:ok, t()} | {:error, Image.error()}

Seeks the video head to a frame or millisecond offset.

Note that seeking is not supported on live video streams such as a webcam.

Arguments

  • video is any Image.Video.t/0 returned from open/2.

  • options is a keyword list with exactly one of:

    • frame: non_neg_integer() — seek to a frame offset.

    • millisecond: non_neg_integer() — seek to a millisecond offset.

Returns

  • {:ok, video} on success or

  • {:error, %Image.Error{}}.

Example

iex> {:ok, video} = Image.Video.open("./test/support/video/video_sample.mp4")
iex> {:ok, _} = Image.Video.seek(video, frame: 0)
iex> {:ok, _} = Image.Video.seek(video, millisecond: 1_000)
iex> {:error, %Image.Error{reason: :negative_offset}} = Image.Video.seek(video, frame: -1)
iex> :ok
:ok

seek!(video, options)

@spec seek!(t(), seek_options()) :: t() | no_return()

Seeks the video head to a frame or millisecond offset, raising on error. See seek/2.

Guards

is_frame(frame, frame_count)

(macro)

Guards that a frame offset is valid for a video

is_stream(stream_id)

(macro)

Guards that a stream identifier is valid for a video device

is_valid_millis(millis, duration_seconds)

(macro)

Guards that a millisecond count is valid for a video

Types

open_options()

@type open_options() :: []

Options for Image.Video.open/2. Currently empty — the :backend option supported by the previous eVision-backed implementation has been removed.

seek_options()

@type seek_options() ::
  [{:frame, non_neg_integer()}] | [{:millisecond, non_neg_integer()}]

The valid options for Image.Video.seek/2 and Image.Video.image_from_video/2.

source()

@type source() :: Path.t() | :default_camera | non_neg_integer() | String.t()

A video source. Either a file path / URL accepted by FFmpeg, :default_camera for the system's first webcam, or an explicit device path / integer index.

stream_options()

@type stream_options() :: [
  start: non_neg_integer(),
  finish: integer(),
  step: pos_integer(),
  frame: non_neg_integer() | nil,
  millisecond: non_neg_integer() | nil
]

Options for Image.Video.stream!/2.

t()

@type t() :: %Image.Video{
  duration_seconds: float(),
  fps: float(),
  frame_count: non_neg_integer(),
  height: pos_integer() | nil,
  reader: Xav.Reader.t() | nil,
  source: source(),
  width: pos_integer() | nil
}

The representation of an open video.

:reader holds the underlying Xav.Reader struct. The derived :fps, :duration_seconds, :frame_count, :width, and :height fields are computed at open time so callers can pattern-match without re-querying FFmpeg.