View Source Vix Introduction

Run in Livebook

Overview

Welcome to an in-depth exploration of Vix, a powerful Elixir library for image processing. This guide will walk you through using Vix to handle common and advanced image processing tasks efficiently. Vix is particularly valuable for applications requiring high-performance image manipulation, such as web applications, scientific image analysis, digital art creation, and batch processing of large image datasets.

Why Vix?

Vix provides Elixir bindings to libvips, a robust and mature image processing library. Here's why Vix stands out:

  • Speed: Processes images up to 10x faster than alternatives by using sophisticated algorithms and parallel processing
  • Memory efficiency: Rather than loading entire images into memory, Vix streams data in small chunks, making it ideal for processing large images
  • Rich feature set: Access to hundreds of image operations from basic transformations to complex filters and analysis tools
  • Elixir integration: Seamlessly integrates with Elixir's functional programming paradigm, offering immutable operations and pipeline-friendly APIs

Setup

First, let's set up our development environment with the necessary dependencies:

Mix.install([
  {:vix, "~> 0.5"},
  {:kino, "~> 0.7"},  # For interactive examples
  {:req, "~> 0.4"}    # For fetching sample images
])

# Let's check our Vix version. Vix comes with pre-built libvips binaries
IO.puts("Using libvips version: " <> Vix.Vips.version())

Core Concepts

The Image Struct

In Vix, images are represented by %Vix.Vips.Image{} structs. Following functional programming principles, these structs are immutable - every operation creates a new image rather than modifying the existing one. This approach ensures thread safety and makes it easier to reason about image transformations.

Let's explore the various ways to create and work with images:

alias Vix.Vips.Image
alias Vix.Vips.Operation

# 1. Loading from file. Change the image path
# {:ok, from_file} = Image.new_from_file("input.jpg")

# 2. Loading from memory - useful for handling uploaded files or HTTP responses
# image_binary = File.read!("input.jpg")
# {:ok, from_binary} = Image.new_from_buffer(image_binary)

# 3. Creating a solid color image - useful for backgrounds or overlays
red = Image.build_image!(100, 100, [255, 0, 0])

# Vix implements `Kino.Render` protocol, so you can see the image by just
# returning it for the Livebook cell.

To make our examples more practical, let's create a helper module to fetch sample images and display them:

defmodule ImageHelper do
  def get_sample_image(width, height) do
    response = Req.get!("https://picsum.photos/#{width}/#{height}", decode_body: false)
    {:ok, image} = Image.new_from_buffer(response.body)
    image
  end

  def show(images, columns \\ nil)
  def show(%Image{} = img, _), do: show([img], 1)

  def show(images, columns) when is_list(images) do
    columns = columns || length(images)

    images
    |> Kino.Layout.grid(boxed: true, columns: columns)
    |> Kino.render()

    :ok
  end
end

Let's fetch a sample image to use throughout our examples:

import ImageHelper

# For our examples, let's use a sample image.
image = get_sample_image(800, 600)

Image Properties and Metadata

Understanding image properties is crucial for processing. Vix provides comprehensive access to image metadata, which can be essential for making processing decisions or maintaining image information:

# Get all available metadata fields - useful for debugging and understanding image characteristics
{:ok, fields} = Image.header_field_names(image)
IO.puts("Available fields: #{Enum.join(fields, ", ")}")

# Get specific field value - useful for understanding image format and processing history
{:ok, value} = Image.header_value(image, "vips-loader")
IO.puts("Loader used: #{value}")

# Get image dimensions and number of color channels - essential for proper image manipulation
{width, height, bands} = Image.shape(image)

Basic Operations

Resizing and Scaling

Image resizing is one of the most common operations in image processing. Each method has its specific use case and trade-offs between speed and quality:

# Fast thumbnail generation - optimized for speed, perfect for preview generation
thumbnail = Operation.thumbnail_image!(image, 400)
# For even better performance use `Operation.thumbnail!` and pass path directly
# thumbnail = Operation.thumbnail!("input.jpg", 300)

# High-quality resize with Lanczos3 kernel - best for preserving image quality
# Lanczos3 provides excellent results for both upscaling and downscaling
resized = Operation.resize!(image, 0.5, kernel: :VIPS_KERNEL_LANCZOS3)

# Scale to specific dimensions while maintaining proper aspect ratio
# Useful for fitting images into specific containers while preventing distortion
scaled = Operation.resize!(image, 400 / Image.width(image), vscale: 300 / Image.height(image))

# Smart cropping - uses edge detection and entropy analysis to keep important parts
# Perfect for automated content-aware thumbnail generation
{smart_crop, _} = Operation.smartcrop!(image, 300, 200)

show([thumbnail, resized, scaled, smart_crop], 2)

Color Operations

Color manipulation is essential for image enhancement, artistic effects, and preparing images for specific use cases:

# Convert to grayscale - useful for reducing complexity or preparing for analysis
# Also great for artistic effects or preparing images for machine learning
grayscale = Operation.colourspace!(image, :VIPS_INTERPRETATION_B_W)

# Adjust RGB channels individually for color balance
# Values > 1 increase channel intensity, < 1 decrease it
# This can correct color casts or create artistic effects
adjusted = Operation.linear!(image,
  [1.2, 1.0, 0.8],  # RGB multipliers: boost red, normal green, reduce blue
  [0, 0, 0]         # RGB offsets: no additional adjustment
)

# Extract alpha channel (transparency) if present
# Useful for masking operations or analyzing image transparency
{:ok, alpha} = Operation.extract_band(image, bands - 1)

# Add an alpha channel - useful for creating partially transparent images
# Essential for overlays and composition effects
with_alpha = Operation.bandjoin!([image, Image.build_image!(width, height, [125])])

show([grayscale, adjusted, with_alpha])

Operators

Vix provides convenient operator overloading for common image manipulations. These operators make the code more readable and intuitive:

use Vix.Operator, only: [+: 2, -: 2, *: 2]  # Import specific operators

# Adjust brightness using operators - multiplication scales pixel values
brighter = image * 1.2  # Increase brightness by 20%
darker = image * 0.8    # Decrease brightness by 20%

# Combining operators for complex effects
# This creates an enhanced image with adjusted brightness, contrast, and blur
enhanced = (image * 1.2) - 10 + Operation.gaussblur!(image, 2)

show([brighter, darker, enhanced])

The Access Protocol

Vix implements Elixir's Access protocol, providing a powerful way to work with image channels and regions. This makes it easy to extract and manipulate specific parts of an image:

# Get the red channel from an RGB image
# Useful for channel-specific analysis or effects
red = image[0]

# Get a 200x100 pixel square from the top-left corner
# Perfect for creating image tiles or focusing on specific regions
top_left = image[[0..199, 0..99]]

# Get the bottom-right 200x100 pixel square
# Negative indices count from the end, just like in Elixir lists
bottom_right = image[[-200..-1, -100..-1]]

# Alternatively you can use keyword list for more readable code
# Get a 200x100 pixel square from the top-left corner, and only red-green channels
red_top_left = image[[width: 0..199, height: 0..99, band: 0..1]]

show([top_left, bottom_right, red_top_left])

# Rearrange color channels - useful for color space manipulation
# RGB -> GBR: Swapping channels can create interesting color effects
remaped = Operation.bandjoin!([image[1], image[2], image[0]])

show([red, remaped])

Advanced Techniques

Creating Processing Pipelines

One of Vix's strengths is its ability to chain operations efficiently using Elixir's pipe operator. This allows you to create complex image processing pipelines that are both readable and performant. Each operation in the pipeline creates a new immutable image, ensuring thread safety and making it easy to debug:

defmodule ImagePipeline do
  def process(image) do
    image
    |> Operation.resize!(0.8)
    |> Operation.sharpen!()
    |> Operation.linear!([1.1], [-0.1])
  end
end

ImagePipeline.process(image)

Image Composition

Combining images with text or other elements is essential for creating watermarks, annotations, or complex composite images. This example demonstrates professional text overlay with proper DPI handling and alpha channel management:

# Create high-quality text overlay with anti-aliasing
{text, _} =
  Operation.text!(
    "Hello from Vix!",
    width: 400,
    dpi: 300,         # High DPI ensures sharp text at any size
    font: "sans-serif bold",
    rgba: true        # Alpha channel enables smooth blending
  )

# Position text precisely on the image
# The embed operation handles proper padding and positioning
positioned_text =
  Operation.embed!(
    text,
    50,              # X offset from left
    50,              # Y offset from top
    Image.width(image),
    Image.height(image)
  )

# Blend using alpha composition for professional results
composite = Operation.composite2!(image, positioned_text, :VIPS_BLEND_MODE_OVER)

show([text, composite])

Filters and Effects

Vix provides a comprehensive set of filters and effects suitable for both practical image enhancement and creative artistic expression. Understanding these operations allows you to create sophisticated image processing applications:

# Gaussian blur - essential for noise reduction and creating depth-of-field effects
# Higher sigma values create stronger blur effects
blurred = Operation.gaussblur!(image, 3.0)

# Edge detection using the Canny algorithm
# Perfect for image analysis and artistic effects
# Multiply by 64 to make edges more visible in the output
edges = Operation.canny!(image, sigma: 1.4) * 64

# Advanced sharpening with fine-grained control
# sigma: controls the radius of the effect
# x1: adjusts the threshold between flat and jagged areas
# m2: determines overall sharpening strength
sharpened =
  Operation.sharpen!(image,
    sigma: 1.0,
    x1: 2.0,
    m2: 20
  )

# Emboss effect using convolution matrix
# The matrix defines the relationship between each pixel and its neighbors
# This creates a 3D-like effect by emphasizing directional changes
{:ok, conv_matrix} =
  Image.new_from_list([
    [-2, -1, 0],   # Top row emphasizes vertical edges
    [-1, 1, 1],    # Middle row provides center weighting
    [0, 1, 2]      # Bottom row creates directional lighting effect
  ])

embossed = Operation.conv!(image, conv_matrix)

show([blurred, edges, sharpened, embossed], 2)

Image Analysis

Vix includes powerful tools for analyzing image characteristics, making it suitable for both artistic applications and scientific image analysis. The histogram functionality is particularly useful for understanding and adjusting image exposure and color distribution:

# Create histogram from grayscale image
# Converting to grayscale first simplifies the analysis
histogram =
  image
  |> Operation.colourspace!(:VIPS_INTERPRETATION_B_W)
  |> Operation.hist_find!()

# Convert histogram to List for programmatic analysis
# Useful for automated image adjustment algorithms
Image.to_list!(histogram)
|> List.flatten()
|> IO.inspect(limit: 15, label: :histogram)

# Create visual representation of the grayscale histogram
# Normalizing ensures consistent visualization regardless of image size
bw_plot =
  histogram
  |> Operation.hist_norm!()  # Scale values to 0..255 range
  |> Operation.hist_plot!()  # Create visual representation

# Generate and plot full color histogram
# Useful for analyzing color balance and distribution
color_plot =
  image
  |> Operation.hist_find!()    # Calculate histogram for each channel
  |> Operation.hist_norm!()    # Normalize values
  |> Operation.hist_plot!()    # Create visual representation

show([bw_plot, color_plot])

Error Handling

Vix offers two complementary approaches to error handling, allowing you to choose the most appropriate strategy for your application:

# Pattern 1: Explicit error handling with with
# Useful for complex workflows where you need fine-grained error control
result =
  with {:ok, resized} <- Operation.resize(image, 0.5),
       {:ok, processed} <- Operation.sharpen(resized, sigma: 1.0) do
    {:ok, processed}
  else
    {:error, reason} ->
      IO.puts("Failed: #{reason}")
      {:error, reason}
  end

# Pattern 2: Bang (!) variants for simplified error propagation
# Ideal for scripts or when you want errors to halt execution
image
|> Operation.resize!(0.5)
|> Operation.sharpen!(sigma: 1.0)

Creating a Color Gradient

This example demonstrates how to programmatically generate color gradients using Vix's color space manipulation capabilities:

defmodule ColorGradient do
  def create(width, height) do
    use Vix.Operator, only: [*: 2, +: 2, /: 2]

    # Generate linear hue gradient
    # identity! creates a gradient from 0 to width
    # Multiply by 255/width normalizes values to 0..255 range
    hue = Operation.identity!(size: width, ushort: true) * 255 / width

    # Convert to HSV color space for vibrant colors
    # Add full saturation and value channels
    hsv =
      hue
      |> Operation.bandjoin_const!([255, 255])  # Add S and V channels
      |> Operation.copy!(interpretation: :VIPS_INTERPRETATION_HSV)

    # Create final gradient by repeating horizontally
    rainbow = Operation.embed!(hsv, 0, 0, width, height, extend: :VIPS_EXTEND_REPEAT)

    show([hue, hsv, rainbow], 1)
  end
end

ColorGradient.create(600, 100)

Creating a Photo Collage

Create professional-looking photo collages with automatic layout, borders, and titles. This example demonstrates combining multiple images into a single composition:

defmodule Collage do
  def create(images, across, gap) do
    # create collage by arranging images in a grid pattern
    # across parameter determines number of images per row
    # gap specifies spacing between images
    collage = Operation.arrayjoin!(images, across: across, shim: gap)

    # Add a consistent border around the entire collage
    # This creates a frame effect and ensures clean edges
    {width, height, _} = Image.shape(collage)
    collage = Operation.embed!(collage, gap, gap, width + gap * 2, height + gap * 2)

    # Add a title overlay with high DPI for crisp text
    {title, _} = Operation.text!("My Photo Collage", dpi: 300)

    # Composite the title onto the collage with proper positioning
    Operation.composite2!(collage, title, :VIPS_BLEND_MODE_OVER, x: 20, y: 20)
  end
end

# Create sample collage using multiple images
# Download sample images for demonstration
images = for _ <- 1..4, do: get_sample_image(600, 400)

Collage.create(images, 2, 10)

Creating Instagram-style Filters

Modern photo apps often use preset filters to enhance images. Here's how to create custom filters using Vix's operations:

defmodule PhotoFilters do
  def vintage(image) do
    image
    # Adjust color balance for warm, aged look
    |> Operation.linear!([0.9, 0.7, 0.6], [-0.1, 0.1, 0.2])
    # Add subtle blur to soften details
    |> Operation.gaussblur!(0.5)
    # Enhance contrast for dramatic effect
    |> Operation.linear!([1.2], [-0.1])
  end

  def dramatic(image) do
    # Create vignette mask using Gaussian distribution
    # This darkens the edges while keeping the center bright
    mask = Operation.gaussmat!(Image.width(image), 0.5, min: 0, max: 0.8)

    image
    # Increase contrast significantly
    |> Operation.linear!([1.4], [-0.2])
    # Enhance edge details for cinematic look
    |> Operation.sharpen!(sigma: 1.0, x1: 2.0)
    # Apply vignette effect using overlay blend
    |> Operation.composite2!(mask, :VIPS_BLEND_MODE_OVERLAY)
  end
end

# Apply our custom filters to sample image
vintage_photo = PhotoFilters.vintage(image)
dramatic_photo = PhotoFilters.dramatic(image)

show([vintage_photo, dramatic_photo])

Performance Considerations

When working with Vix in production environments, keep these performance optimization strategies in mind:

  1. Lazy Operation Execution:

    • Operations are only executed when the final result is needed
    • Chain operations together to minimize intermediate processing
    • Use bang variants (!) when you're confident about inputs and want to avoid error checking overhead

  2. Memory-Efficient Processing:

    # Use sequential access for large images to reduce memory usage
{:ok, image} = Image.new_from_file("large.jpg", access: :sequential)

  3. Optimization Tips:

    • Use thumbnail operations for quick resizes when absolute quality isn't critical
    • Chain operations to avoid creating unnecessary intermediate images
    • Consider format-specific loaders when you need fine-grained control
    • Use appropriate color spaces for your operations (e.g., LAB for color analysis)

Next Steps

To continue your journey with Vix, here are some valuable resources:

  • Documentation:

  • Community:

    • Join the Elixir Forum to discuss Vix with other developers
    • Share your custom filters and processing pipelines with the community

Advanced Topics for Further Exploration

Several advanced features deserve deeper investigation:

  • Complex Convolution Operations:

    • Custom kernel design for specialized filters
    • Multi-pass convolutions for complex effects
    • Edge handling strategies

  • Color Management:

    • Working with ICC profiles for color accuracy
    • Color space transformations
    • Calibration and profiling

  • Animation Support:

    • Processing animated GIFs
    • Creating animations from static images
    • Timeline-based effects

  • Advanced Composition:

    • Complex masking operations
    • Layer blending modes
    • Alpha channel manipulation

Remember that Vix operations are non-destructive - they always create new image objects. This makes it safe to experiment with different processing pipelines while keeping your original images intact. The functional nature of Vix operations makes it easy to compose complex transformations while maintaining code clarity and testability.