ExPmtiles (ExPmtiles v0.3.4)

View Source

A module for working with PMTiles files.

PMTiles is a single-file format for storing tiled map data. This module provides functionality to read and access tiles from PMTiles files stored either locally or on S3.

Features

  • Read PMTiles files from local storage or S3
  • Access tiles by zoom level and coordinates (z/x/y)
  • Automatic directory caching and decompression
  • Support for various compression types (gzip, none)
  • Tile ID calculations and conversions

Usage

# Open a local PMTiles file
instance = ExPmtiles.new("path/to/file.pmtiles", :local)

# Open a PMTiles file from S3
instance = ExPmtiles.new("region", "my-bucket", "path/to/file.pmtiles", :s3)

# Get a tile by coordinates
case ExPmtiles.get_zxy(instance, 10, 512, 256) do
  {{offset, length, data}, updated_instance} ->
    # Use the tile data
    data
  {nil, updated_instance} ->
    # Tile not found
    nil
end

# Convert coordinates to tile ID
tile_id = ExPmtiles.zxy_to_tile_id(10, 512, 256)

# Convert tile ID back to coordinates
{z, x, y} = ExPmtiles.tile_id_to_zxy(tile_id)

Compression Support

The module supports the following compression types:

  • :none - No compression
  • :gzip - Gzip compression
  • :unknown - Unknown compression type

Note: Brotli and Zstd compression are not currently supported.

Tile Types

Supported tile types include:

  • :mvt - Mapbox Vector Tiles
  • :png - PNG images
  • :jpg - JPEG images
  • :webp - WebP images
  • :avif - AVIF images
  • :unknown - Unknown tile type

HTTP Connection Pooling

For optimal S3 performance, this module uses a dedicated Hackney connection pool. The pool is automatically started when the application starts. You can configure the pool size via application configuration:

config :ex_pmtiles,
  http_pool_size: 100,  # Maximum number of connections (default: 100)
  http_timeout: 15_000  # Request timeout in milliseconds (default: 15_000)

Summary

Functions

decompress(data, compression_type)

Decompresses data based on the specified compression type.

deserialize_directory(buf)

Deserializes a PMTiles directory from binary data.

find_tile(entries, tile_id)

Finds a tile entry in a directory by tile ID.

get_bytes(instance, offset, length)

Retrieves bytes from the PMTiles file using the configured storage backend.

get_tile(instance, offset, length)

Gets raw tile data from the PMTiles file at the specified offset and length.

get_zxy(instance, z, x, y, dir_cache_path \\ nil, pending_directories_table \\ nil)

Gets a tile by its zoom level and coordinates.

new(path, atom)

Creates a new PMTiles instance for a locally-stored file.

new(region, bucket, path, atom)

Creates a new PMTiles instance for an S3-stored file.

parse_header(arg)

Parses the PMTiles file header from binary data.

read_varint(binary, shift \\ 0, result \\ 0)

Reads a variable-length integer from binary data.

tile_id_to_zxy(tile_id)

Converts a PMTiles tile ID back to zoom level and coordinates.

zxy_to_tile_id(z, x, y)

Converts zoom level and coordinates to a PMTiles tile ID.

Functions

decompress(data, compression_type)

Decompresses data based on the specified compression type.

Parameters

  • data - The compressed binary data
  • compression_type - The compression type (:gzip, :none, etc.)

Returns

  • binary() - The decompressed data

Raises

  • Error for unsupported compression types (brotli, zstd)

Examples

iex> ExPmtiles.decompress(compressed_data, :gzip)
<<...>>

iex> ExPmtiles.decompress(data, :none)
data

deserialize_directory(buf)

Deserializes a PMTiles directory from binary data.

PMTiles directories contain entries with tile IDs, offsets, lengths, and run lengths. This function parses the binary format into a list of directory entries.

Parameters

  • buf - Binary data containing the directory

Returns

  • list() - List of directory entries with keys: :tile_id, :offset, :length, :run_length

Examples

iex> ExPmtiles.deserialize_directory(<<...>>)
[
  %{tile_id: 0, offset: 0, length: 1024, run_length: 1},
  %{tile_id: 1, offset: 1024, length: 512, run_length: 0}
]

find_tile(entries, tile_id)

Finds a tile entry in a directory by tile ID.

Uses binary search to efficiently locate the tile entry. Handles run-length encoding where multiple consecutive tiles may share the same entry.

Parameters

  • entries - List of directory entries
  • tile_id - The tile ID to search for

Returns

  • map() - The matching directory entry
  • nil - If no matching entry is found

Examples

iex> ExPmtiles.find_tile(entries, 1024)
%{tile_id: 1024, offset: 2048, length: 512, run_length: 1}

get_bytes(instance, offset, length)

Retrieves bytes from the PMTiles file using the configured storage backend.

Parameters

  • instance - The PMTiles instance
  • offset - Byte offset in the file
  • length - Number of bytes to read

Returns

  • binary() - The requested bytes
  • nil - If the bytes cannot be read

Examples

iex> ExPmtiles.get_bytes(instance, 0, 1024)
<<...>>

get_tile(instance, offset, length)

Gets raw tile data from the PMTiles file at the specified offset and length.

Parameters

  • instance - The PMTiles instance
  • offset - Byte offset in the file
  • length - Number of bytes to read

Returns

  • binary() - The raw tile data
  • nil - If the data cannot be read

Examples

iex> ExPmtiles.get_tile(instance, 1024, 512)
<<...>>

get_zxy(instance, z, x, y, dir_cache_path \\ nil, pending_directories_table \\ nil)

Gets a tile by its zoom level and coordinates.

This function converts the z/x/y coordinates to a tile ID and searches for the tile in the PMTiles directory structure. It handles both leaf and internal directories automatically.

Parameters

  • instance - The PMTiles instance
  • z - Zoom level (integer)
  • x - X coordinate (integer)
  • y - Y coordinate (integer)

Returns

  • {{offset, length, data}, updated_instance} - Tuple containing tile information and updated instance
  • {nil, updated_instance} - If tile is not found or outside zoom bounds

Examples

iex> ExPmtiles.get_zxy(instance, 10, 512, 256)
{{1024, 512, <<...>>}, updated_instance}

iex> ExPmtiles.get_zxy(instance, 25, 0, 0)
{nil, instance}  # Zoom level out of bounds

new(path, atom)

Creates a new PMTiles instance for a locally-stored file.

Parameters

  • path - The local file path to the PMTiles file
  • :local - Source type identifier

Returns

  • %ExPmtiles{} - A configured PMTiles instance with parsed header
  • nil - If the file cannot be accessed or is invalid

Examples

iex> ExPmtiles.new("data/world.pmtiles", :local)
%ExPmtiles{bucket: nil, path: "data/world.pmtiles", source: :local, header: %{...}}

new(region, bucket, path, atom)

Creates a new PMTiles instance for an S3-stored file.

Parameters

  • bucket - The S3 bucket name
  • path - The path to the PMTiles file within the bucket
  • :s3 - Source type identifier

Returns

  • %ExPmtiles{} - A configured PMTiles instance with parsed header
  • nil - If the file cannot be accessed or is invalid

Examples

iex> ExPmtiles.new("my-bucket", "maps/world.pmtiles", :s3)
%ExPmtiles{bucket: "my-bucket", path: "maps/world.pmtiles", source: :s3, header: %{...}}

parse_header(arg)

Parses the PMTiles file header from binary data.

The PMTiles header contains metadata about the file including offsets, compression settings, zoom levels, and geographic bounds.

Parameters

  • binary - Binary data containing the PMTiles header (first 16,384 bytes)

Returns

  • map() - Header information with keys:
    • :magic_number - File magic number ("PMTiles")
    • :spec_version - PMTiles specification version
    • :root_offset, :root_length - Root directory location
    • :metadata_offset, :metadata_length - Metadata location
    • :leaf_dir_offset, :leaf_dir_length - Leaf directories location
    • :tile_data_offset, :tile_data_length - Tile data location
    • :num_addr_tiles, :num_tile_entries, :num_tile_contents - Tile counts
    • :clustered? - Whether tiles are clustered
    • :internal_compression, :tile_compression - Compression types
    • :tile_type - Type of tiles stored
    • :min_zoom, :max_zoom - Zoom level bounds
    • :min_position, :max_position - Geographic bounds
    • :center_zoom, :center_position - Center point

Examples

iex> ExPmtiles.parse_header(header_binary)
%{
  magic_number: "PMTiles",
  spec_version: 3,
  min_zoom: 0,
  max_zoom: 14,
  ...
}

read_varint(binary, shift \\ 0, result \\ 0)

Reads a variable-length integer from binary data.

PMTiles uses variable-length integers (varints) for efficient encoding of small numbers. This function reads one varint from the beginning of the binary.

Parameters

  • binary - Binary data containing the varint
  • shift - Internal parameter for bit shifting (default: 0)
  • result - Internal parameter for accumulating result (default: 0)

Returns

  • {integer(), binary()} - Tuple of the decoded integer and remaining binary
  • Raises error if binary is empty or malformed

Examples

iex> ExPmtiles.read_varint(<<1>>)
{1, ""}

iex> ExPmtiles.read_varint(<<128, 1>>)
{128, ""}

tile_id_to_zxy(tile_id)

Converts a PMTiles tile ID back to zoom level and coordinates.

Performs the inverse operation of zxy_to_tile_id/3, converting a tile ID back to its corresponding z/x/y coordinates.

Parameters

  • tile_id - The tile ID to convert

Returns

  • {z, x, y} - Tuple of zoom level and coordinates

Raises

Examples

iex> ExPmtiles.tile_id_to_zxy(1048576)
{10, 512, 256}

iex> ExPmtiles.tile_id_to_zxy(0)
{0, 0, 0}

zxy_to_tile_id(z, x, y)

Converts zoom level and coordinates to a PMTiles tile ID.

Uses the Hilbert curve mapping to convert z/x/y coordinates to a unique tile ID. This is the standard method used by PMTiles for tile identification.

Parameters

  • z - Zoom level (integer, 0-26)
  • x - X coordinate (integer, 0 to 2^z - 1)
  • y - Y coordinate (integer, 0 to 2^z - 1)

Returns

  • integer() - The tile ID

Raises

  • ArgumentError - If zoom level exceeds 26 or coordinates are out of bounds

Examples

iex> ExPmtiles.zxy_to_tile_id(10, 512, 256)
1048576

iex> ExPmtiles.zxy_to_tile_id(0, 0, 0)
0