LoggerLogfmt

A logfmt formatter for Elixir's Logger that outputs structured logs in the logfmt format.

Overview

LoggerLogfmt provides a simple, text-based structured logging format where each log line consists of key-value pairs. The format is both human-readable and machine-parseable, making it ideal for log aggregation and analysis.

Example output:

timestamp="2024-01-15 10:30:45.123" level=info message="User logged in" user_id=42 request_id=abc123

Features

• 🎯 Flexible Configuration - Customize which fields appear in your logs
• 🔒 Metadata Filtering - Whitelist or blacklist sensitive metadata
• ⏱️ Multiple Timestamp Formats - Elixir, ISO8601, or Unix epoch
• 🔤 Automatic Quoting - Proper escaping of special characters
• 🗂️ Nested Maps - Support for nested structures with dot notation
• 🚀 Zero Dependencies - Pure Elixir implementation

Installation

Add logger_logfmt to your list of dependencies in mix.exs:

def deps do
  [
    {:logger_logfmt, "~> 0.1.0"}
  ]
end

Configuration

Basic Setup

Configure the formatter in your config/config.exs:

config :logger, :console,
  format: {Logger.Backends.Logfmt, :format},
  metadata: [:request_id, :user_id]

config :logger, :logfmt,
  format: [:timestamp, :level, :message, :metadata],
  metadata: [:application, :request_id],
  mode: :whitelist,
  timestamp_format: :iso8601

Format Options

The :format option accepts a list of atoms that determine which fields to include:

Example:

config :logger, :logfmt,
  format: [:timestamp, :level, :message, :metadata]

Metadata Filtering

Whitelist Mode (Default)

Only specified metadata keys will be included:

config :logger, :logfmt,
  metadata: [:request_id, :user_id, :application],
  mode: :whitelist

Blacklist Mode

All metadata except specified keys will be included:

config :logger, :logfmt,
  metadata: [:password, :credit_card, :secret_token],
  mode: :blacklist

Timestamp Formats

Configure the timestamp format for log entries:

# Elixir format: "2024-01-15 10:30:45.123"
config :logger, :logfmt, timestamp_format: :elixir

# ISO8601 format (default): "2024-01-15T10:30:45.123"
config :logger, :logfmt, timestamp_format: :iso8601

# Unix epoch time: 1705318245
config :logger, :logfmt, timestamp_format: :epoch_time

Metadata Timestamp Format

Configure how DateTime/NaiveDateTime values in metadata are formatted:

# Unix epoch time (default): created_at=1705318245
config :logger, :logfmt, metadata_timestamp_format: :epoch_time

# ISO8601 format: created_at="2024-01-15T10:30:45.123456Z"
config :logger, :logfmt, metadata_timestamp_format: :iso8601

# Elixir format: created_at="2024-01-15 10:30:45.123456"
config :logger, :logfmt, metadata_timestamp_format: :elixir

When using :epoch_time, the time unit is auto-detected from the key suffix:

• *_ms → milliseconds
• *_us → microseconds
• *_ns → nanoseconds
• No suffix → seconds

Custom Field Keys

Rename the keys used for standard fields:

config :logger, :logfmt,
  timestamp_key: "ts",
  level_key: "severity",
  message_key: "msg",
  domain_key: "app_domain",
  node_key: "node_name",
  pid_key: "process",
  file_key: "source_file",
  line_key: "source_line"

Usage Examples

Basic Logging

require Logger

Logger.info("Application started")
# Output: timestamp="2024-01-15 10:30:45.123" level=info message="Application started"

Logger.error("Database connection failed")
# Output: timestamp="2024-01-15 10:30:45.123" level=error message="Database connection failed"

Logging with Metadata

Logger.info("User action", user_id: 42, action: "login", ip: "192.168.1.1")
# Output: timestamp="2024-01-15 10:30:45.123" level=info message="User action" user_id=42 action=login ip=192.168.1.1

Logger.warn("High memory usage", memory_mb: 512.5, threshold_mb: 500)
# Output: timestamp="2024-01-15 10:30:45.123" level=warn message="High memory usage" memory_mb=512.5 threshold_mb=500

Logging Nested Data

user = %{name: "John", address: %{city: "Berlin", zip: "10115"}}
Logger.info("User registered", user: user)
# Output: timestamp="2024-01-15 10:30:45.123" level=info message="User registered" user.name=John user.address.city=Berlin user.address.zip=10115

DateTime and NaiveDateTime Values

DateTime and NaiveDateTime values in metadata are automatically formatted based on the metadata_timestamp_format option (defaults to :epoch_time):

Logger.info("Event occurred", created_at: DateTime.utc_now())
# Output: timestamp="..." level=info message="Event occurred" created_at=1701619200

# Time unit is auto-detected from key suffix:
Logger.info("Event", created_at_ms: DateTime.utc_now())  # milliseconds
Logger.info("Event", created_at_us: DateTime.utc_now())  # microseconds
Logger.info("Event", created_at_ns: DateTime.utc_now())  # nanoseconds

Special Characters Handling

Logger.info("Message with \"quotes\" and spaces", tag: "special=chars")
# Output: timestamp="2024-01-15 10:30:45.123" level=info message="Message with \"quotes\" and spaces" tag="special=chars"

Integration Examples

Phoenix Application

In your Phoenix app's config/config.exs:

config :logger, :console,
  format: {Logger.Backends.Logfmt, :format},
  metadata: [:request_id, :user_id]

config :logger, :logfmt,
  format: [:timestamp, :level, :message, :metadata],
  metadata: [:request_id, :user_id, :method, :path],
  mode: :whitelist,
  timestamp_format: :iso8601

Add request metadata in your endpoint:

defmodule MyAppWeb.Endpoint do
  use Phoenix.Endpoint, otp_app: :my_app

  plug Plug.RequestId
  plug Plug.Logger

  plug :add_logger_metadata

  defp add_logger_metadata(conn, _opts) do
    Logger.metadata(
      request_id: Logger.metadata()[:request_id],
      method: conn.method,
      path: conn.request_path
    )
    conn
  end
end

Production Logging

For production environments with log aggregation:

# config/prod.exs
config :logger, :console,
  format: {Logger.Backends.Logfmt, :format},
  metadata: :all

config :logger, :logfmt,
  format: [:timestamp, :level, :message, :node, :pid, :metadata],
  timestamp_format: :iso8601,
  mode: :blacklist,
  metadata: [:gl, :mfa, :__sentry__, :ansi_color]

API Documentation

Full API documentation is available at HexDocs.

Main Modules

• Logger.Backends.Logfmt - Main formatter with configuration options
• Logger.Backends.Logfmt.Encoder - Handles encoding of different data types
• Logger.Backends.Logfmt.Quoter - Manages quoting and escaping

Testing

Run the test suite:

cd libs/logger_logfmt
mix test

Run with coverage:

mix test --cover

Contributing

Contributions are welcome! Please feel free to submit a Pull Request.

License

Apache License 2.0 - See LICENSE file for details.