Getting Started

Copy Markdown View Source

This guide walks you through installing TimelessLogs, writing your first log, and querying it back.

Installation

Add to your mix.exs:

def deps do
  [
    {:timeless_logs, "~> 0.10"}
  ]
end

Then fetch dependencies:

mix deps.get

Minimal configuration

TimelessLogs only requires a data directory:

# config/config.exs
config :timeless_logs,
  data_dir: "priv/timeless_logs"

That's it. TimelessLogs installs itself as an Elixir :logger handler on application start. All Logger calls are automatically captured, compressed, and indexed.

Writing your first log

Since TimelessLogs integrates with Elixir's Logger, just log normally:

require Logger

Logger.info("Application started")
Logger.error("Connection timeout", service: "api", path: "/checkout")
Logger.warning("Memory usage high", host: "web-1", usage_pct: 92)

Metadata key/value pairs are always captured with the entry. A small set of stable, low-cardinality keys is also indexed for fast querying, such as service, path, method, status, table, job, cache, reason, and key.

Via HTTP

If you enable the HTTP API, you can also ingest logs via NDJSON:

# config/config.exs
config :timeless_logs,
  data_dir: "priv/timeless_logs",
  http: true  # port 9428, no auth
curl -X POST http://localhost:9428/insert/jsonline -d \
  '{"_msg": "Request completed", "_time": "2024-01-15T10:30:00Z", "level": "info", "request_id": "abc123"}'

Querying logs

Elixir API

# Recent errors
{:ok, result} = TimelessLogs.query(level: :error)
# => {:ok, %TimelessLogs.Result{entries: [...], total: 42, limit: 100, offset: 0}}

# Errors from the last hour
{:ok, result} = TimelessLogs.query(
  level: :error,
  since: DateTime.add(DateTime.utc_now(), -3600))

# Search by indexed metadata
{:ok, result} = TimelessLogs.query(metadata: %{service: "api"})

# Substring search on messages
{:ok, result} = TimelessLogs.query(message: "timeout")

# Combined filters with pagination
{:ok, result} = TimelessLogs.query(
  level: :warning,
  message: "memory",
  limit: 50,
  offset: 0,
  order: :asc)

Each entry in the result is a TimelessLogs.Entry struct:

%TimelessLogs.Entry{
  timestamp: 1700000000000000,  # microseconds
  level: :error,
  message: "Connection timeout",
  metadata: %{"service" => "api", "path" => "/checkout"}
}

Via HTTP

# Recent errors
curl 'http://localhost:9428/select/logsql/query?level=error&limit=50'

# Time range query
curl 'http://localhost:9428/select/logsql/query?level=error&start=2024-01-15T00:00:00Z&end=2024-01-16T00:00:00Z'

# Message search
curl 'http://localhost:9428/select/logsql/query?message=timeout'

Streaming large result sets

For memory-efficient access, use stream/1. Blocks are decompressed on demand:

TimelessLogs.stream(level: :error)
|> Enum.take(10)

TimelessLogs.stream(since: DateTime.add(DateTime.utc_now(), -86400))
|> Stream.filter(fn entry -> String.contains?(entry.message, "timeout") end)
|> Enum.to_list()

Checking storage stats

{:ok, stats} = TimelessLogs.stats()
# => %TimelessLogs.Stats{total_blocks: 48, total_entries: 125_000, disk_size: 24_000_000, ...}

Next steps