Aurinko.Paginator (Aurinko v0.2.1)

Copy Markdown View Source

Lazy stream-based pagination for all Aurinko list endpoints.

Instead of manually tracking next_page_token and next_delta_token, wrap any list function in a stream and consume it lazily.

Usage

# Lazy stream — only fetches pages as consumed
stream = Aurinko.Paginator.stream(token, &Aurinko.Email.list_messages/2,
  limit: 50,
  q: "is:unread"
)

# Take the first 100 messages across all pages
messages = stream |> Stream.take(100) |> Enum.to_list()

# Or collect all into memory
all_messages = Enum.to_list(stream)

# Stream calendar events
events =
  Aurinko.Paginator.stream(token, fn t, opts ->
    Aurinko.Calendar.list_events(t, "primary", opts)
  end, time_min: ~U[2024-01-01 00:00:00Z], time_max: ~U[2024-12-31 23:59:59Z])
  |> Enum.to_list()

Sync streaming

For delta-sync streams, use sync_stream/4:

# Streams all updated records (pages handled automatically)
{:ok, sync} = Aurinko.Email.start_sync(token, days_within: 30)

Aurinko.Paginator.sync_stream(token, sync.sync_updated_token, fn t, delta_or_page_token, is_page ->
  if is_page do
    Aurinko.Email.sync_updated(token, delta_or_page_token)
  else
    Aurinko.Email.sync_updated(token, delta_or_page_token)
  end
end)
|> Stream.each(&process_message/1)
|> Stream.run()

Summary

Functions

Collect all pages synchronously and return a flat list of records.

Create a lazy Stream that automatically paginates through all pages.

Create a lazy Stream over a sync-updated or sync-deleted endpoint.

Types

fetch_fn()

@type fetch_fn() :: (String.t(), keyword() ->
                 {:ok, Aurinko.Types.Pagination.t()} | {:error, term()})

Functions

collect_all(token, fetch_fn, opts \\ [])

@spec collect_all(String.t(), fetch_fn(), keyword()) ::
  {:ok, list()} | {:error, term()}

Collect all pages synchronously and return a flat list of records.

Convenience wrapper around stream/3 for when you want all results.

stream(token, fetch_fn, opts \\ [])

@spec stream(String.t(), fetch_fn(), keyword()) :: Enumerable.t()

Create a lazy Stream that automatically paginates through all pages.

The stream yields individual records (not page structs). Pages are fetched on demand as the stream is consumed.

Parameters

  • token — Aurinko access token
  • fetch_fn — A function (token, opts) -> {:ok, %Pagination{}}. Any list API function works.
  • opts — Options forwarded to fetch_fn on every call (e.g. :limit, :q)

Options

  • :on_error:halt (default) or :skip — what to do on an API error mid-stream

sync_stream(token, initial_token, fetch_fn, opts \\ [])

@spec sync_stream(
  String.t(),
  String.t(),
  (String.t(), String.t() ->
     {:ok, Aurinko.Types.Pagination.t()} | {:error, term()}),
  keyword()
) :: Enumerable.t()

Create a lazy Stream over a sync-updated or sync-deleted endpoint.

Handles next_page_token (more pages in this sync batch) and next_delta_token (batch complete — use this token next time).

Returns individual records. The final delta token is NOT yielded as a record — use the :on_delta callback to capture it.

Options

  • :on_delta(delta_token -> any) — called when a new delta token is received