View Source Wiki.EventStreams (mediawiki_client v0.4.8)

This module reads from an infinite stream of server-sent events annotating actions such as editing or patrolling, as they happen on Wikimedia projects.

For more about the public wiki streams and their format, see EventStreams on Wikitech

Examples

Start reading the page creation feed, and expose as a GenStage.stream:

Wiki.EventStreams.start_link(streams: "page-create")
Wiki.EventStreams.stream()
|> Stream.take(6)
|> Enum.to_list

Combine multiple feeds,

Wiki.EventStreams.start_link(streams: ["revision-create", "revision-score"])
Wiki.EventStreams.stream()
|> Stream.take(6)
|> Enum.to_list

TODO

  • Currently only a single supervisor tree is supported, so calling applications can only read from one stream.
  • Track the restart ID, disconnect from the feed at some maximum queue size. Reconnect as demand resumes. Application-lifetime or permanent storage for the restart ID tracking, for consumers that need an at-least-once guarantee.

Summary

Types

client_option()
client_options()
  • :adapter - Override the HTTP adapter.
  • :endpoint - Override the default endpoint URL.
  • :stream_to - Optional application which will receive the events, otherwise they go to the process starting the EventStream.
  • :streams - One or more atoms with the stream names to subscribe to.
  • :user_agent - Custom user-agent header string

Functions

child_spec(init_arg)

Returns a specification to start this module under a supervisor.

start_link(args)

Start a supervisor tree to receive and relay server-side events.

stream(options \\ [])

Indefinitely capture subscribed events and relay them as a Stream.

Types

Link to this type

client_option()

View Source 
@type client_option() ::
  {:adapter, module()}
  | {:endpoint, binary()}
  | {:stream_to, GenServer.server()}
  | {:streams, atom() | [atom()]}
  | {:user_agent, binary()}
Link to this type

client_options()

View Source 
@type client_options() :: [client_option()]
  • :adapter - Override the HTTP adapter.
  • :endpoint - Override the default endpoint URL.
  • :stream_to - Optional application which will receive the events, otherwise they go to the process starting the EventStream.
  • :streams - One or more atoms with the stream names to subscribe to.
  • :user_agent - Custom user-agent header string

Functions

Link to this function

child_spec(init_arg)

View Source

Returns a specification to start this module under a supervisor.

See Supervisor.

Link to this function

start_link(args)

View Source 
@spec start_link(client_options()) :: GenServer.on_start()

Start a supervisor tree to receive and relay server-side events.

Arguments

  • options - Keyword list,
    • {:adapter, module} - Override HTTPoison adapter.
    • {:endpoint, url} - Override default endpoint.

    • {:send_to, pid | module} - Instead of using the built-in streaming relay, send the events directly to your own process.

    • {:streams, atom | [atom]} - Select which streams to listen to. An updated list can be found here. Required.

Link to this function

stream(options \\ [])

View Source 
@spec stream(client_options()) :: Enumerable.t()

Indefinitely capture subscribed events and relay them as a Stream.