View Source OffBroadway.Splunk.Producer (off_broadway_splunk v2.1.2)

GenStage Producer for a Splunk Event Stream. Broadway producer acts as a consumer for Splunk report or alerts.

Producer Options

  • :name - Required. The report or alert name for the Splunk job we want to consume events from.

  • :receive_interval (non_neg_integer/0) - The duration (in milliseconds) for which the producer waits before making a request for more messages. The default value is 5000.

  • :refetch_interval (non_neg_integer/0) - The duration (in milliseconds) to wait before fetching new jobs to be processed. The default value is 60000.

  • :only_new (boolean/0) - If set to true, the pipeline will skip adding any existing jobs to the initial queue. The default value is false.

  • :only_latest (boolean/0) - If set to true, the pipeline will only add the most recent job to the initial queue. The default value is false.

  • :shutdown_timeout (timeout/0) - The duration (in milliseconds) Broadway should wait before timing out when trying to stop the pipeline. The default value is :infinity.

  • :on_success (atom/0) - Configures the acking behaviour for successful messages. See the "Acknowledgements" section below for all the possible values. The default value is :ack.

  • :on_failure (atom/0) - Configures the acking behaviour for failed messages. See the "Acknowledgements" section below for all the possible values. The default value is :noop.

  • :splunk_client - A module that implements the OffBroadway.Splunk.Client behaviour. This module is responsible for fetching and acknowledging the messages from Splunk. All options passed to the producer will also be forwarded to the client. The default value is OffBroadway.Splunk.SplunkClient.

  • :config (keyword/0) - A set of config options that overrides the default config for the splunk_client module. Any option set here can also be configured in config.exs. The default value is [].

    • :base_url (String.t/0) - Base URL to Splunk instance.

    • :api_token (String.t/0) - API token used to authenticate on the Splunk instance.

    • :api_version - Some API endpoints are available in multiple versions. Sets the API version to use (where applicable). The default value is "v2".

    • :max_events - If set to a positive integer, automatically shut down the pipeline after consuming max_events messages from the Splunk API.

Acknowledgements

You can use the on_success and on_failure options to control how messages are acknowledged. You can set these options when starting the Splunk producer or change them for each message through Broadway.Message.configure_ack/2. By default, successful messages are acked (:ack) and failed messages are not (:noop).

The possible values for :on_success and :on_failure are:

  • :ack - acknowledge the message. Splunk does not have any concept of acking messages, because we are just consuming messages from a web api endpoint. For now we are just executing a :telemetry event for acked messages.

  • :noop - do not acknowledge the message. No action are taken.

Telemetry

This library exposes the following telemetry events:

  • [:off_broadway_splunk, :receive_jobs, :start] - Dispatched before fetching jobs from Splunk.

    • measurement: %{time: System.monotonic_time}
    • metadata: %{name: string, count: integer}

  • [:off_broadway_splunk, :receive_jobs, :stop] - Dispatched when fetching jobs from Splunk is complete.

    • measurement: %{time: native_time}
    • metadata: %{name: string, count: integer}

  • [:off_broadway_splunk, :receive_jobs, :exception] - Dispatched after a failure while fetching jobs from Splunk.

    • measurement: %{duration: native_time}

    • metadata:

      %{
  name: string,
  reason: reason,
  stacktrace: stacktrace
}

  • [:off_broadway_splunk, :process_job, :start] - Dispatched before starting to process messages for a job.

    • measurement: %{time: System.system_time}
    • metadata: %{name: string, sid: string}

  • [:off_broadway_splunk, :process_job, :stop] - Dispatched after all messages have been processed for a job.

    • measurement: %{time: System.system_time, processed_events: integer, processed_requests: integer}
    • metadata: %{name: string, sid: string}

  • [:off_broadway_splunk, :receive_messages, :start] - Dispatched before receiving messages from Splunk.

    • measurement: %{time: System.monotonic_time}
    • metadata: %{name: string, sid: string, demand: integer}

  • [:off_broadway_splunk, :receive_messages, :stop] - Dispatched after messages have been received from Splunk and "wrapped".

    • measurement: %{time: native_time}

    • metadata:

      %{
  name: string,
  sid: string,
  received: integer,
  demand: integer
}

  • [:off_broadway_splunk, :receive_messages, :exception] - Dispatched after a failure while receiving messages from Splunk.

    • measurement: %{duration: native_time}

    • metadata:

      %{
  name: string,
  sid: string,
  demand: integer,
  reason: reason,
  stacktrace: stacktrace
}

  • [:off_broadway_splunk, :receive_messages, :ack] - Dispatched when acking a message.

    • measurement: %{time: System.system_time, count: 1}

    • meatadata:

      %{
  name: string,
  receipt: receipt
}