Module to simplify access to configuration values with default values.

There should be no configuration required to start using :ref_inspector if you rely on the default values:

config :ref_inspector,
  database_files: ["referers.yml"],
  database_path: Application.app_dir(:ref_inspector, "priv"),
  http_opts: [],
  remote_urls: [{"referers.yml", "https://s3-eu-west-1.amazonaws.com/snowplow-hosted-assets/third-party/referer-parser/referers-latest.yaml"}],
  startup_silent: false,
  startup_sync: true,
  yaml_file_reader: {:yamerl_constr, :file, [[:str_node_as_binary]]}

The default :database_path is evaluated at runtime and not compiled into a release!

how-to-configure

How to Configure

There are two ways to change the configuration values with the preferred way depending on your environment and personal taste.

static-configuration

Static Configuration

If you can ensure the configuration are static and not dependent on i.e. the server your application is running on, you can use a static approach by modifying your config.exs file:

config :ref_inspector,
  database_files: ["referers_search.yml", "referers_social.yml"],
  database_path: "/path/to/ref_inspector/database_files"

dynamic-configuration

Dynamic Configuration

If a compile time configuration is not possible or does not match the usual approach taken in your application you can use a runtime approach.

This is done by defining an initializer module that will automatically be called by RefInspector.Supervisor upon startup/restart. The configuration is expected to consist of a {mod, fun} or {mod, fun, args} tuple:

# {mod, fun}
config :ref_inspector,
  init: {MyInitModule, :my_init_mf}

# {mod, fun, args}
config :ref_inspector,
  init: {MyInitModule, :my_init_mfargs, [:foo, :bar]}

defmodule MyInitModule do
  @spec my_init_mf() :: :ok
  def my_init_mf(), do: my_init_mfargs(:foo, :bar)

  @spec my_init_mfargs(atom, atom) :: :ok
  def my_init_mfargs(:foo, :bar) do
    priv_dir = Application.app_dir(:my_app, "priv")

    Application.put_env(:ref_inspector, :database_path, priv_dir)
  end
end

The function is required to always return :ok.

startup-behaviour

Startup Behaviour

Databases are loaded synchronously when starting the application.

You can change this behaviour to have the application force an asynchronous database loading during the initial startup:

config :ref_inspector,
  startup_sync: false

This can lead to the first parsing calls to work with an empty database and therefore not return the results you expect.

starting-silently

Starting Silently

When starting the application you will receive warnings if the database is not available. If you want to hide these messages you can configure the startup the be completely silent:

config :ref_inspector,
  startup_silent: true

database-configuration

Database Configuration

Configuring the database to use can be done using three related values:

• :database_files
• :database_path
• :remote_urls

The :database_path is the directory to look for when loading the databases. It is also the place where RefInspector.Downloader stores a copy of the configured files.

For the actual files loaded there is :database_files, a list of filenames to load in the order specified. All files are expected to be inside the configured database path.

When downloading the databases through RefInspector.Downloader the value :remote_urls is of utmost importance. It defines where each file is located.

config :ref_inspector,
  remote_urls: [
    "http://example.com/database.yml",
    {"database_local.yml", "http://example.com/database_remote.yml"}
  ]

To configure a remote database name you can either define a plain URL. It will be stored locally under the filename that is extracted from the url. In above example that would be "database.yml".

If the remote and local names match you can configure a {local, remote} tuple to deactivate the automatic name extraction.

internal-domains

Internal Domains

To exclude some domains from parsing you can mark them as :internal using your configuration:

config :ref_inspector,
  internal: ["www.example.com", "www.example.org"]

If a referer matches one of the configured domains (== ends with, paths ignored!), it will return a result with the medium :internal. Both :source and :term will be left at the initial/unknown state not intended for further processing.

download-configuration

Download Configuration

Using the default configuration all download requests for your database files are done using :hackney. To pass custom configuration values to hackney you can use the key :http_opts:

config :ref_inspector,
  http_opts: [proxy: "http://mycompanyproxy.com"]

Please see :hackney.request/5 for a complete list of available options.

If you want to change the library used to download the databases you can configure a module implementing the RefInspector.Downloader.Adapter behaviour:

config :ref_inspector,
  downloader_adapter: MyDownloaderAdapter

yaml-file-reader-configuration

YAML File Reader Configuration

By default the library :yamerl will be used to read and decode the yaml database files. You can configure this reader to be a custom module:

config :ref_inspector,
  yaml_file_reader: {module, function}

config :ref_inspector,
  yaml_file_reader: {module, function, extra_args}

The configured module will receive the file to read as the first argument with any optionally configured extra arguments after that.