Crawler v0.2.0 Crawler.Fetcher.Policer
Checks a series of conditions to determine whether it is okay to continue,
i.e. to allow Crawler.Fetcher.fetch/1 to begin its tasks.
Link to this section Summary
Functions
Examples
Link to this section Functions
Link to this function
police(opts)
Examples
iex> Policer.police([depth: 1, max_depths: 2, url: "http://policer/"])
{:ok, [depth: 1, max_depths: 2, url: "http://policer/"]}
iex> Policer.police([depth: 2, max_depths: 2])
{:error, "Fetch failed 'within_fetch_depth?', with opts: [depth: 2, max_depths: 2]."}
iex> Policer.police([depth: 1, max_depths: 2, url: "ftp://hello.world"])
{:error, "Fetch failed 'acceptable_uri_scheme?', with opts: [depth: 1, max_depths: 2, url: \"ftp://hello.world\"]."}
iex> Crawler.Store.add("http://policer/exist/")
iex> Policer.police([depth: 1, max_depths: 2, url: "http://policer/exist/"])
{:error, "Fetch failed 'not_fetched_yet?', with opts: [depth: 1, max_depths: 2, url: \"http://policer/exist/\"]."}