Crawler.Fetcher.Recorder (Crawler v1.1.2) View Source

Records information about each crawl for internal use.

Link to this section Summary

Functions

record(opts)

Records information about each crawl for internal use.

store_page(body, opts)

Stores page data in Crawler.Store.DB for internal or external consumption.

Link to this section Functions

record(opts)

Records information about each crawl for internal use.

Examples

iex> Recorder.record(url: "url1", depth: 2)
{:ok, %{depth: 3, url: "url1"}}

iex> Recorder.record(url: "url2", depth: 2)
iex> Store.find("url2")
%Page{url: "url2"}

store_page(body, opts)

Stores page data in Crawler.Store.DB for internal or external consumption.