Crawler.Fetcher.Recorder (Crawler v1.1.2) View Source

Records information about each crawl for internal use.

Link to this section Summary

Functions

Records information about each crawl for internal use.

Stores page data in Crawler.Store.DB for internal or external consumption.

Link to this section Functions

Records information about each crawl for internal use.

Examples

iex> Recorder.record(url: "url1", depth: 2)
{:ok, %{depth: 3, url: "url1"}}

iex> Recorder.record(url: "url2", depth: 2)
iex> Store.find("url2")
%Page{url: "url2"}

Stores page data in Crawler.Store.DB for internal or external consumption.