Crawler
v1.5.0

View Source Crawler.Dispatcher (Crawler v1.5.0)

Dispatches requests to a queue for crawling.

Summary

Functions

dispatch(request, opts)

Takes the request argument which is a tuple containing either

Functions

Link to this function

dispatch(request, opts)

View Source

Takes the request argument which is a tuple containing either:

  • {_, link, _, url} when it's a link that got transformed into a URL
  • {_, url} when it's a URL already

And issues Crawler.crawl/2 to initiate the crawl.