Copyright © 2020 Antoine Gagné
Authors: Antoine Gagné (gagnantoine@gmail.com).
child() = treewalker_crawlers_sup:child()
url() = treewalker_page:url()
add_crawler/2 | Add a new crawler with the default configuration. |
add_crawler/3 | Add a new crawler with the specified configuration. |
start_crawler/1 | Start the specified crawler. |
stop_crawler/1 | Stop the specified crawler. |
Add a new crawler with the default configuration.
add_crawler(Name::term(), Url::url(), Custom::map()) -> {ok, child()} | {ok, child(), term()} | {error, term()}
Add a new crawler with the specified configuration.
The available options are as follow:
- scraper
: Module implementing the treewalker_scraper
behaviour.
- scraper_options
: The options to pass to the module implementing the
treewalker_scraper
behaviour.
- fetcher
: Module implementing the treewalker_fetcher
behaviour.
- fetcher_options
: The options to pass to the module implementing the
treewalker_fetcher
behaviour.
max_depth
: The max depth that the crawler will crawl.
start_crawler(Name::term()) -> ok
Start the specified crawler.
stop_crawler(Name::term()) -> ok
Stop the specified crawler.
Generated by EDoc