| Overview |  |
Copyright © 2020 Antoine Gagné
Authors: Antoine Gagné (gagnantoine@gmail.com).
child() = treewalker_crawlers_sup:child()
url() = treewalker_page:url()
| add_crawler/2 | Add a new crawler with the default configuration. |
| add_crawler/3 | Add a new crawler with the specified configuration. |
| start_crawler/1 | Start the specified crawler. |
| stop_crawler/1 | Stop the specified crawler. |
Add a new crawler with the default configuration.
add_crawler(Name::term(), Url::url(), Custom::map()) -> {ok, child()} | {ok, child(), term()} | {error, term()}
Add a new crawler with the specified configuration.
The available options are as follow:
- scraper: Module implementing the treewalker_scraper behaviour.
- scraper_options: The options to pass to the module implementing the
treewalker_scraper behaviour.
- fetcher: Module implementing the treewalker_fetcher behaviour.
- fetcher_options: The options to pass to the module implementing the
treewalker_fetcher behaviour.
max_depth: The max depth that the crawler will crawl.
start_crawler(Name::term()) -> ok
Start the specified crawler.
stop_crawler(Name::term()) -> ok
Stop the specified crawler.
Generated by EDoc