treewalker v0.4.1

View Source treewalker_scraper behaviour (treewalker v0.4.1)

Scrap the page for useful data and links.

Link to this section Summary

Types

content/0
options/0
page_data/0
url/0

Callbacks

scrap/3
scrap_links/3

Functions

scrap(Url, PageData, ScrappingOptions)
scrap_links(Url, PageData, ScrappingOptions)

Link to this section Types

Link to this type

content/0

View Source 
-type content() :: treewalker_page:content().
Link to this type

options/0

View Source 
-type options() :: term().
Link to this type

page_data/0

View Source 
-type page_data() :: binary().
Link to this type

url/0

View Source 
-type url() :: treewalker_page:url().

Link to this section Callbacks

Link to this callback

scrap/3

View Source 
-callback scrap(Url :: url(), PageData :: page_data(), ScrappingOptions :: options()) ->
         {ok, Scrapped :: content()} | {error, Reason :: term()}.
Link to this callback

scrap_links/3

View Source 
-callback scrap_links(Url :: url(), PageData :: page_data(), ScrappingOptions :: options()) ->
               {ok, Links :: [url()]} | {error, Reason :: term()}.

Link to this section Functions

Link to this function

scrap(Url, PageData, ScrappingOptions)

View Source 
-spec scrap(url(), page_data(), options()) -> {ok, content()}.
Link to this function

scrap_links(Url, PageData, ScrappingOptions)

View Source 
-spec scrap_links(url(), page_data(), options()) -> {ok, [url()]} | {error, term()}.