View Source Crawler.Example.GoogleSearch (Crawler v1.5.0)
This example performs a Google search, then scrapes the results to find Github projects and output their name and description.
Example output:
Agent.get(Data, & &1) #=> %{
"crawler" => %{
desc: "A high performance web crawler / scraper in Elixir.",
url: "https://github.com/fredwu/crawler"
},
"crawly" => %{
desc: "Crawly, a high-level web crawling & scraping framework for Elixir.",
url: "https://github.com/elixir-crawly/crawly"
},
"elixir_scraper" => %{
desc: "Elixir/Hound web scraper example",
url: "https://github.com/jaydorsey/elixir_scraper"
},
"mechanize" => %{
desc: "Build web scrapers and automate interaction with websites in Elixir with ease!",
url: "https://github.com/gushonorato/mechanize"
}
}