Specs

scrape(String.t(), map()) :: list()[tuple()]

Scrape Dealer Rater Reviews

This function attempts to scrape reviews from the passed in url and returns a list of tuples, the first element being the review itself and the second element containing the username of the reviewer.

The simple css selectors employed are .review-content for the content of the review itself, and the combination of .italic and .font-18 for the username of the reviewer. This is an intentionally chosen shortcut. A slightly more robust approach might use the .review-container selector instead since it would seem less likely to change. I found it relatively bloated and so opted for a quicker approach that felt more elegant.

A future approach might include user-defined selectors rather than hard coded ones, but as the use case is currently very narrowly defined (solely scraping reviews from deallerrater.com) this approach seemed unnecessarily complicated.

Examples

iex> url = "https://web.archive.org/web/20201127110830/https://www.dealerrater.com/dealer/McKaig-Chevrolet-Buick-A-Dealer-For-The-People-dealer-reviews-23685/"
iex> reviews = EvaluateReview.scrape(url, [])
iex> reviews |> Enum.with_index() |> Enum.each(fn {{a, b},_} -> IO.puts("review: #{a}, reviewer: #{b}") end)