Crawly.Fetchers.CrawlyRenderServer (Crawly v0.17.0) View Source

Implements Crawly.Fetchers.Fetcher behavior for Crawly Render Server Javascript rendering.

Crawly Render Server is a lightweight puppeteer based Javascript rendering engine server. Quite experimental. See more: https://github.com/elixir-crawly/crawly-render-server

It exposes /render endpoint that renders JS on incoming requests. For example: curl -X POST http://localhost:3000/render -H 'Content-Type: application/json' -d '{

 "url": "https://example.com",
 "headers": {"User-Agent": "Custom User Agent"}

}'

In this case you have to configure the fetcher in the following way: fetcher: {Crawly.Fetchers.CrawlyRenderServer, [base_url: "http://localhost:3000/render"]}

Link to this section Summary

Link to this section Functions

Link to this function

fetch(request, client_options)

View Source

Callback implementation for Crawly.Fetchers.Fetcher.fetch/2.