View Source Crawler.HTTP (Crawler v1.5.0)

Custom HTTPoison base module for potential customisation.

Summary

Functions

Issues a DELETE request to the given url.

Issues a DELETE request to the given url, raising an exception in case of failure.

Issues a GET request to the given url.

Issues a GET request to the given url, raising an exception in case of failure.

Issues a HEAD request to the given url.

Issues a HEAD request to the given url, raising an exception in case of failure.

Issues an OPTIONS request to the given url.

Issues a OPTIONS request to the given url, raising an exception in case of failure.

Issues a PATCH request to the given url.

Issues a PATCH request to the given url, raising an exception in case of failure.

Issues a POST request to the given url.

Issues a POST request to the given url, raising an exception in case of failure.

Callback implementation for HTTPoison.Base.process_headers/1.

process_url(url) deprecated

Callback implementation for HTTPoison.Base.process_url/1.

Issues a PUT request to the given url.

Issues a PUT request to the given url, raising an exception in case of failure.

Issues an HTTP request using an HTTPoison.Request struct.

Issues an HTTP request with the given method to the given url.

Issues an HTTP request with the given method to the given url, raising an exception in case of failure.

Starts HTTPoison and its dependencies.

Requests the next message to be streamed for a given HTTPoison.AsyncResponse.

Types

Functions

Link to this function

delete(url, headers \\ [], options \\ [])

View Source

Issues a DELETE request to the given url.

Returns {:ok, response} if the request is successful, {:error, reason} otherwise.

See request/5 for more detailed information.

Link to this function

delete!(url, headers \\ [], options \\ [])

View Source

Issues a DELETE request to the given url, raising an exception in case of failure.

If the request does not fail, the response is returned.

See request!/5 for more detailed information.

Link to this function

get(url, headers \\ [], options \\ [])

View Source
@spec get(binary(), headers(), Keyword.t()) ::
  {:ok, HTTPoison.Response.t() | HTTPoison.AsyncResponse.t()}
  | {:error, HTTPoison.Error.t()}

Issues a GET request to the given url.

Returns {:ok, response} if the request is successful, {:error, reason} otherwise.

See request/5 for more detailed information.

Link to this function

get!(url, headers \\ [], options \\ [])

View Source

Issues a GET request to the given url, raising an exception in case of failure.

If the request does not fail, the response is returned.

See request!/5 for more detailed information.

Link to this function

head(url, headers \\ [], options \\ [])

View Source
@spec head(binary(), headers(), Keyword.t()) ::
  {:ok, HTTPoison.Response.t() | HTTPoison.AsyncResponse.t()}
  | {:error, HTTPoison.Error.t()}

Issues a HEAD request to the given url.

Returns {:ok, response} if the request is successful, {:error, reason} otherwise.

See request/5 for more detailed information.

Link to this function

head!(url, headers \\ [], options \\ [])

View Source

Issues a HEAD request to the given url, raising an exception in case of failure.

If the request does not fail, the response is returned.

See request!/5 for more detailed information.

Link to this function

options(url, headers \\ [], options \\ [])

View Source

Issues an OPTIONS request to the given url.

Returns {:ok, response} if the request is successful, {:error, reason} otherwise.

See request/5 for more detailed information.

Link to this function

options!(url, headers \\ [], options \\ [])

View Source

Issues a OPTIONS request to the given url, raising an exception in case of failure.

If the request does not fail, the response is returned.

See request!/5 for more detailed information.

Link to this function

patch(url, body, headers \\ [], options \\ [])

View Source

Issues a PATCH request to the given url.

Returns {:ok, response} if the request is successful, {:error, reason} otherwise.

See request/5 for more detailed information.

Link to this function

patch!(url, body, headers \\ [], options \\ [])

View Source

Issues a PATCH request to the given url, raising an exception in case of failure.

If the request does not fail, the response is returned.

See request!/5 for more detailed information.

Link to this function

post(url, body, headers \\ [], options \\ [])

View Source

Issues a POST request to the given url.

Returns {:ok, response} if the request is successful, {:error, reason} otherwise.

See request/5 for more detailed information.

Link to this function

post!(url, body, headers \\ [], options \\ [])

View Source

Issues a POST request to the given url, raising an exception in case of failure.

If the request does not fail, the response is returned.

See request!/5 for more detailed information.

Link to this function

process_headers(headers)

View Source
This function is deprecated. Use process_response_headers/1 instead.
@spec process_headers(list()) :: any()

Callback implementation for HTTPoison.Base.process_headers/1.

Link to this function

process_request_body(body)

View Source
@spec process_request_body(body()) :: body()

Callback implementation for HTTPoison.Base.process_request_body/1.

Link to this function

process_request_headers(headers)

View Source
@spec process_request_headers(headers()) :: headers()

Callback implementation for HTTPoison.Base.process_request_headers/1.

Link to this function

process_request_options(options)

View Source
@spec process_request_options(options()) :: options()

Callback implementation for HTTPoison.Base.process_request_options/1.

Link to this function

process_request_params(params)

View Source
@spec process_request_params(params()) :: params()

Callback implementation for HTTPoison.Base.process_request_params/1.

Link to this function

process_request_url(url)

View Source
@spec process_request_url(url()) :: url()

Callback implementation for HTTPoison.Base.process_request_url/1.

Link to this function

process_response(response)

View Source
@spec process_response(HTTPoison.Base.response()) :: any()

Callback implementation for HTTPoison.Base.process_response/1.

Link to this function

process_response_body(body)

View Source
@spec process_response_body(binary()) :: any()

Callback implementation for HTTPoison.Base.process_response_body/1.

Link to this function

process_response_chunk(chunk)

View Source
@spec process_response_chunk(binary()) :: any()

Callback implementation for HTTPoison.Base.process_response_chunk/1.

Link to this function

process_response_headers(headers)

View Source
@spec process_response_headers(list()) :: any()

Callback implementation for HTTPoison.Base.process_response_headers/1.

Link to this function

process_response_status_code(status_code)

View Source
@spec process_response_status_code(integer()) :: any()

Callback implementation for HTTPoison.Base.process_response_status_code/1.

Link to this function

process_status_code(status_code)

View Source
This function is deprecated. Use process_response_status_code/1 instead.
@spec process_status_code(integer()) :: any()

Callback implementation for HTTPoison.Base.process_status_code/1.

This function is deprecated. Use process_request_url/1 instead.
@spec process_url(url()) :: url()

Callback implementation for HTTPoison.Base.process_url/1.

Link to this function

put(url, body \\ "", headers \\ [], options \\ [])

View Source

Issues a PUT request to the given url.

Returns {:ok, response} if the request is successful, {:error, reason} otherwise.

See request/5 for more detailed information.

Link to this function

put!(url, body \\ "", headers \\ [], options \\ [])

View Source

Issues a PUT request to the given url, raising an exception in case of failure.

If the request does not fail, the response is returned.

See request!/5 for more detailed information.

Issues an HTTP request using an HTTPoison.Request struct.

This function returns {:ok, response}, {:ok, async_response}, or {:ok, maybe_redirect} if the request is successful, {:error, reason} otherwise.

Redirect handling

If the option :follow_redirect is given, HTTP redirects are automatically follow if the method is set to :get or :head and the response's status_code is 301, 302 or 307.

If the method is set to :post, then the only status_code that get's automatically followed is 303.

If any other method or status_code is returned, then this function returns a returns a {:ok, %HTTPoison.MaybeRedirect{}} containing the redirect_url for you to re-request with the method set to :get.

Examples

request = %HTTPoison.Request{
  method: :post,
  url: "https://my.website.com",
  body: "{\"foo\": 3}",
  headers: [{"Accept", "application/json"}]
}

request(request)
Link to this function

request(method, url, body \\ "", headers \\ [], options \\ [])

View Source

Issues an HTTP request with the given method to the given url.

This function is usually used indirectly by get/3, post/4, put/4, etc

Args:

  • method - HTTP method as an atom (:get, :head, :post, :put, :delete, etc.)
  • url - target url as a binary string or char list
  • body - request body. See more below
  • headers - HTTP headers as an orddict (e.g., [{"Accept", "application/json"}])
  • options - Keyword list of options

Body: see type HTTPoison.Request

Options: see type HTTPoison.Request

This function returns {:ok, response}, {:ok, async_response}, or {:ok, maybe_redirect} if the request is successful, {:error, reason} otherwise.

Redirect handling

If the option :follow_redirect is given, HTTP redirects are automatically follow if the method is set to :get or :head and the response's status_code is 301, 302 or 307.

If the method is set to :post, then the only status_code that get's automatically followed is 303.

If any other method or status_code is returned, then this function returns a returns a {:ok, %HTTPoison.MaybeRedirect{}} containing the redirect_url for you to re-request with the method set to :get.

Examples

request(:post, "https://my.website.com", "{\"foo\": 3}", [{"Accept", "application/json"}])
Link to this function

request!(method, url, body \\ "", headers \\ [], options \\ [])

View Source

Issues an HTTP request with the given method to the given url, raising an exception in case of failure.

request!/5 works exactly like request/5 but it returns just the response in case of a successful request, raising an exception in case the request fails.

Starts HTTPoison and its dependencies.

@spec stream_next(HTTPoison.AsyncResponse.t()) ::
  {:ok, HTTPoison.AsyncResponse.t()} | {:error, HTTPoison.Error.t()}

Requests the next message to be streamed for a given HTTPoison.AsyncResponse.

See request!/5 for more detailed information.