Req (req v0.2.1) View Source

Req is an HTTP client with a focus on ease of use and composability, built on top of Finch.

Features

Usage

The easiest way to use Req is with Mix.install/2 (requires Elixir v1.12+):

Mix.install([
  {:req, "~> 0.2.0"}
])

Req.get!("https://api.github.com/repos/elixir-lang/elixir").body["description"]
#=> "Elixir is a dynamic, functional language designed for building scalable and maintainable applications"

If you want to use Req in a Mix project, you can add the above dependency to your mix.exs.

Low-level API

Under the hood, Req works by passing a request through a series of steps.

The request struct, %Req.Request{}, initially contains data like HTTP method and request headers. You can also add request, response, and error steps to it.

Request steps are used to refine the data that will be sent to the server.

After making the actual HTTP request, we'll either get a HTTP response or an error. The request, along with the response or error, will go through response or error steps, respectively.

Nothing is actually executed until we run the pipeline with Req.run/1.

The high-level API shown before:

Req.get!("https://api.github.com/repos/elixir-lang/elixir")

is equivalent to this composition of lower-level API functions and steps:

Req.build(:get, "https://api.github.com/repos/elixir-lang/elixir")
|> Req.put_default_steps()
|> Req.run!()

(See Req.build/3, Req.put_default_steps/2, and Req.run!/1 for more information.)

We can also build more complex flows like returning a response from a request step or an error from a response step. We will explore those next.

Request steps

A request step is a function that accepts a request and returns one of the following:

  • A request

  • A {request, response_or_error} tuple. In that case no further request steps are executed and the return value goes through response or error steps

Examples:

def put_default_headers(request) do
  update_in(request.headers, &[{"user-agent", "req"} | &1])
end

def read_from_cache(request) do
  case ResponseCache.fetch(request) do
    {:ok, response} -> {request, response}
    :error -> request
  end
end

Response and error steps

A response step is a function that accepts a {request, response} tuple and returns one of the following:

  • A {request, response} tuple

  • A {request, exception} tuple. In that case, no further response steps are executed but the exception goes through error steps

Similarly, an error step is a function that accepts a {request, exception} tuple and returns one of the following:

  • A {request, exception} tuple

  • A {request, response} tuple. In that case, no further error steps are executed but the response goes through response steps

Examples:

def decode({request, response}) do
  case List.keyfind(response.headers, "content-type", 0) do
    {_, "application/json" <> _} ->
      {request, update_in(response.body, &Jason.decode!/1)}

    _ ->
      {request, response}
  end
end

def log_error({request, exception}) do
  Logger.error(["#{request.method} #{request.uri}: ", Exception.message(exception)])
  {request, exception}
end

Halting

Any step can call Req.Request.halt/1 to halt the pipeline. This will prevent any further steps from being invoked.

Examples:

def circuit_breaker(request) do
  if CircuitBreaker.open?() do
    {Req.Request.halt(request), RuntimeError.exception("circuit breaker is open")}
  else
    request
  end
end

Link to this section Summary

High-level API

Returns default options.

Sets default options.

Makes a DELETE request.

Makes a GET request.

Makes a POST request.

Makes a PUT request.

Makes an HTTP request.

Makes an HTTP request and returns a response or raises an error.

Low-level API

Appends error steps.

Appends request steps.

Appends response steps.

Builds a request pipeline.

Prepends error steps.

Prepends request steps.

Prepends response steps.

Runs a request pipeline.

Runs a request pipeline and returns a response or raises an error.

Make the HTTP request using Finch.

Request steps

Sets request authentication.

Adds common request headers.

Encodes the request body based on its shape.

Encodes request headers.

Sets request authentication for a matching host from a netrc file.

Sets base URL for all requests.

Handles HTTP cache using if-modified-since header.

Adds params to request query string.

Sets the "Range" request header.

Runs the given steps.

Response steps

Decodes response body based on the detected format.

Decompresses the response body based on the content-encoding header.

Follows redirects.

Error steps

Retries a request in face of errors.

Link to this section High-level API

Returns default options.

See default_options/1 for more information.

Link to this function

default_options(options)

View Source

Sets default options.

The default options are used by get!/2, post!/3, put!/3, delete!/2, request/3, and request!/3 functions.

Avoid setting default options in libraries as they are global.

Link to this function

delete!(url, options \\ [])

View Source

Makes a DELETE request.

See request/3 for a list of supported options.

Link to this function

get!(url, options \\ [])

View Source

Makes a GET request.

See request/3 for a list of supported options.

Link to this function

post!(url, body, options \\ [])

View Source

Makes a POST request.

See request/3 for a list of supported options.

Link to this function

put!(url, body, options \\ [])

View Source

Makes a PUT request.

See request/3 for a list of supported options.

Link to this function

request(method, url, options \\ [])

View Source

Makes an HTTP request.

Options

  • :headers - request headers, defaults to []

  • :body - request body, defaults to ""

  • :finch - Finch pool to use, defaults to Req.Finch which is automatically started by the application. See Finch module documentation for more information on starting pools.

  • :finch_options - Options passed down to Finch when making the request, defaults to []. See Finch.request/3 for more information.

The options are passed down to put_default_steps/2, see its documentation for more information how they are being used.

The options are merged with default options set with default_options/1.

Link to this function

request!(method, url, options \\ [])

View Source

Makes an HTTP request and returns a response or raises an error.

See request/3 for more information.

Link to this section Low-level API

Link to this function

append_error_steps(request, steps)

View Source

Appends error steps.

Link to this function

append_request_steps(request, steps)

View Source

Appends request steps.

Link to this function

append_response_steps(request, steps)

View Source

Appends response steps.

Link to this function

build(method, url, options \\ [])

View Source

Builds a request pipeline.

Options

  • :header - request headers, defaults to []

  • :body - request body, defaults to ""

  • :finch - Finch pool to use, defaults to Req.Finch which is automatically started by the application. See Finch module documentation for more information on starting pools.

  • :finch_options - Options passed down to Finch when making the request, defaults to []. See Finch.request/3 for more information.

Link to this function

prepend_error_steps(request, steps)

View Source

Prepends error steps.

Link to this function

prepend_request_steps(request, steps)

View Source

Prepends request steps.

Link to this function

prepend_response_steps(request, steps)

View Source

Prepends response steps.

Runs a request pipeline.

Returns {:ok, response} or {:error, exception}.

Runs a request pipeline and returns a response or raises an error.

See run/1 for more information.

Make the HTTP request using Finch.

This is a request step but it is not documented as such because you don't need to add it to your request pipeline. It is automatically added by run/1 as always the very last request step.

This function shows you that making the actual HTTP call is just another request step, which means that you can write your own step that uses another underlying HTTP client like :httpc, :hackney, etc.

Link to this section Request steps

Sets request authentication.

auth can be one of:

  • {username, password} - uses Basic HTTP authentication

Examples

iex> Req.get!("https://httpbin.org/basic-auth/foo/bar", auth: {"bad", "bad"}).status
401
iex> Req.get!("https://httpbin.org/basic-auth/foo/bar", auth: {"foo", "bar"}).status
200
Link to this function

default_headers(request)

View Source

Adds common request headers.

Currently the following headers are added:

  • "user-agent" - "req/0.2.1"

  • "accept-encoding" - "gzip"

Encodes the request body based on its shape.

If body is of the following shape, it's encoded and its content-type set accordingly. Otherwise it's unchanged.

ShapeEncoderContent-Type
{:form, data}URI.encode_query/1"application/x-www-form-urlencoded"
{:json, data}Jason.encode_to_iodata!/1"application/json"

Examples

iex> Req.post!("https://httpbin.org/post", {:form, comments: "hello!"}).body["form"]
%{"comments" => "hello!"}

Encodes request headers.

Turns atom header names into strings, replacing - with _. For example, :user_agent becomes "user-agent". Non-atom header names are kept as is.

If a header value is a NaiveDateTime or DateTime, it is encoded as "HTTP date". Otherwise, the header value is encoded with String.Chars.to_string/1.

Examples

iex> Req.get!("https://httpbin.org/user-agent", headers: [user_agent: :my_agent]).body
%{"user-agent" => "my_agent"}
Link to this function

load_netrc(request, path)

View Source

Sets request authentication for a matching host from a netrc file.

Examples

iex> Req.get!("https://httpbin.org/basic-auth/foo/bar").status
401
iex> Req.get!("https://httpbin.org/basic-auth/foo/bar", netrc: true).status
200
iex> Req.get!("https://httpbin.org/basic-auth/foo/bar", netrc: "/path/to/custom_netrc").status
200
Link to this function

put_base_url(request, base_url)

View Source

Sets base URL for all requests.

Examples

iex> options = [base_url: "https://httpbin.org"]
iex> Req.get!("/status/200", options).status
200
iex> Req.get!("/status/201", options).status
201
Link to this function

put_default_steps(request, options \\ [])

View Source

Adds default steps.

Request steps

Response steps

Error steps

Options

Link to this function

put_if_modified_since(request, options \\ [])

View Source

Handles HTTP cache using if-modified-since header.

Only successful (200 OK) responses are cached.

This step also prepends a response step that loads and writes the cache. Be careful when prepending other response steps, make sure the cache is loaded/written as soon as possible.

Options

  • :dir - the directory to store the cache, defaults to <user_cache_dir>/req (see: :filename.basedir/3)

Examples

iex> url = "https://hexdocs.pm/elixir/Kernel.html"
iex> response1 = Req.get!(url, cache: true)
iex> response2 = Req.get!(url, cache: true)
iex> response1 == response2
true
Link to this function

put_params(request, params)

View Source

Adds params to request query string.

Examples

iex> Req.get!("https://httpbin.org/anything/query", params: [x: "1", y: "2"]).body["args"]
%{"x" => "1", "y" => "2"}
Link to this function

put_range(request, range)

View Source

Sets the "Range" request header.

range can be one of the following:

  • a string - returned as is

  • a first..last range - converted to "bytes=<first>-<last>"

Examples

iex> Req.get!("https://repo.hex.pm/builds/elixir/builds.txt", range: 0..67)
%Req.Response{
  status: 206,
  headers: [{"content-range", "bytes 0-67/45400"}, ...],
  body: "master df65074a8143cebec810dfb91cafa43f19dcdbaf 2021-04-23T15:36:18Z"
}
Link to this function

run_steps(request, steps)

View Source

Runs the given steps.

Examples

iex> inspect_host = fn request -> IO.inspect(request.url.host) ; request end
iex> Req.get!("https://httpbin.org/status/200", steps: [inspect_host]).status
Outputs: httpbin.org
iex> 200

Link to this section Response steps

Decodes response body based on the detected format.

Supported formats:

FormatDecoder
jsonJason.decode!/1
gzip:zlib.gunzip/1
tar:erl_tar.extract/2
zip:zip.unzip/2
csvNimbleCSV.RFC4180.parse_string/2 (if NimbleCSV is installed)

Examples

iex> Req.get!("https://hex.pm/api/packages/finch").body["meta"]
%{
  "description" => "An HTTP client focused on performance.",
  "licenses" => ["MIT"],
  "links" => %{"GitHub" => "https://github.com/keathley/finch"},
  ...
}

Decompresses the response body based on the content-encoding header.

Examples

iex> response = Req.get!("https://httpbin.org/gzip")
iex> response.headers
[
  {"content-encoding", "gzip"},
  {"content-type", "application/json"},
  ...
]
iex> response.body
%{
  "gzipped" => true,
  ...
}

Follows redirects.

Examples

iex> Req.get!("http://api.github.com").status
# 23:24:11.670 [debug]  Req.follow_redirects/2: Redirecting to https://api.github.com/
200

Link to this section Error steps

Retries a request in face of errors.

This function can be used as either or both response and error step. It retries a request that resulted in:

  • a response with status 5xx

  • an exception

Options

  • :delay - sleep this number of milliseconds before making another attempt, defaults to 2000

  • :max_retries - maximum number of retry attempts, defaults to 2 (for a total of 3 requests to the server, including the initial one.)

Examples

With default options:

iex> Req.get!("https://httpbin.org/status/500,200", retry: true).status
# 19:02:08.463 [error] Req.retry/3: Got response with status 500. Will retry in 2000ms, 2 attempts left
# 19:02:10.710 [error] Req.retry/3: Got response with status 500. Will retry in 2000ms, 1 attempt left
200

With custom options:

iex> Req.get!("http://localhost:9999", retry: [delay: 100, max_retries: 3])
# 17:00:38.371 [error] Req.retry/3: Got exception. Will retry in 100ms, 3 attempts left
# 17:00:38.371 [error] ** (Mint.TransportError) connection refused
# 17:00:38.473 [error] Req.retry/3: Got exception. Will retry in 100ms, 2 attempts left
# 17:00:38.473 [error] ** (Mint.TransportError) connection refused
# 17:00:38.575 [error] Req.retry/3: Got exception. Will retry in 100ms, 1 attempt left
# 17:00:38.575 [error] ** (Mint.TransportError) connection refused
** (Mint.TransportError) connection refused