Supports securely downloading HTTPS content.
This module provides HTTP GET functionality using the built-in :httpc
client with certificate verification enabled by default. It follows
the erlef security guidelines
for secure TLS connections.
The primary public API consists of:
get/2- download content from a URL, returning the body on success.get_with_headers/2- download content from a URL, returning both headers and body on success.certificate_locations/0- return the list of possible certificate store locations.
Summary
Functions
Return all possible locations of a certificate file.
Return the dynamically discovered certificate file locations.
Securely download HTTPS content from a URL.
Securely download HTTPS content from a URL, returning headers and body.
Functions
@spec certificate_locations() :: [String.t()]
Return all possible locations of a certificate file.
Returns dynamically discovered locations followed by well-known static locations on common operating systems.
Returns
- A list of file path strings for all candidate certificate locations.
@spec dynamic_certificate_locations() :: [String.t()]
Return the dynamically discovered certificate file locations.
These include application configuration and optional hex packages
such as CAStore and certifi.
Returns
- A list of file path strings for discovered certificate locations.
@spec get(String.t() | {String.t(), list()}, options :: keyword()) :: {:ok, binary()} | {:not_modified, any()} | {:error, any()}
Securely download HTTPS content from a URL.
This function uses the built-in :httpc client but enables certificate
verification which is not enabled by :httpc by default.
See also https://erlef.github.io/security-wg/secure_coding_and_deployment_hardening/ssl
Arguments
urlis a binary URL or a{url, list_of_headers}tuple. If provided the headers are a list of{'header_name', 'header_value'}tuples. Note that the name and value are both charlists, not strings.optionsis a keyword list of options.
Options
:verify_peeris a boolean value indicating if peer verification should be done for this request. The default istruein which case the default:ssloptions follow the erlef guidelines.:timeoutis the number of milliseconds available for the request to complete. The default is "120000". This option may also be set with theLOCALIZE_HTTP_TIMEOUTenvironment variable.:connection_timeoutis the number of milliseconds available for a connection to be established to the remote host. The default is "60000". This option may also be set with theLOCALIZE_HTTP_CONNECTION_TIMEOUTenvironment variable.
Returns
{:ok, body}if the return is successful.{:not_modified, headers}if the request would result in returning the same results as one matching an etag.{:error, error}if the download is unsuccessful. An error will also be logged in these cases.
Unsafe HTTPS
If the environment variable LOCALIZE_UNSAFE_HTTPS is set to anything
other than "FALSE", "false", "nil" or "NIL" then no peer
verification of certificates is performed. Setting this variable is
not recommended but may be required where peer verification fails for
unidentified reasons.
Certificate stores
In order to keep dependencies to a minimum, get/2 attempts to locate
an already installed certificate store. It will try to locate a store
in the following order which is intended to satisfy most host systems.
The certificate store is expected to be a path name on the host system.
# A certificate store configured by the developer
Application.get_env(:localize, :cacertfile)
# Populated if hex package `CAStore` is configured
CAStore.file_path()
# Populated if hex package `certifi` is configured
:certifi.cacertfile()
# Debian/Ubuntu/Gentoo etc.
"/etc/ssl/certs/ca-certificates.crt"
# Fedora/RHEL 6
"/etc/pki/tls/certs/ca-bundle.crt"
# OpenSUSE
"/etc/ssl/ca-bundle.pem"
# OpenELEC
"/etc/pki/tls/cacert.pem"
# CentOS/RHEL 7
"/etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem"
# OpenSSL on MacOS
"/usr/local/etc/openssl/cert.pem"
# MacOS & Alpine Linux
"/etc/ssl/cert.pem"
@spec get_with_headers(String.t() | {String.t(), list()}, options :: keyword()) :: {:ok, list(), binary()} | {:not_modified, any()} | {:error, any()}
Securely download HTTPS content from a URL, returning headers and body.
This function uses the built-in :httpc client but enables certificate
verification which is not enabled by :httpc by default.
See also https://erlef.github.io/security-wg/secure_coding_and_deployment_hardening/ssl
Arguments
urlis a binary URL or a{url, list_of_headers}tuple. If provided the headers are a list of{'header_name', 'header_value'}tuples. Note that the name and value are both charlists, not strings.optionsis a keyword list of options.
Options
:verify_peeris a boolean value indicating if peer verification should be done for this request. The default istruein which case the default:ssloptions follow the erlef guidelines.:timeoutis the number of milliseconds available for the request to complete. The default is "120000". This option may also be set with theLOCALIZE_HTTP_TIMEOUTenvironment variable.:connection_timeoutis the number of milliseconds available for a connection to be established to the remote host. The default is "60000". This option may also be set with theLOCALIZE_HTTP_CONNECTION_TIMEOUTenvironment variable.:https_proxyis the URL of an HTTPS proxy to be used. The default isnil.
Returns
{:ok, headers, body}if the return is successful.{:not_modified, headers}if the request would result in returning the same results as one matching an etag.{:error, error}if the download is unsuccessful. An error will also be logged in these cases.
HTTPS Proxy
Localize.Utils.Http.get_with_headers/2 will look for a proxy URL in
the following locations in the order presented:
options[:https_proxy]Localize compile-time configuration under the key
:localize[:https_proxy].The environment variable
HTTPS_PROXY.The environment variable
https_proxy.