ExAws v1.1.1 ExAws.S3

Operations on AWS S3

Basic Operations

The vast majority of operations here represent a single operation on S3.

Examples

S3.list_objects |> ExAws.request! #=> {:ok, %{body: [list, of, objects]}}
S3.list_objects |> ExAws.stream! |> Enum.to_list #=> [list, of, objects]

S3.put_object("my-bucket", "path/to/bucket", contents) |> ExAws.request!

Higher Level Operations

There are also some operations which operate at a higher level to make it easier to download and upload very large files.

Multipart uploads

"path/to/big/file"
|> S3.Upload.stream_file
|> S3.upload("my-bucket", "path/on/s3")
|> ExAws.request! #=> {:ok, :done}

Download large file to disk

S3.download_file("my-bucket", "path/on/s3", "path/to/dest/file")
|> ExAws.request! #=> {:on, :done}

More high level functionality

Task.async_stream makes some high level flows so easy you don’t need explicit ExAws support.

For example, here is how to concurrently upload many files.

upload_file = fn {src_path, dest_path} ->
  S3.put_object("my_bucket", dest_path, File.read!(src_path))
  |> ExAws.request!
end

paths = %{"path/to/src0" => "path/to/dest0", "path/to/src1" => "path/to/dest1"}

paths
|> Task.async_stream(upload_file, max_concurrency: 10)
|> Stream.run

Summary

Functions

Delete a bucket

Delete a bucket cors

Delete a bucket lifecycle

Delete a bucket policy

Delete a bucket replication

Delete a bucket tagging

Delete a bucket website

Delete multiple objects within a bucket

Delete object object in bucket

Download an S3 Object to a file

Get bucket acl

Get bucket cors

Get bucket lifecycle

Get bucket location

Get bucket logging

Get bucket notification

Get bucket policy

Get bucket replication

Get bucket payment configuration

Get bucket tagging

Get bucket versioning

Get bucket website

Get an object from a bucket

Get an object’s access control policy

Get a torrent for a bucket

Determine if a bucket exists

Determine of an object exists

List multipart uploads for a bucket

List objects in bucket

List the parts of a multipart upload

Restore an object to a particular version

Generates a pre-signed URL for this object

Creates a bucket in the specified region

Update or create a bucket bucket access control

Update or create a bucket CORS policy

Update or create a bucket lifecycle configuration

Update or create a bucket logging configuration

Update or create a bucket notification configuration

Update or create a bucket policy configuration

Update or create a bucket replication configuration

Update or create a bucket requestPayment configuration

Update or create a bucket tagging configuration

Update or create a bucket versioning configuration

Update or create a bucket website configuration

Create an object within a bucket

Create or update an object’s access control FIXME

Types

acl_opts()
acl_opts() :: [{:acl, canned_acl} | grant]
amz_meta_opts()
amz_meta_opts() :: [{atom, binary} | {binary, binary}, ...]
canned_acl()
canned_acl ::
  :private |
  :public_read |
  :public_read_write |
  :authenticated_read |
  :bucket_owner_read |
  :bucket_owner_full_control
customer_encryption_opts()
customer_encryption_opts() :: [customer_algorithm: binary, customer_key: binary, customer_key_md5: binary]
download_file_opts()
download_file_opts() :: [max_concurrency: pos_integer, chunk_size: pos_integer, timeout: 60000]
encryption_opts()
encryption_opts ::
  binary |
  [{:aws_kms_key_id, binary}] |
  customer_encryption_opts
get_object_opts()
get_object_opts() :: [{:response, get_object_response_opts} | head_object_opts]
get_object_response_opts()
get_object_response_opts() :: [content_language: binary, expires: binary, cach_control: binary, content_disposition: binary, content_encoding: binary]
grant()
grant ::
  {:grant_read, grantee} |
  {:grant_read_acp, grantee} |
  {:grant_write_acp, grantee} |
  {:grant_full_control, grantee}
grantee()
grantee() :: [email: binary, id: binary, uri: binary]
head_object_opts()
head_object_opts() :: [encryption: customer_encryption_opts, range: binary, if_modified_since: binary, if_unmodified_since: binary, if_match: binary, if_none_match: binary]
initiate_multipart_upload_opts()
initiate_multipart_upload_opts() :: [{:cache_control, binary} | {:content_disposition, binary} | {:content_encoding, binary} | {:content_type, binary} | {:expires, binary} | {:storage_class, :standard | :redunced_redundancy} | {:website_redirect_location, binary} | {:encryption, encryption_opts} | acl_opts]
list_objects_opts()
list_objects_opts() :: [delimiter: binary, marker: binary, prefix: binary, encoding_type: binary, max_keys: 0..1000]
presigned_url_opts()
presigned_url_opts() :: [expires_in: integer, virtual_host: boolean, query_params: [{binary, binary}]]
pub_object_copy_opts()
pub_object_copy_opts() :: [{:metadata_directive, :COPY | :REPLACE} | {:copy_source_if_modified_since, binary} | {:copy_source_if_unmodified_since, binary} | {:copy_source_if_match, binary} | {:copy_source_if_none_match, binary} | {:website_redirect_location, binary} | {:destination_encryption, encryption_opts} | {:source_encryption, customer_encryption_opts} | {:cache_control, binary} | {:content_disposition, binary} | {:content_encoding, binary} | {:content_length, binary} | {:content_type, binary} | {:expect, binary} | {:expires, binary} | {:storage_class, :standard | :redunced_redundancy} | {:website_redirect_location, binary} | {:meta, amz_meta_opts} | acl_opts]
put_object_opts()
put_object_opts() :: [{:cache_control, binary} | {:content_disposition, binary} | {:content_encoding, binary} | {:content_length, binary} | {:content_type, binary} | {:expect, binary} | {:expires, binary} | {:storage_class, :standard | :redunced_redundancy} | {:website_redirect_location, binary} | {:encryption, encryption_opts} | {:meta, amz_meta_opts} | acl_opts]
upload_opts()
upload_opts() :: [{:max_concurrency, pos_integer} | initiate_multipart_upload_opts]
upload_part_copy_opts()
upload_part_copy_opts() :: [copy_source_range: Range.t, copy_source_if_modified_since: binary, copy_source_if_unmodified_since: binary, copy_source_if_match: binary, copy_source_if_none_match: binary, destination_encryption: encryption_opts, source_encryption: customer_encryption_opts]

Functions

abort_multipart_upload(bucket, object, upload_id)
abort_multipart_upload(bucket :: binary, object :: binary, upload_id :: binary) :: ExAws.Operation.S3.t

Abort a multipart upload

complete_multipart_upload(bucket, object, upload_id, parts)
complete_multipart_upload(bucket :: binary, object :: binary, upload_id :: binary, parts :: [{binary | pos_integer, binary}, ...]) :: ExAws.Operation.S3.t

Complete a multipart upload

delete_all_objects(bucket, objects, opts \\ [])
delete_all_objects(bucket :: binary, objects :: [binary | {binary, binary}, ...], opts :: [{:quiet, true}]) :: ExAws.Operation.S3DeleteAllObjects.t

Delete all listed objects.

When performed, this function will continue making delete_multiple_objects requests deleting 1000 objects at a time until all are deleted.

Can be streamed.

delete_bucket(bucket)
delete_bucket(bucket :: binary) :: ExAws.Operation.S3.t

Delete a bucket

delete_bucket_cors(bucket)
delete_bucket_cors(bucket :: binary) :: ExAws.Operation.S3.t

Delete a bucket cors

delete_bucket_lifecycle(bucket)
delete_bucket_lifecycle(bucket :: binary) :: ExAws.Operation.S3.t

Delete a bucket lifecycle

delete_bucket_policy(bucket)
delete_bucket_policy(bucket :: binary) :: ExAws.Operation.S3.t

Delete a bucket policy

delete_bucket_replication(bucket)
delete_bucket_replication(bucket :: binary) :: ExAws.Operation.S3.t

Delete a bucket replication

delete_bucket_tagging(bucket)
delete_bucket_tagging(bucket :: binary) :: ExAws.Operation.S3.t

Delete a bucket tagging

delete_bucket_website(bucket)
delete_bucket_website(bucket :: binary) :: ExAws.Operation.S3.t

Delete a bucket website

delete_multiple_objects(bucket, objects, opts \\ [])
delete_multiple_objects(bucket :: binary, objects :: [binary | {binary, binary}, ...], opts :: [{:quiet, true}]) :: ExAws.Operation.S3.t

Delete multiple objects within a bucket

Limited to 1000 objects.

delete_object(bucket, object, opts \\ [])

Delete object object in bucket

download_file(bucket, path, dest, opts \\ [])
download_file(bucket :: binary, path :: binary, dest :: binary, opts :: download_file_opts) :: ExAws.S3.Download.t

Download an S3 Object to a file.

This operation download multiple parts of an S3 object concurrently, allowing you to maximize throughput.

Defaults to a concurrency of 8, chunk size of 1MB, and a timeout of 1 minute.

get_bucket_acl(bucket)
get_bucket_acl(bucket :: binary) :: ExAws.Operation.S3.t

Get bucket acl

get_bucket_cors(bucket)
get_bucket_cors(bucket :: binary) :: ExAws.Operation.S3.t

Get bucket cors

get_bucket_lifecycle(bucket)
get_bucket_lifecycle(bucket :: binary) :: ExAws.Operation.S3.t

Get bucket lifecycle

get_bucket_location(bucket)
get_bucket_location(bucket :: binary) :: ExAws.Operation.S3.t

Get bucket location

get_bucket_logging(bucket)
get_bucket_logging(bucket :: binary) :: ExAws.Operation.S3.t

Get bucket logging

get_bucket_notification(bucket)
get_bucket_notification(bucket :: binary) :: ExAws.Operation.S3.t

Get bucket notification

get_bucket_object_versions(bucket, opts \\ [])
get_bucket_object_versions(bucket :: binary, opts :: Keyword.t) :: ExAws.Operation.S3.t

Get bucket object versions

get_bucket_policy(bucket)
get_bucket_policy(bucket :: binary) :: ExAws.Operation.S3.t

Get bucket policy

get_bucket_replication(bucket)
get_bucket_replication(bucket :: binary) :: ExAws.Operation.S3.t

Get bucket replication

get_bucket_request_payment(bucket)
get_bucket_request_payment(bucket :: binary) :: ExAws.Operation.S3.t

Get bucket payment configuration

get_bucket_tagging(bucket)
get_bucket_tagging(bucket :: binary) :: ExAws.Operation.S3.t

Get bucket tagging

get_bucket_versioning(bucket)
get_bucket_versioning(bucket :: binary) :: ExAws.Operation.S3.t

Get bucket versioning

get_bucket_website(bucket)
get_bucket_website(bucket :: binary) :: ExAws.Operation.S3.t

Get bucket website

get_object(bucket, object, opts \\ [])
get_object(bucket :: binary, object :: binary, opts :: get_object_opts) :: ExAws.Operation.S3.t

Get an object from a bucket

get_object_acl(bucket, object, opts \\ [])
get_object_acl(bucket :: binary, object :: binary, opts :: Keyword.t) :: ExAws.Operation.S3.t

Get an object’s access control policy

get_object_torrent(bucket, object)
get_object_torrent(bucket :: binary, object :: binary) :: ExAws.Operation.S3.t

Get a torrent for a bucket

head_bucket(bucket)
head_bucket(bucket :: binary) :: ExAws.Operation.S3.t

Determine if a bucket exists

head_object(bucket, object, opts \\ [])
head_object(bucket :: binary, object :: binary, opts :: head_object_opts) :: ExAws.Operation.S3.t

Determine of an object exists

initiate_multipart_upload(bucket, object, opts \\ [])
initiate_multipart_upload(bucket :: binary, object :: binary, opts :: initiate_multipart_upload_opts) :: ExAws.Operation.S3.t

Initiate a multipart upload

list_buckets(opts \\ [])
list_buckets(opts :: Keyword.t) :: ExAws.Operation.S3.t

List buckets

list_multipart_uploads(bucket, opts \\ [])
list_multipart_uploads(bucket :: binary, opts :: Keyword.t) :: ExAws.Operation.S3.t

List multipart uploads for a bucket

list_objects(bucket, opts \\ [])
list_objects(bucket :: binary, opts :: list_objects_opts) :: ExAws.Operation.S3.t

List objects in bucket

Can be streamed.

Examples

S3.list_objects("my-bucket") |> ExAws.request

S3.list_objects("my-bucket") |> ExAws.stream!
S3.list_objects("my-bucket", delimiter: "/", prefix: "backup") |> ExAws.stream!
S3.list_objects("my-bucket", prefix: "some/inner/location/path") |> ExAws.stream!
S3.list_objects("my-bucket", max_keys: 5, encoding_type: "url") |> ExAws.stream!
list_parts(bucket, object, upload_id, opts \\ [])
list_parts(bucket :: binary, object :: binary, upload_id :: binary, opts :: Keyword.t) :: ExAws.Operation.S3.t

List the parts of a multipart upload

options_object(bucket, object, origin, request_method, request_headers \\ [])
options_object(bucket :: binary, object :: binary, origin :: binary, request_method :: atom, request_headers :: [binary]) :: ExAws.Operation.S3.t

Determine the CORS configuration for an object

post_object_restore(bucket, object, number_of_days, opts \\ [])
post_object_restore(bucket :: binary, object :: binary, number_of_days :: pos_integer, opts :: [{:version_id, binary}]) :: ExAws.Operation.S3.t

Restore an object to a particular version

presigned_url(config, http_method, bucket, object, opts \\ [])
presigned_url(config :: %{}, http_method :: atom, bucket :: binary, object :: binary, opts :: presigned_url_opts) ::
  {:ok, binary} |
  {:error, binary}

Generates a pre-signed URL for this object.

When option param :virtual_host is true, the {#bucket} name will be used as the hostname. This will cause the returned URL to be ‘http’ and not ‘https’.

Additional (signed) query parameters can be added to the url by setting option param :query_params to a list of {"key", "value"} pairs. Useful if you are uploading parts of a multipart upload directly from the browser.

put_bucket(bucket, region, opts \\ [])

Creates a bucket in the specified region

put_bucket_acl(bucket, grants)
put_bucket_acl(bucket :: binary, opts :: acl_opts) :: ExAws.Operation.S3.t

Update or create a bucket bucket access control

put_bucket_cors(bucket, cors_rules)
put_bucket_cors(bucket :: binary, cors_config :: %{}) :: ExAws.Operation.S3.t

Update or create a bucket CORS policy

put_bucket_lifecycle(bucket, livecycle_config)
put_bucket_lifecycle(bucket :: binary, lifecycle_config :: %{}) :: no_return

Update or create a bucket lifecycle configuration

put_bucket_logging(bucket, logging_config)
put_bucket_logging(bucket :: binary, logging_config :: %{}) :: no_return

Update or create a bucket logging configuration

put_bucket_notification(bucket, notification_config)
put_bucket_notification(bucket :: binary, notification_config :: %{}) :: no_return

Update or create a bucket notification configuration

put_bucket_policy(bucket, policy)
put_bucket_policy(bucket :: binary, policy :: %{}) :: ExAws.Operation.S3.t

Update or create a bucket policy configuration

put_bucket_replication(bucket, replication_config)
put_bucket_replication(bucket :: binary, replication_config :: %{}) :: no_return

Update or create a bucket replication configuration

put_bucket_request_payment(bucket, payer)
put_bucket_request_payment(bucket :: binary, payer :: :requester | :bucket_owner) :: no_return

Update or create a bucket requestPayment configuration

put_bucket_tagging(bucket, tags)
put_bucket_tagging(bucket :: binary, tags :: %{}) :: no_return

Update or create a bucket tagging configuration

put_bucket_versioning(bucket, version_config)
put_bucket_versioning(bucket :: binary, version_config :: binary) :: no_return

Update or create a bucket versioning configuration

put_bucket_website(bucket, website_config)
put_bucket_website(bucket :: binary, website_config :: binary) :: no_return

Update or create a bucket website configuration

put_object(bucket, object, body, opts \\ [])
put_object(bucket :: binary, object :: binary, body :: binary, opts :: put_object_opts) :: ExAws.Operation.S3.t

Create an object within a bucket

put_object_acl(bucket, object, acl)
put_object_acl(bucket :: binary, object :: binary, acl :: acl_opts) :: ExAws.Operation.S3.t

Create or update an object’s access control FIXME

put_object_copy(dest_bucket, dest_object, src_bucket, src_object, opts \\ [])
put_object_copy(dest_bucket :: binary, dest_object :: binary, src_bucket :: binary, src_object :: binary, opts :: pub_object_copy_opts) :: ExAws.Operation.S3.t

Copy an object

upload(source, bucket, path, opts \\ [])
upload(source :: Enumerable.t, bucket :: String.t, path :: String.t, opts :: upload_opts) :: ExAws.S3.Upload.t

Multipart upload to S3.

Handles initialization, uploading parts concurrently, and multipart upload completion.

Uploading a stream

Streams that emit binaries may be uploaded directly to S3. Each binary will be uploaded as a chunk, so it must be at least 5 megabytes in size. The S3.Upload.stream_file helper takes care of reading the file in 5 megabyte chunks.

"path/to/big/file"
|> S3.Upload.stream_file
|> S3.upload("my-bucket", "path/on/s3")
|> ExAws.request! #=> :done

Options

These options are specific to this function

  • :max_concurrency — The number of concurrent processes reading from this stream. Only applies when uploading a stream.

All other options (ex. :content_type) are passed through to ExAws.S3.initiate_multipart_upload/3.

upload_part(bucket, object, upload_id, part_number, body, opts \\ [])
upload_part(bucket :: binary, object :: binary, upload_id :: binary, part_number :: pos_integer, body :: binary, opts :: [encryption_opts | {:expect, binary}]) :: ExAws.Operation.S3.t

Upload a part for a multipart upload

upload_part_copy(dest_bucket, dest_object, src_bucket, src_object, opts \\ [])
upload_part_copy(dest_bucket :: binary, dest_object :: binary, src_bucket :: binary, src_object :: binary, opts :: upload_part_copy_opts) :: ExAws.Operation.S3.t

Upload a part for a multipart copy