ExAws v1.1.4 ExAws.S3 View Source
Operations on AWS S3
Basic Operations
The vast majority of operations here represent a single operation on S3.
Examples
S3.list_objects |> ExAws.request! #=> {:ok, %{body: [list, of, objects]}}
S3.list_objects |> ExAws.stream! |> Enum.to_list #=> [list, of, objects]
S3.put_object("my-bucket", "path/to/bucket", contents) |> ExAws.request!Higher Level Operations
There are also some operations which operate at a higher level to make it easier to download and upload very large files.
Multipart uploads
"path/to/big/file"
|> S3.Upload.stream_file
|> S3.upload("my-bucket", "path/on/s3")
|> ExAws.request! #=> {:ok, :done}Download large file to disk
S3.download_file("my-bucket", "path/on/s3", "path/to/dest/file")
|> ExAws.request! #=> {:on, :done}More high level functionality
Task.async_stream makes some high level flows so easy you don’t need explicit ExAws support.
For example, here is how to concurrently upload many files.
upload_file = fn {src_path, dest_path} ->
S3.put_object("my_bucket", dest_path, File.read!(src_path))
|> ExAws.request!
end
paths = %{"path/to/src0" => "path/to/dest0", "path/to/src1" => "path/to/dest1"}
paths
|> Task.async_stream(upload_file, max_concurrency: 10)
|> Stream.runLink to this section Summary
Functions
Abort a multipart upload
Complete a multipart upload
Delete all listed objects
Delete a bucket
Delete a bucket cors
Delete a bucket lifecycle
Delete a bucket policy
Delete a bucket replication
Delete a bucket tagging
Delete a bucket website
Delete multiple objects within a bucket
Delete object object in bucket
Download an S3 Object to a file
Get bucket acl
Get bucket cors
Get bucket lifecycle
Get bucket location
Get bucket logging
Get bucket notification
Get bucket object versions
Get bucket policy
Get bucket replication
Get bucket payment configuration
Get bucket tagging
Get bucket versioning
Get bucket website
Get an object from a bucket
Get an object’s access control policy
Get a torrent for a bucket
Determine if a bucket exists
Determine of an object exists
Initiate a multipart upload
List buckets
List multipart uploads for a bucket
List objects in bucket
List the parts of a multipart upload
Determine the CORS configuration for an object
Restore an object to a particular version
Generates a pre-signed URL for this object
Creates a bucket in the specified region
Update or create a bucket bucket access control
Update or create a bucket CORS policy
Update or create a bucket lifecycle configuration
Update or create a bucket logging configuration
Update or create a bucket notification configuration
Update or create a bucket policy configuration
Update or create a bucket replication configuration
Update or create a bucket requestPayment configuration
Update or create a bucket tagging configuration
Update or create a bucket versioning configuration
Update or create a bucket website configuration
Create an object within a bucket
Create or update an object’s access control FIXME
Multipart upload to S3
Upload a part for a multipart upload
Upload a part for a multipart copy
Link to this section Types
amz_meta_opts() :: [{atom, binary} | {binary, binary}, ...]
canned_acl :: :private | :public_read | :public_read_write | :authenticated_read | :bucket_owner_read | :bucket_owner_full_control
customer_encryption_opts() :: [customer_algorithm: binary, customer_key: binary, customer_key_md5: binary]
download_file_opts() :: [max_concurrency: pos_integer, chunk_size: pos_integer, timeout: 60000]
encryption_opts :: binary | [{:aws_kms_key_id, binary}] | customer_encryption_opts
get_object_opts() :: [{:response, get_object_response_opts} | head_object_opts]
get_object_response_opts() :: [content_language: binary, expires: binary, cach_control: binary, content_disposition: binary, content_encoding: binary]
head_object_opts() :: [encryption: customer_encryption_opts, range: binary, if_modified_since: binary, if_unmodified_since: binary, if_match: binary, if_none_match: binary]
initiate_multipart_upload_opts() :: [{:cache_control, binary} | {:content_disposition, binary} | {:content_encoding, binary} | {:content_type, binary} | {:expires, binary} | {:storage_class, :standard | :redunced_redundancy} | {:website_redirect_location, binary} | {:encryption, encryption_opts} | acl_opts]
list_objects_opts() :: [delimiter: binary, marker: binary, prefix: binary, encoding_type: binary, max_keys: 0..1000]
presigned_url_opts() :: [expires_in: integer, virtual_host: boolean, query_params: [{binary, binary}]]
pub_object_copy_opts() :: [{:metadata_directive, :COPY | :REPLACE} | {:copy_source_if_modified_since, binary} | {:copy_source_if_unmodified_since, binary} | {:copy_source_if_match, binary} | {:copy_source_if_none_match, binary} | {:website_redirect_location, binary} | {:destination_encryption, encryption_opts} | {:source_encryption, customer_encryption_opts} | {:cache_control, binary} | {:content_disposition, binary} | {:content_encoding, binary} | {:content_length, binary} | {:content_type, binary} | {:expect, binary} | {:expires, binary} | {:storage_class, :standard | :redunced_redundancy} | {:website_redirect_location, binary} | {:meta, amz_meta_opts} | acl_opts]
put_object_opts() :: [{:cache_control, binary} | {:content_disposition, binary} | {:content_encoding, binary} | {:content_length, binary} | {:content_type, binary} | {:expect, binary} | {:expires, binary} | {:storage_class, :standard | :redunced_redundancy} | {:website_redirect_location, binary} | {:encryption, encryption_opts} | {:meta, amz_meta_opts} | acl_opts]
upload_opts() :: [{:max_concurrency, pos_integer} | initiate_multipart_upload_opts]
upload_part_copy_opts() :: [copy_source_range: Range.t, copy_source_if_modified_since: binary, copy_source_if_unmodified_since: binary, copy_source_if_match: binary, copy_source_if_none_match: binary, destination_encryption: encryption_opts, source_encryption: customer_encryption_opts]
Link to this section Functions
abort_multipart_upload(bucket :: binary, object :: binary, upload_id :: binary) :: ExAws.Operation.S3.t
Abort a multipart upload
complete_multipart_upload(bucket :: binary, object :: binary, upload_id :: binary, parts :: [{binary | pos_integer, binary}, ...]) :: ExAws.Operation.S3.t
Complete a multipart upload
delete_all_objects(bucket :: binary, objects :: [binary | {binary, binary}, ...], opts :: [{:quiet, true}]) :: ExAws.Operation.S3DeleteAllObjects.t
Delete all listed objects.
When performed, this function will continue making delete_multiple_objects
requests deleting 1000 objects at a time until all are deleted.
Can be streamed.
delete_bucket(bucket :: binary) :: ExAws.Operation.S3.t
Delete a bucket
delete_bucket_cors(bucket :: binary) :: ExAws.Operation.S3.t
Delete a bucket cors
delete_bucket_lifecycle(bucket :: binary) :: ExAws.Operation.S3.t
Delete a bucket lifecycle
delete_bucket_policy(bucket :: binary) :: ExAws.Operation.S3.t
Delete a bucket policy
delete_bucket_replication(bucket :: binary) :: ExAws.Operation.S3.t
Delete a bucket replication
delete_bucket_tagging(bucket :: binary) :: ExAws.Operation.S3.t
Delete a bucket tagging
delete_bucket_website(bucket :: binary) :: ExAws.Operation.S3.t
Delete a bucket website
delete_multiple_objects(bucket :: binary, objects :: [binary | {binary, binary}, ...], opts :: [{:quiet, true}]) :: ExAws.Operation.S3.t
Delete multiple objects within a bucket
Limited to 1000 objects.
Delete object object in bucket
download_file(bucket :: binary, path :: binary, dest :: binary, opts :: download_file_opts) :: ExAws.S3.Download.t
Download an S3 Object to a file.
This operation download multiple parts of an S3 object concurrently, allowing you to maximize throughput.
Defaults to a concurrency of 8, chunk size of 1MB, and a timeout of 1 minute.
get_bucket_acl(bucket :: binary) :: ExAws.Operation.S3.t
Get bucket acl
get_bucket_cors(bucket :: binary) :: ExAws.Operation.S3.t
Get bucket cors
get_bucket_lifecycle(bucket :: binary) :: ExAws.Operation.S3.t
Get bucket lifecycle
get_bucket_location(bucket :: binary) :: ExAws.Operation.S3.t
Get bucket location
get_bucket_logging(bucket :: binary) :: ExAws.Operation.S3.t
Get bucket logging
get_bucket_notification(bucket :: binary) :: ExAws.Operation.S3.t
Get bucket notification
get_bucket_object_versions(bucket :: binary, opts :: Keyword.t) :: ExAws.Operation.S3.t
Get bucket object versions
get_bucket_policy(bucket :: binary) :: ExAws.Operation.S3.t
Get bucket policy
get_bucket_replication(bucket :: binary) :: ExAws.Operation.S3.t
Get bucket replication
get_bucket_request_payment(bucket :: binary) :: ExAws.Operation.S3.t
Get bucket payment configuration
get_bucket_tagging(bucket :: binary) :: ExAws.Operation.S3.t
Get bucket tagging
get_bucket_versioning(bucket :: binary) :: ExAws.Operation.S3.t
Get bucket versioning
get_bucket_website(bucket :: binary) :: ExAws.Operation.S3.t
Get bucket website
get_object(bucket :: binary, object :: binary, opts :: get_object_opts) :: ExAws.Operation.S3.t
Get an object from a bucket
get_object_acl(bucket :: binary, object :: binary, opts :: Keyword.t) :: ExAws.Operation.S3.t
Get an object’s access control policy
get_object_torrent(bucket :: binary, object :: binary) :: ExAws.Operation.S3.t
Get a torrent for a bucket
head_bucket(bucket :: binary) :: ExAws.Operation.S3.t
Determine if a bucket exists
head_object(bucket :: binary, object :: binary, opts :: head_object_opts) :: ExAws.Operation.S3.t
Determine of an object exists
initiate_multipart_upload(bucket :: binary, object :: binary, opts :: initiate_multipart_upload_opts) :: ExAws.Operation.S3.t
Initiate a multipart upload
list_buckets(opts :: Keyword.t) :: ExAws.Operation.S3.t
List buckets
list_multipart_uploads(bucket :: binary, opts :: Keyword.t) :: ExAws.Operation.S3.t
List multipart uploads for a bucket
list_objects(bucket :: binary, opts :: list_objects_opts) :: ExAws.Operation.S3.t
List objects in bucket
Can be streamed.
Examples
S3.list_objects("my-bucket") |> ExAws.request
S3.list_objects("my-bucket") |> ExAws.stream!
S3.list_objects("my-bucket", delimiter: "/", prefix: "backup") |> ExAws.stream!
S3.list_objects("my-bucket", prefix: "some/inner/location/path") |> ExAws.stream!
S3.list_objects("my-bucket", max_keys: 5, encoding_type: "url") |> ExAws.stream!list_parts(bucket :: binary, object :: binary, upload_id :: binary, opts :: Keyword.t) :: ExAws.Operation.S3.t
List the parts of a multipart upload
options_object(bucket :: binary, object :: binary, origin :: binary, request_method :: atom, request_headers :: [binary]) :: ExAws.Operation.S3.t
Determine the CORS configuration for an object
post_object_restore(bucket :: binary, object :: binary, number_of_days :: pos_integer, opts :: [{:version_id, binary}]) :: ExAws.Operation.S3.t
Restore an object to a particular version
presigned_url(config :: map, http_method :: atom, bucket :: binary, object :: binary, opts :: presigned_url_opts) :: {:ok, binary} | {:error, binary}
Generates a pre-signed URL for this object.
When option param :virtual_host is true, the {#bucket} name will be used as
the hostname. This will cause the returned URL to be ‘http’ and not ‘https’.
Additional (signed) query parameters can be added to the url by setting option param
:query_params to a list of {"key", "value"} pairs. Useful if you are uploading parts of
a multipart upload directly from the browser.
Creates a bucket in the specified region
put_bucket_acl(bucket :: binary, opts :: [acl_opts]) :: ExAws.Operation.S3.t
Update or create a bucket bucket access control
put_bucket_cors(bucket :: binary, cors_config :: map) :: ExAws.Operation.S3.t
Update or create a bucket CORS policy
put_bucket_lifecycle(bucket :: binary, lifecycle_config :: map) :: no_return
Update or create a bucket lifecycle configuration
put_bucket_logging(bucket :: binary, logging_config :: map) :: no_return
Update or create a bucket logging configuration
put_bucket_notification(bucket :: binary, notification_config :: map) :: no_return
Update or create a bucket notification configuration
put_bucket_policy(bucket :: binary, policy :: map) :: ExAws.Operation.S3.t
Update or create a bucket policy configuration
put_bucket_replication(bucket :: binary, replication_config :: map) :: no_return
Update or create a bucket replication configuration
put_bucket_request_payment(bucket :: binary, payer :: :requester | :bucket_owner) :: no_return
Update or create a bucket requestPayment configuration
put_bucket_tagging(bucket :: binary, tags :: map) :: no_return
Update or create a bucket tagging configuration
put_bucket_versioning(bucket :: binary, version_config :: binary) :: no_return
Update or create a bucket versioning configuration
put_bucket_website(bucket :: binary, website_config :: binary) :: no_return
Update or create a bucket website configuration
put_object(bucket :: binary, object :: binary, body :: binary, opts :: put_object_opts) :: ExAws.Operation.S3.t
Create an object within a bucket
put_object_acl(bucket :: binary, object :: binary, acl :: [acl_opts]) :: ExAws.Operation.S3.t
Create or update an object’s access control FIXME
put_object_copy(dest_bucket :: binary, dest_object :: binary, src_bucket :: binary, src_object :: binary, opts :: pub_object_copy_opts) :: ExAws.Operation.S3.t
Copy an object
upload(source :: Enumerable.t, bucket :: String.t, path :: String.t, opts :: upload_opts) :: ExAws.S3.Upload.t
Multipart upload to S3.
Handles initialization, uploading parts concurrently, and multipart upload completion.
Uploading a stream
Streams that emit binaries may be uploaded directly to S3. Each binary will be uploaded
as a chunk, so it must be at least 5 megabytes in size. The S3.Upload.stream_file
helper takes care of reading the file in 5 megabyte chunks.
"path/to/big/file"
|> S3.Upload.stream_file
|> S3.upload("my-bucket", "path/on/s3")
|> ExAws.request! #=> :doneOptions
These options are specific to this function
:max_concurrency— The number of concurrent processes reading from this stream. Only applies when uploading a stream.
All other options (ex. :content_type) are passed through to
ExAws.S3.initiate_multipart_upload/3.
upload_part(bucket :: binary, object :: binary, upload_id :: binary, part_number :: pos_integer, body :: binary, opts :: [encryption_opts | {:expect, binary}]) :: ExAws.Operation.S3.t
Upload a part for a multipart upload
upload_part_copy(dest_bucket :: binary, dest_object :: binary, src_bucket :: binary, src_object :: binary, opts :: upload_part_copy_opts) :: ExAws.Operation.S3.t
Upload a part for a multipart copy