AWS.EMRcontainers (aws-elixir v0.8.0) View Source

Amazon EMR on EKS provides a deployment option for Amazon EMR that allows you to run open-source big data frameworks on Amazon Elastic Kubernetes Service (Amazon EKS).

With this deployment option, you can focus on running analytics workloads while Amazon EMR on EKS builds, configures, and manages containers for open-source applications. For more information about Amazon EMR on EKS concepts and tasks, see What is Amazon EMR on EKS.

Amazon EMR containers is the API name for Amazon EMR on EKS. The emr-containers prefix is used in the following scenarios:

  • It is the prefix in the CLI commands for Amazon EMR on EKS. For example, aws emr-containers start-job-run.

  • It is the prefix before IAM policy actions for Amazon EMR on EKS. For example, "Action": [ "emr-containers:StartJobRun"]. For more information, see Policy actions for Amazon EMR on EKS.

  • It is the prefix used in Amazon EMR on EKS service endpoints. For example, emr-containers.us-east-2.amazonaws.com. For more information, see Amazon EMR on EKS Service Endpoints.

Link to this section Summary

Functions

Displays detailed information about a job run.

Displays detailed information about a managed endpoint.

Displays detailed information about a specified virtual cluster.

Lists the tags assigned to the resources.

Link to this section Functions

Link to this function

cancel_job_run(client, id, virtual_cluster_id, input, options \\ [])

View Source

Cancels a job run.

A job run is a unit of work, such as a Spark jar, PySpark script, or SparkSQL query, that you submit to Amazon EMR on EKS.

Link to this function

create_managed_endpoint(client, virtual_cluster_id, input, options \\ [])

View Source

Creates a managed endpoint.

A managed endpoint is a gateway that connects EMR Studio to Amazon EMR on EKS so that EMR Studio can communicate with your virtual cluster.

Link to this function

create_virtual_cluster(client, input, options \\ [])

View Source

Creates a virtual cluster.

Virtual cluster is a managed entity on Amazon EMR on EKS. You can create, describe, list and delete virtual clusters. They do not consume any additional resource in your system. A single virtual cluster maps to a single Kubernetes namespace. Given this relationship, you can model virtual clusters the same way you model Kubernetes namespaces to meet your requirements.

Link to this function

delete_managed_endpoint(client, id, virtual_cluster_id, input, options \\ [])

View Source

Deletes a managed endpoint.

A managed endpoint is a gateway that connects EMR Studio to Amazon EMR on EKS so that EMR Studio can communicate with your virtual cluster.

Link to this function

delete_virtual_cluster(client, id, input, options \\ [])

View Source

Deletes a virtual cluster.

Virtual cluster is a managed entity on Amazon EMR on EKS. You can create, describe, list and delete virtual clusters. They do not consume any additional resource in your system. A single virtual cluster maps to a single Kubernetes namespace. Given this relationship, you can model virtual clusters the same way you model Kubernetes namespaces to meet your requirements.

Link to this function

describe_job_run(client, id, virtual_cluster_id, options \\ [])

View Source

Displays detailed information about a job run.

A job run is a unit of work, such as a Spark jar, PySpark script, or SparkSQL query, that you submit to Amazon EMR on EKS.

Link to this function

describe_managed_endpoint(client, id, virtual_cluster_id, options \\ [])

View Source

Displays detailed information about a managed endpoint.

A managed endpoint is a gateway that connects EMR Studio to Amazon EMR on EKS so that EMR Studio can communicate with your virtual cluster.

Link to this function

describe_virtual_cluster(client, id, options \\ [])

View Source

Displays detailed information about a specified virtual cluster.

Virtual cluster is a managed entity on Amazon EMR on EKS. You can create, describe, list and delete virtual clusters. They do not consume any additional resource in your system. A single virtual cluster maps to a single Kubernetes namespace. Given this relationship, you can model virtual clusters the same way you model Kubernetes namespaces to meet your requirements.

Link to this function

list_job_runs(client, virtual_cluster_id, created_after \\ nil, created_before \\ nil, max_results \\ nil, name \\ nil, next_token \\ nil, states \\ nil, options \\ [])

View Source

Lists job runs based on a set of parameters.

A job run is a unit of work, such as a Spark jar, PySpark script, or SparkSQL query, that you submit to Amazon EMR on EKS.

Link to this function

list_managed_endpoints(client, virtual_cluster_id, created_after \\ nil, created_before \\ nil, max_results \\ nil, next_token \\ nil, states \\ nil, types \\ nil, options \\ [])

View Source

Lists managed endpoints based on a set of parameters.

A managed endpoint is a gateway that connects EMR Studio to Amazon EMR on EKS so that EMR Studio can communicate with your virtual cluster.

Link to this function

list_tags_for_resource(client, resource_arn, options \\ [])

View Source

Lists the tags assigned to the resources.

Link to this function

list_virtual_clusters(client, container_provider_id \\ nil, container_provider_type \\ nil, created_after \\ nil, created_before \\ nil, max_results \\ nil, next_token \\ nil, states \\ nil, options \\ [])

View Source

Lists information about the specified virtual cluster.

Virtual cluster is a managed entity on Amazon EMR on EKS. You can create, describe, list and delete virtual clusters. They do not consume any additional resource in your system. A single virtual cluster maps to a single Kubernetes namespace. Given this relationship, you can model virtual clusters the same way you model Kubernetes namespaces to meet your requirements.

Link to this function

start_job_run(client, virtual_cluster_id, input, options \\ [])

View Source

Starts a job run.

A job run is a unit of work, such as a Spark jar, PySpark script, or SparkSQL query, that you submit to Amazon EMR on EKS.

Link to this function

tag_resource(client, resource_arn, input, options \\ [])

View Source

Assigns tags to resources.

A tag is a label that you assign to an AWS resource. Each tag consists of a key and an optional value, both of which you define. Tags enable you to categorize your AWS resources by attributes such as purpose, owner, or environment. When you have many resources of the same type, you can quickly identify a specific resource based on the tags you've assigned to it. For example, you can define a set of tags for your Amazon EMR on EKS clusters to help you track each cluster's owner and stack level. We recommend that you devise a consistent set of tag keys for each resource type. You can then search and filter the resources based on the tags that you add.

Link to this function

untag_resource(client, resource_arn, input, options \\ [])

View Source

Removes tags from resources.