GoogleApi.Dataproc.V1.Model.BasicYarnAutoscalingConfig (google_api_dataproc v0.48.0) View Source

Basic autoscaling configurations for YARN.

Attributes

  • gracefulDecommissionTimeout (type: String.t, default: nil) - Required. Timeout for YARN graceful decommissioning of Node Managers. Specifies the duration to wait for jobs to complete before forcefully removing workers (and potentially interrupting jobs). Only applicable to downscaling operations.Bounds: 0s, 1d.
  • scaleDownFactor (type: float(), default: nil) - Required. Fraction of average YARN pending memory in the last cooldown period for which to remove workers. A scale-down factor of 1 will result in scaling down so that there is no available memory remaining after the update (more aggressive scaling). A scale-down factor of 0 disables removing workers, which can be beneficial for autoscaling a single job. See How autoscaling works (https://cloud.google.com/dataproc/docs/concepts/configuring-clusters/autoscaling#how_autoscaling_works) for more information.Bounds: 0.0, 1.0.
  • scaleDownMinWorkerFraction (type: float(), default: nil) - Optional. Minimum scale-down threshold as a fraction of total cluster size before scaling occurs. For example, in a 20-worker cluster, a threshold of 0.1 means the autoscaler must recommend at least a 2 worker scale-down for the cluster to scale. A threshold of 0 means the autoscaler will scale down on any recommended change.Bounds: 0.0, 1.0. Default: 0.0.
  • scaleUpFactor (type: float(), default: nil) - Required. Fraction of average YARN pending memory in the last cooldown period for which to add workers. A scale-up factor of 1.0 will result in scaling up so that there is no pending memory remaining after the update (more aggressive scaling). A scale-up factor closer to 0 will result in a smaller magnitude of scaling up (less aggressive scaling). See How autoscaling works (https://cloud.google.com/dataproc/docs/concepts/configuring-clusters/autoscaling#how_autoscaling_works) for more information.Bounds: 0.0, 1.0.
  • scaleUpMinWorkerFraction (type: float(), default: nil) - Optional. Minimum scale-up threshold as a fraction of total cluster size before scaling occurs. For example, in a 20-worker cluster, a threshold of 0.1 means the autoscaler must recommend at least a 2-worker scale-up for the cluster to scale. A threshold of 0 means the autoscaler will scale up on any recommended change.Bounds: 0.0, 1.0. Default: 0.0.

Link to this section Summary

Functions

Unwrap a decoded JSON object into its complex fields.

Link to this section Types

Specs

t() :: %GoogleApi.Dataproc.V1.Model.BasicYarnAutoscalingConfig{
  gracefulDecommissionTimeout: String.t() | nil,
  scaleDownFactor: float() | nil,
  scaleDownMinWorkerFraction: float() | nil,
  scaleUpFactor: float() | nil,
  scaleUpMinWorkerFraction: float() | nil
}

Link to this section Functions

Specs

decode(struct(), keyword()) :: struct()

Unwrap a decoded JSON object into its complex fields.