View Source API Reference google_api_dataplex v0.11.0

Modules

API client metadata for GoogleApi.Dataplex.V1.

API calls for all endpoints tagged Projects.

Handle Tesla connections for GoogleApi.Dataplex.V1.

A generic empty message that you can re-use to avoid defining duplicated empty messages in your APIs. A typical example is to use it as the request or the response type of an API method. For instance: service Foo { rpc Bar(google.protobuf.Empty) returns (google.protobuf.Empty); }

Action represents an issue requiring administrator action for resolution.

Failed to apply security policy to the managed resource(s) under a lake, zone or an asset. For a lake or zone resource, one or more underlying assets has a failure applying security policy to the associated managed resource.

Action details for incompatible schemas detected by discovery.

Action details for invalid or unsupported data files detected by discovery.

Action details for invalid or unsupported partitions detected by discovery.

Action details for absence of data detected by discovery.

Action details for resource references in assets that cannot be located.

Action details for unauthorized resource issues raised to indicate that the service account associated with the lake instance is not authorized to access or manage the resource associated with an asset.

An aspect is a single piece of metadata describing an entry.

AspectSource contains source system related information for the aspect.

Aspect Type is a template for creating Aspects, and represents the JSON-schema for a given Entry, e.g., BigQuery Table Schema.

An asset represents a cloud resource that is being managed within a lake as a member of a zone.

Settings to manage the metadata discovery and publishing for an asset.

Describe CSV and similar semi-structured data formats.

The aggregated data statistics for the asset reported by discovery.

Identifies the cloud resource that is referenced by this asset.

Status of the resource referenced by an asset.

Security policy status of the asset. Data security policy, i.e., readers, writers & owners, should be specified in the lake/zone/asset IAM policy.

Aggregated status of the underlying assets of a lake or zone.

Content represents a user-visible notebook or a sql script

Configuration for the Sql Script content.

DataAccessSpec holds the access control configuration to be enforced on data stored within resources (eg: rows, columns in BigQuery Tables). When associated with data, the data is only accessible to principals explicitly granted access through the DataAccessSpec. Principals with access to the containing resource are not implicitly granted access.

Denotes one dataAttribute in a dataTaxonomy, for example, PII. DataAttribute resources can be defined in a hierarchy. A single dataAttribute resource can contain specs of multiple types PII - ResourceAccessSpec : - readers :foo@bar.com - DataAccessSpec : - readers :bar@foo.com

DataAttributeBinding represents binding of attributes to resources. Eg: Bind 'CustomerInfo' entity with 'PII' attribute.

Represents a subresource of the given resource, and associated bindings with it. Currently supported subresources are column and partition schema fields within a table.

DataProfileResult defines the output of DataProfileScan. Each field of the table will have field type specific profile result.

The result of post scan actions of DataProfileScan job.

Contains name, type, mode and field type specific profile information.

The configuration of post scan actions of DataProfileScan job.

The specification for fields to include or exclude in data profile scan.

DataQualityColumnResult provides a more detailed, per-column view of the results.

A dimension captures data quality intent about a defined subset of the rules specified.

DataQualityDimensionResult provides a more detailed, per-dimension view of the results.

The result of post scan actions of DataQualityScan job.

A rule captures data quality intent about a data source.

Evaluates whether each column value lies between a specified range.

Evaluates whether each column value matches a specified regex.

DataQualityRuleResult provides a more detailed, per-rule view of the results.

Evaluates whether each row passes the specified condition.The SQL expression needs to use BigQuery standard SQL syntax and should produce a boolean value per row as the result.Example: col1 >= 0 AND col2 < 10

Evaluates whether each column value is contained by a specified set.

Queries for rows returned by the provided SQL statement. If any rows are are returned, this rule fails.The SQL statement needs to use BigQuery standard SQL syntax, and must not contain any semicolons.${data()} can be used to reference the rows being evaluated, i.e. the table after all additional filters (row filters, incremental data filters, sampling) are applied.Example: SELECT * FROM ${data()} WHERE price < 0

Evaluates whether the column aggregate statistic lies between a specified range.

Evaluates whether the provided expression is true.The SQL expression needs to use BigQuery standard SQL syntax and should produce a scalar boolean result.Example: MIN(col1) >= 0

Information about the result of a data quality rule for data quality scan. The monitored resource is 'DataScan'.

The configuration of post scan actions of DataQualityScan.

This trigger is triggered whenever a scan job run ends, regardless of the result.

This trigger is triggered when the scan job itself fails, regardless of the result.

The individuals or groups who are designated to receive notifications upon triggers.

This trigger is triggered when the DQ score in the job result is less than a specified input score.

Represents a user-visible job which provides the insights for the related data source.For example: Data Quality: generates queries based on the rules and runs against the data to get data quality check results. Data Profile: analyzes the data in table(s) and generates insights about the structure, content and relationships (such as null percent, cardinality, min/max/mean, etc).

These messages contain information about the execution of a datascan. The monitored resource is 'DataScan' Next ID: 13

A DataScanJob represents an instance of DataScan execution.

DataTaxonomy represents a set of hierarchical DataAttributes resources, grouped with a common theme Eg: 'SensitiveDataTaxonomy' can have attributes to manage PII data. It is defined at project level.

The payload associated with Discovery data processing.

Represents tables and fileset metadata contained within a zone.

Provides compatibility information for various metadata stores.

Provides compatibility information for a specific metadata store.

An entry is a representation of a data asset which can be described by various metadata.

An Entry Group represents a logical grouping of one or more Entries.

EntrySource contains source system related information for the entry.

Ancestor contains information about individual items in the hierarchy of an Entry.

Entry Type is a template for creating Entries.

Attributes

  • type (type: String.t, default: nil) - Required aspect type for the entry type.

Environment represents a user-visible compute infrastructure for analytics within a lake.

URI Endpoints to access sessions associated with the Environment.

Configuration for the underlying infrastructure used to run workloads.

Compute resources associated with the analyze interactive workloads.

Configuration for sessions created for this environment.

Status of sessions created for this environment.

Payload associated with Governance related log events.

Information about Entity resource that the log event is associated with.

A job represents an instance of a task.

The payload associated with Job logs that contains events describing jobs that have run within a Lake.

A lake is a centralized repository for managing enterprise data across the organization distributed across many cloud projects, and stored in a variety of storage services such as Google Cloud Storage and BigQuery. The resources attached to a lake are referred to as managed resources. Data within these managed resources can be structured or unstructured. A lake provides data admins with tools to organize, secure and manage their data at scale, and provides data scientists and data engineers an integrated experience to easily search, discover, analyze and transform data and associated metadata.

Settings to manage association of Dataproc Metastore with a lake.

Status of Lake and Dataproc Metastore service instance association.

Attributes

  • entries (type: list(GoogleApi.Dataplex.V1.Model.GoogleCloudDataplexV1Entry.t), default: nil) - The list of entries.
  • nextPageToken (type: String.t, default: nil) - Pagination token.

Represents the metadata of a long-running operation.

Represents partition metadata contained within entity instances.

ResourceAccessSpec holds the access control configuration to be enforced on the resources, for example, Cloud Storage bucket, BigQuery dataset, BigQuery table.

Attributes

  • args (type: map(), default: nil) - Optional. Execution spec arguments. If the map is left empty, the task will run with existing execution spec args from task definition. If the map contains an entry with a new key, the same will be added to existing set of args. If the map contains an entry with an existing arg key in task definition, the task will run with new arg value for that entry. Clearing an existing arg will require arg value to be explicitly set to a hyphen "-". The arg value cannot be empty.
  • labels (type: map(), default: nil) - Optional. User-defined labels for the task. If the map is left empty, the task will run with existing labels from task definition. If the map contains an entry with a new key, the same will be added to existing set of labels. If the map contains an entry with an existing label key in task definition, the task will run with new label value for that entry. Clearing an existing label will require label value to be explicitly set to a hyphen "-". The label value cannot be empty.

Attributes

  • job (type: GoogleApi.Dataplex.V1.Model.GoogleCloudDataplexV1Job.t, default: nil) - Jobs created by RunTask API.

The data scanned during processing (e.g. in incremental DataScan)

A data range denoted by a pair of start/end values of a field.

Schema information describing the structure and layout of the data.

Represents a key field within the entity's partition structure. You could have up to 20 partition fields, but only the first 10 partitions have the filtering ability due to performance consideration. Note: Partition fields are immutable.

Represents a column field within a table schema.

Attributes

  • nextPageToken (type: String.t, default: nil) - Pagination token.
  • results (type: list(GoogleApi.Dataplex.V1.Model.GoogleCloudDataplexV1SearchEntriesResult.t), default: nil) - The results matching the search query.
  • totalSize (type: integer(), default: nil) - The estimated total number of matching entries. Not guaranteed to be accurate.
  • unreachable (type: list(String.t), default: nil) - Unreachable locations. Search results don't include data from those locations.

A single result of a SearchEntries request.

Snippets for the entry, contains HTML-style highlighting for matched tokens, will be used in UI.

Represents an active analyze session running for a user.

These messages contain information about sessions within an environment. The monitored resource is 'Environment'.

Describes the access mechanism of the data within its storage location.

Describes the format of the data within its storage location.

Describes CSV and similar semi-structured data formats.

A task represents a user-visible job.

Execution related settings, like retry and service_account.

Configuration for the underlying infrastructure used to run workloads.

Container Image Runtime Configuration used with Batch execution.

User-specified config for running a Spark task.

DataScan scheduling and trigger settings.

The scan is scheduled to run periodically.

A zone represents a logical group of related assets within a lake. A zone can be used to map to organizational structure or represent stages of data readiness from raw to curated. It provides managing behavior that is shared or inherited by all contained assets.

Settings to manage the metadata discovery and publishing in a zone.

Describe CSV and similar semi-structured data formats.

Settings for resources attached as assets within a zone.

The response message for Locations.ListLocations.

A resource that represents a Google Cloud location.

Specifies the audit configuration for a service. The configuration determines which permission types are logged, and what identities, if any, are exempted from logging. An AuditConfig must have one or more AuditLogConfigs.If there are AuditConfigs for both allServices and a specific service, the union of the two AuditConfigs is used for that service: the log_types specified in each AuditConfig are enabled, and the exempted_members in each AuditLogConfig are exempted.Example Policy with multiple AuditConfigs: { "audit_configs": [ { "service": "allServices", "audit_log_configs": [ { "log_type": "DATA_READ", "exempted_members": [ "user:jose@example.com" ] }, { "log_type": "DATA_WRITE" }, { "log_type": "ADMIN_READ" } ] }, { "service": "sampleservice.googleapis.com", "audit_log_configs": [ { "log_type": "DATA_READ" }, { "log_type": "DATA_WRITE", "exempted_members": [ "user:aliya@example.com" ] } ] } ] } For sampleservice, this policy enables DATA_READ, DATA_WRITE and ADMIN_READ logging. It also exempts jose@example.com from DATA_READ logging, and aliya@example.com from DATA_WRITE logging.

Provides the configuration for logging a type of permissions. Example: { "audit_log_configs": [ { "log_type": "DATA_READ", "exempted_members": [ "user:jose@example.com" ] }, { "log_type": "DATA_WRITE" } ] } This enables 'DATA_READ' and 'DATA_WRITE' logging, while exempting jose@example.com from DATA_READ logging.

Associates members, or principals, with a role.

An Identity and Access Management (IAM) policy, which specifies access controls for Google Cloud resources.A Policy is a collection of bindings. A binding binds one or more members, or principals, to a single role. Principals can be user accounts, service accounts, Google groups, and domains (such as G Suite). A role is a named list of permissions; each role can be an IAM predefined role or a user-created custom role.For some types of Google Cloud resources, a binding can also specify a condition, which is a logical expression that allows access to a resource only if the expression evaluates to true. A condition can add constraints based on attributes of the request, the resource, or both. To learn which resources support conditions in their IAM policies, see the IAM documentation (https://cloud.google.com/iam/help/conditions/resource-policies).JSON example: { "bindings": [ { "role": "roles/resourcemanager.organizationAdmin", "members": [ "user:mike@example.com", "group:admins@example.com", "domain:google.com", "serviceAccount:my-project-id@appspot.gserviceaccount.com" ] }, { "role": "roles/resourcemanager.organizationViewer", "members": [ "user:eve@example.com" ], "condition": { "title": "expirable access", "description": "Does not grant access after Sep 2020", "expression": "request.time < timestamp('2020-10-01T00:00:00.000Z')", } } ], "etag": "BwWWja0YfJA=", "version": 3 } YAML example: bindings: - members: - user:mike@example.com - group:admins@example.com - domain:google.com - serviceAccount:my-project-id@appspot.gserviceaccount.com role: roles/resourcemanager.organizationAdmin - members: - user:eve@example.com role: roles/resourcemanager.organizationViewer condition: title: expirable access description: Does not grant access after Sep 2020 expression: request.time < timestamp('2020-10-01T00:00:00.000Z') etag: BwWWja0YfJA= version: 3 For a description of IAM and its features, see the IAM documentation (https://cloud.google.com/iam/docs/).

Request message for SetIamPolicy method.

Request message for TestIamPermissions method.

Response message for TestIamPermissions method.

The request message for Operations.CancelOperation.

The response message for Operations.ListOperations.

This resource represents a long-running operation that is the result of a network API call.

The Status type defines a logical error model that is suitable for different programming environments, including REST APIs and RPC APIs. It is used by gRPC (https://github.com/grpc). Each Status message contains three pieces of data: error code, error message, and error details.You can find out more about this error model and how to work with it in the API Design Guide (https://cloud.google.com/apis/design/errors).

Represents a textual expression in the Common Expression Language (CEL) syntax. CEL is a C-like expression language. The syntax and semantics of CEL are documented at https://github.com/google/cel-spec.Example (Comparison): title: "Summary size limit" description: "Determines if a summary is less than 100 chars" expression: "document.summary.size() < 100" Example (Equality): title: "Requestor is owner" description: "Determines if requestor is the document owner" expression: "document.owner == request.auth.claims.email" Example (Logic): title: "Public documents" description: "Determine whether the document should be publicly visible" expression: "document.type != 'private' && document.type != 'internal'" Example (Data Manipulation): title: "Notification string" description: "Create a notification string with a timestamp." expression: "'New message received at ' + string(document.create_time)" The exact variables and functions that may be referenced within an expression are determined by the service that evaluates it. See the service documentation for additional information.