smart_city v5.0.7 SmartCity.Ingestion View Source

Struct defining an ingestion update event.

const Ingestion = {
  "id": "",
  "name", "",
  "allow_duplicates": boolean,
  "cadence": "",
  "extractSteps": [],
  "schema": [],
  "targetDatasetId": "",
  "sourceFormat": "",
  "topLevelSelector": "",
  "transformations": [],
}

Link to this section Summary

Types

not_required(type)

t()

Functions

delete(struct, key)

get(struct, key, default \\ nil)

new(msg)

Returns a new SmartCity.Ingestion. Can be created from Map with string or atom keys. Raises an ArgumentError when passed invalid input

put(struct, key, val)

Link to this section Types

not_required(type)

not_required(type) :: type | nil

t()

t() :: %SmartCity.Ingestion{
  allow_duplicates: not_required(boolean()),
  cadence: not_required(String.t()),
  extractSteps: [map()],
  id: String.t(),
  name: String.t(),
  schema: [map()],
  sourceFormat: String.t(),
  targetDataset: String.t(),
  topLevelSelector: not_required(String.t()),
  transformations: [SmartCity.Ingestion.Transformation.t()]
}

Link to this section Functions

delete(struct, key)

get(struct, key, default \\ nil)

new(msg)

new(map()) :: t()

Returns a new SmartCity.Ingestion. Can be created from Map with string or atom keys. Raises an ArgumentError when passed invalid input

Parameters

msg: Map with string or atom keys that defines the ingestion metadata

Required Keys:

- targetDataset
- sourceFormat
- name

cadence will default to "never"
allow_duplicates will default to true