smart_city
v6.0.0

View Source SmartCity.Ingestion (smart_city v6.0.0)

Struct defining an ingestion update event.

const Ingestion = {
  "id": "",
  "name", "",
  "allow_duplicates": boolean,
  "cadence": "",
  "extractSteps": [],
  "schema": [],
  "targetDatasets": ["", ""],
  "sourceFormat": "",
  "topLevelSelector": "",
  "transformations": [],
}

Link to this section Summary

Types

not_required(type)
t()

Functions

delete(struct, key)
get(struct, key, default \\ nil)
new(msg)

Returns a new SmartCity.Ingestion. Can be created from Map with string or atom keys. Raises an ArgumentError when passed invalid input

put(struct, key, val)

Link to this section Types

Link to this type

not_required(type)

View Source 
@type not_required(type) :: type | nil
Link to this type

t()

View Source 
@type t() :: %SmartCity.Ingestion{
  allow_duplicates: not_required(boolean()),
  cadence: not_required(String.t()),
  extractSteps: [map()],
  id: String.t(),
  name: String.t(),
  schema: [map()],
  sourceFormat: String.t(),
  targetDatasets: [String.t()],
  topLevelSelector: not_required(String.t()),
  transformations: [SmartCity.Ingestion.Transformation.t()]
}

Link to this section Functions

Link to this function

delete(struct, key)

View Source
Link to this function

get(struct, key, default \\ nil)

View Source
Link to this function

new(msg)

View Source 
@spec new(map()) :: t()

Returns a new SmartCity.Ingestion. Can be created from Map with string or atom keys. Raises an ArgumentError when passed invalid input

parameters

 Parameters

  • msg: Map with string or atom keys that defines the ingestion metadata

Required Keys:

- targetDatasets
- sourceFormat
- name
  • cadence will default to "never"
  • allow_duplicates will default to true
Link to this function

put(struct, key, val)

View Source