Module avro

General Avro handling code.

Description

General Avro handling code.

Data Types

array_type()

array_type() = #avro_array_type{type = avro:type_or_name(), custom = [avro:custom_prop()]}

avro_encoding()

avro_encoding() = avro_json | avro_binary

avro_type()

avro_type() = primitive_type() | array_type() | enum_type() | fixed_type() | map_type() | record_type() | union_type()

avro_value()

avro_value() = avro:canonicalized_value() | #avro_value{type = avro:type_or_name(), data = avro:avro_value()} | [#avro_value{type = avro:type_or_name(), data = avro:avro_value()}] | [{avro:name(), #avro_value{type = avro:type_or_name(), data = avro:avro_value()}}] | avro_map:data() | {json, binary()} | {binary, binary()}

canonicalized_value()

canonicalized_value() = null | boolean() | integer() | float() | binary()

codec_options()

codec_options() = [proplists:property()]

crc64_fingerprint()

crc64_fingerprint() = avro_fingerprint:crc64()

custom_prop()

custom_prop() = {custom_prop_name(), custom_prop_value()}

custom_prop_name()

custom_prop_name() = binary()

custom_prop_value()

custom_prop_value() = jsone:json_value()

decode_fun()

decode_fun() = fun((type_or_name(), binary()) -> out())

decoder_hook_fun()

decoder_hook_fun() = fun((avro_type(), name() | integer(), avro:in(), fun((avro:in()) -> avro:out())) -> avro:out())

decoder_options()

decoder_options() = #{encoding := avro_binary | avro_json, map_type := proplist | map, record_type := proplist | map, is_wrapped := boolean(), hook := decoder_hook_fun()}

encode_fun()

encode_fun() = fun((type_or_name(), in()) -> iodata() | avro_value())

encoding()

encoding() = avro_encoding()

enum_index()

enum_index() = non_neg_integer()

enum_symbol()

enum_symbol() = binary()

enum_symbol_raw()

enum_symbol_raw() = atom() | string() | binary()

enum_type()

enum_type() = #avro_enum_type{name = avro:name(), namespace = avro:namespace(), aliases = [avro:name()], doc = avro:typedoc(), symbols = [avro:enum_symbol()], fullname = avro:fullname(), custom = [avro:custom_prop()]}

fixed_type()

fixed_type() = #avro_fixed_type{name = avro:name(), namespace = avro:namespace(), aliases = [avro:name()], size = pos_integer(), fullname = avro:fullname(), custom = [avro:custom_prop()]}

fullname()

fullname() = binary()

in()

in() = null | boolean() | integer() | float() | binary() | iolist() | map() | [avro:in()] | [{name_raw(), avro:in()}]

lkup_fun()

lkup_fun() = fun((fullname()) -> avro_type())

map_type()

map_type() = #avro_map_type{type = avro:type_or_name(), custom = [avro:custom_prop()]}

name()

name() = binary()

name_raw()

name_raw() = atom() | string() | binary()

namespace()

namespace() = binary()

namespace_raw()

namespace_raw() = atom() | string() | binary()

ordering()

ordering() = ascending | descending | ignore

out()

out() = null | boolean() | integer() | float() | binary() | [avro:out()] | [{name(), avro:out()}]

primitive_type()

primitive_type() = #avro_primitive_type{name = avro:name(), custom = [avro:custom_prop()]}

record_field()

record_field() = #avro_record_field{name = name(), doc = typedoc(), type = type_or_name(), default = undefined | avro:in() | avro_value(), order = ordering(), aliases = [name()]}

record_type()

record_type() = #avro_record_type{name = avro:name(), namespace = avro:namespace(), doc = avro:typedoc(), aliases = [avro:name()], fields = [avro:record_field()], fullname = avro:fullname(), custom = [avro:custom_prop()]}

schema_all()

schema_all() = avro_type() | binary() | lkup_fun() | schema_store()

schema_opts()

schema_opts() = proplists:proplist()

schema_store()

schema_store() = avro_schema_store:store()

simple_decoder()

simple_decoder() = fun((binary()) -> out())

simple_encoder()

simple_encoder() = fun((in()) -> iodata())

type_or_name()

type_or_name() = avro_type() | name_raw()

type_prop_name()

type_prop_name() = namespace | doc | aliases | custom_prop_name()

type_prop_value()

type_prop_value() = namespace() | typedoc() | [name()] | custom_prop_value()

type_props()

type_props() = [{type_prop_name(), type_prop_value()}]

typedoc()

typedoc() = string() | binary()

union_index()

union_index() = non_neg_integer()

union_type()

union_type() = #avro_union_type{id2type = avro_union:id2type(), name2id = avro_union:name2id()}

Function Index

build_type_fullname/2Constructs the type's full name from provided name and namespace.
canonical_form_fingerprint/1Encode type into canonical form JSON schema and return its crc64 fingerprint.
cast/2Tries to cast a value (which can be another Avro value or some erlang term) to the specified Avro type performing conversion if required.
crc64_fingerprint/1Calculate hash using the Avro CRC 64 algorithm.
decode/5Decode value return unwarpped values.
decode_schema/1Decode JSON format avro schema into erlavro internals.
decode_schema/2Decode JSON format avro schema into erlavro internals Supported options: * ignore_bad_default_values: boolean() Some library may produce invalid default values, if this option is set, bad default valus will be whatever values obtained from JSON decoder.
encode/4Encode value to json or binary format.
encode_schema/1Encode erlavro internals type records into JSON format.
encode_schema/2Encode erlavro internals type records into JSON format.
encode_wrapped/4Encode value and return the result wrapped with type info.
expand_type/2Equivalent to avro_util:expand_type(Type, Store, compact).
expand_type_bloated/2Equivalent to avro_util:expand_type(Type, Store, bloated).
flatten_type/1Equivalent to avro_util:flatten_type(Type).
get_aliases/1Returns aliases for the type.
get_custom_props/1Get custom type properties such as logical type info.
get_custom_props/2Get custom type properties such as logical type info.
get_type_fullname/1Returns fullname stored inside the type.
get_type_name/1Returns the type's name.
get_type_namespace/1Returns the type's namespace exactly as it is set in the type.
is_compatible/2Check whether two schemas are compatible in sense that data encoded with the writer schema can be decoded by the reader schema.
is_named_type/1Returns true if the type can have its own name defined in schema.
make_decoder/2Make a decoder function.
make_decoder_options/1Build decoder options with default values.
make_encoder/2Make a encoder function.
make_lkup_fun/1Make type lookup function from type definition.
make_lkup_fun/2Make type lookup function.
make_simple_decoder/2Make a decoder function.
make_simple_encoder/2Make a encoder function.
name2type/1Create primitive type definiton from name.
resolve_fullname/2Recursively resolve children type's fullname with enclosing namespace passed down from ancestor types.
split_type_name/2Splits type's name parts to its canonical short name and namespace.
to_term/1Convert avro values to erlang term.

Function Details

build_type_fullname/2

build_type_fullname(TypeName::name_raw(), Namespace::namespace()) -> fullname()

Constructs the type's full name from provided name and namespace.

canonical_form_fingerprint/1

canonical_form_fingerprint(Type::avro_type()) -> crc64_fingerprint()

Encode type into canonical form JSON schema and return its crc64 fingerprint.

cast/2

cast(T1::type_or_name(), Avro_value::avro:in()) -> {ok, avro_value()} | {error, term()}

Tries to cast a value (which can be another Avro value or some erlang term) to the specified Avro type performing conversion if required.

crc64_fingerprint/1

crc64_fingerprint(Bin::binary()) -> crc64_fingerprint()

Calculate hash using the Avro CRC 64 algorithm.

decode/5

decode(Encoding::avro_encoding(), Data::binary(), TypeOrName::type_or_name(), StoreOrLkup::schema_store() | lkup_fun(), Hook::decoder_hook_fun()) -> term()

Decode value return unwarpped values.

decode_schema/1

decode_schema(JSON::binary()) -> avro_type()

Decode JSON format avro schema into erlavro internals.

decode_schema/2

decode_schema(JSON::binary(), Options::proplists:proplist()) -> avro_type()

Decode JSON format avro schema into erlavro internals Supported options: * ignore_bad_default_values: boolean() Some library may produce invalid default values, if this option is set, bad default valus will be whatever values obtained from JSON decoder. However, the encoder built from this schema may crash in case bad default value is used (e.g. when a record field is missing from encoder input) * allow_bad_references: boolean() This option is to allow referencing to a non-existing type. So types can be defined in multiple JSON schema files and all imported to schema store to construct a valid over-all schema. * allow_type_redefine: boolean() This option is to allow one type being defined more than once.

encode/4

encode(StoreOrLkup::schema_store() | lkup_fun(), Type::type_or_name(), Value::term(), X4::avro_encoding()) -> iodata()

Encode value to json or binary format.

encode_schema/1

encode_schema(Type::avro_type()) -> binary()

Encode erlavro internals type records into JSON format.

encode_schema/2

encode_schema(Type::avro_type(), Options::schema_opts()) -> binary()

Encode erlavro internals type records into JSON format. Supported options: * canon: boolean() Encode parsing canonical form format

encode_wrapped/4

encode_wrapped(S::schema_all(), TypeOrName::type_or_name(), Value::term(), Encoding::avro_encoding()) -> avro_value()

Encode value and return the result wrapped with type info. The result can be used as a 'trusted' part of a higher level wrapper structure. e.g. encode a big array of some complex type and use the result as a field value of a parent record

expand_type/2

expand_type(Type::type_or_name(), Sc::schema_all()) -> avro_type() | none()

Equivalent to avro_util:expand_type(Type, Store, compact).

expand_type_bloated/2

expand_type_bloated(Type::type_or_name(), Sc::schema_all()) -> avro_type() | none()

Equivalent to avro_util:expand_type(Type, Store, bloated).

flatten_type/1

flatten_type(Type::avro_type()) -> {avro_type() | fullname(), [avro_type()]} | none()

Equivalent to avro_util:flatten_type(Type).

get_aliases/1

get_aliases(Avro_array_type::avro_type()) -> [fullname()]

Returns aliases for the type. Types without aliases defined are considered to have empty alias list. All aliases have been canonicalized (as fullname).

get_custom_props/1

get_custom_props(Avro_array_type::avro_type()) -> [custom_prop()]

Get custom type properties such as logical type info.

get_custom_props/2

get_custom_props(NameOrType::type_or_name(), Store::schema_all()) -> [custom_prop()]

Get custom type properties such as logical type info. Lookup fun is called to retrieve the type definition from schema store in case it is type name provided.

get_type_fullname/1

get_type_fullname(Avro_array_type::type_or_name()) -> name() | fullname()

Returns fullname stored inside the type. For unnamed types their Avro name is returned.

get_type_name/1

get_type_name(Avro_array_type::avro_type()) -> name()

Returns the type's name. If the type is named then content of its name field is returned which can be short name or full name, depending on how the type was specified. If the type is unnamed then Avro name of the type is returned.

get_type_namespace/1

get_type_namespace(Avro_array_type::avro_value() | avro_type()) -> namespace()

Returns the type's namespace exactly as it is set in the type. Depending on how the type was specified it could the namespace or just an empty binary if the name contains namespace in it. If the type can't have namespace then empty binary is returned.

is_compatible/2

is_compatible(ReaderSchema::avro_type(), WriterSchema::avro_type()) -> true | {false, {not_compatible, term(), term()}} | {false, {reader_missing_defalut_value, term()}}

Check whether two schemas are compatible in sense that data encoded with the writer schema can be decoded by the reader schema. Schema compatibility is described here: https://avro.apache.org/docs/1.8.1/spec.html#Schema+Resolution

is_named_type/1

is_named_type(Avro_enum_type::avro_type()) -> boolean()

Returns true if the type can have its own name defined in schema.

make_decoder/2

make_decoder(Schema::schema_all(), Options::codec_options()) -> decode_fun()

Make a decoder function. Supported codec options: * {encoding, avro_binary | avro_json}, default = avro_binary To get a decoder function for JSON or binary encoded data * {map_type, proplist | map}, default = proplist Whether map should be decoded as proplist or map * {record_type, proplist | map}, default = proplist Whether record should be decoded as proplist or map * hook, default = ?DEFAULT_DECODER_HOOK The default hook is a dummy one (does nothing). see avro_decoder_hooks.erl for details and examples of decoder hooks.

make_decoder_options/1

make_decoder_options(Options::codec_options()) -> decoder_options()

Build decoder options with default values.

make_encoder/2

make_encoder(Schema::schema_all(), Options::codec_options()) -> encode_fun()

Make a encoder function. Supported codec options: * {encoding, avro_binary | avro_json}, default = avro_binary To get a encoder function for JSON or binary encoding * wrapped | {wrapped, true}, default = false when 'wrapped' is not in the option list, or {wrapped, false} is given, return encoded iodata() without type info wrapped around. A wrapped result can be used as 'already encoded' part to inline a wrapper object.

make_lkup_fun/1

make_lkup_fun(Type::avro_type()) -> lkup_fun()

Make type lookup function from type definition. The given type is flattened then the flat types are keyed by their fullnames and aliases.

make_lkup_fun/2

make_lkup_fun(AssignedName::name_raw(), Type::avro_type()) -> lkup_fun()

Make type lookup function. The root type is also keyed by the give assigned-name.

make_simple_decoder/2

make_simple_decoder(JSON::avro_type() | binary(), Options::codec_options()) -> simple_decoder()

Make a decoder function. Supported codec options: * {encoding, avro_binary | avro_json}, default = avro_binary To get a decoder function for JSON or binary encoded data * {map_type, proplist | map}, default = proplist Whether map should be decoded as proplist or map * {record_type, proplist | map}, default = proplist Whether record should be decoded as proplist or map * hook, default = ?DEFAULT_DECODER_HOOK The default hook is a dummy one (does nothing). see avro_decoder_hooks.erl for details and examples of decoder hooks. A simple decoder can only be built from self-contained full schema. And unlike a regular decoder (from make_decoder/2), a simple decoder takes only one binary() input arg.

make_simple_encoder/2

make_simple_encoder(JSON::binary() | avro_type(), Options::codec_options()) -> simple_encoder()

Make a encoder function. Supported codec options: * {encoding, avro_binary | avro_json}, default = avro_binary To get a encoder function for JSON or binary encoding A simple container can only be built from self-contained full schema. And unlike a regular encoder (from make_encoder/2), a simple encoder takes only one avro:in() input arg, and does not support wrapped option.

name2type/1

name2type(Name::name_raw()) -> primitive_type() | name()

Create primitive type definiton from name. In case the give name is not a primitive type name, its canonicalized format is returned.

resolve_fullname/2

resolve_fullname(Type::name() | avro_type(), Ns::namespace()) -> fullname() | avro_type()

Recursively resolve children type's fullname with enclosing namespace passed down from ancestor types.

split_type_name/2

split_type_name(TypeName0::type_or_name(), Namespace0::name_raw()) -> {name(), namespace()}

Splits type's name parts to its canonical short name and namespace.

to_term/1

to_term(Avro_value::avro_value()) -> out()

Convert avro values to erlang term.


Generated by EDoc