Module avro_util

Collections of Avro utility functions shared between other modules.

Authors: Ilya Staheev (ilya.staheev@klarna.com).

Description

Collections of Avro utility functions shared between other modules. Should not be used externally.

Data Types

array_type()

array_type() = #avro_array_type{type = avro:type_or_name(), custom = [avro:custom_prop()]}

avro_type()

avro_type() = primitive_type() | array_type() | enum_type() | fixed_type() | map_type() | record_type() | union_type()

custom_prop_value()

custom_prop_value() = jsone:json_value()

enum_type()

enum_type() = #avro_enum_type{name = avro:name(), namespace = avro:namespace(), aliases = [avro:name()], doc = avro:typedoc(), symbols = [avro:enum_symbol()], fullname = avro:fullname(), custom = [avro:custom_prop()]}

fixed_type()

fixed_type() = #avro_fixed_type{name = avro:name(), namespace = avro:namespace(), aliases = [avro:name()], size = pos_integer(), fullname = avro:fullname(), custom = [avro:custom_prop()]}

lkup()

lkup() = avro:lkup_fun()

map_type()

map_type() = #avro_map_type{type = avro:type_or_name(), custom = [avro:custom_prop()]}

name()

name() = binary()

name_raw()

name_raw() = atom() | string() | binary()

primitive_type()

primitive_type() = #avro_primitive_type{name = avro:name(), custom = [avro:custom_prop()]}

record_type()

record_type() = #avro_record_type{name = avro:name(), namespace = avro:namespace(), doc = avro:typedoc(), aliases = [avro:name()], fields = [avro:record_field()], fullname = avro:fullname(), custom = [avro:custom_prop()]}

type_or_name()

type_or_name() = avro_type() | name_raw()

union_type()

union_type() = #avro_union_type{id2type = avro_union:id2type(), name2id = avro_union:name2id()}

Function Index

canonicalize_aliases/2Convert aliases to full-name representation using provided names and namespaces from the original type.
canonicalize_custom_props/1Ensure all keys and values are binary in custom properties.
canonicalize_name/1Convert atom() | string() to binary().
canonicalize_type_or_name/1Canonicalize name if it is not a type.
delete_opts/2Delete props from prop-list.
encode_defaults/2Go recursively into the type tree to encode default values.
ensure_binary/1Covert atom() | string() binary().
ensure_lkup_fun/1Ensure type lookup fun.
expand_type/2Get type and lookup sub-types recursively.
expand_type/3Expand type in 2 different styles.
flatten_type/1Flatten out all named types, return the extracted format and all (recursively extracted) children types in a list.
get_opt/2Get prop from prop-list, 'error' exception if prop is not found.
get_opt/3Get prop from prop-list, return the given Default if prop is not found.
is_compatible/2Check if writer schema is compatible to reader schema.
make_lkup_fun/2Make a schema store (dict based) and wrap it in a lookup fun.
parse_defaults/2Decode default values into avro:out().
resolve_duplicated_refs/1Resolve duplicated references to internal context.
validate/2Validate type definitions according to given options.
verify_aliases/1Assert validity of aliases.
verify_dotted_name/1Assert validity of a fullname.
verify_names/1Assert validity of a list of names.
verify_type/1Assert that the given type names are not AVRO reserved names.

Function Details

canonicalize_aliases/2

canonicalize_aliases(Aliases, Ns) -> any()

Convert aliases to full-name representation using provided names and namespaces from the original type

canonicalize_custom_props/1

canonicalize_custom_props(Props0::[{KeyIn, ValueIn}]) -> [{KeyOut, ValueOut}]

Ensure all keys and values are binary in custom properties.

canonicalize_name/1

canonicalize_name(Name::name_raw()) -> name()

Convert atom() | string() to binary(). NOTE: Avro names are ascii only, there is no need for special utf8. handling when converting string() to binary().

canonicalize_type_or_name/1

canonicalize_type_or_name(Name::type_or_name()) -> type_or_name()

Canonicalize name if it is not a type.

delete_opts/2

delete_opts(KvList::[{Key, Value}], Keys::[Key]) -> [{Key, Value}]

Delete props from prop-list.

encode_defaults/2

encode_defaults(N::type_or_name(), ParseFun::lkup()) -> avro_type()

Go recursively into the type tree to encode default values. into JSON format, the encoded default values are later used (inline) to construct JSON schema. Type lookup function Lkup is to encode default value for named types

ensure_binary/1

ensure_binary(A::name_raw()) -> binary()

Covert atom() | string() binary().

ensure_lkup_fun/1

ensure_lkup_fun(JSON::avro:schema_all()) -> lkup()

Ensure type lookup fun.

expand_type/2

expand_type(Type::type_or_name(), Sc::avro:schema_all()) -> avro_type() | no_return()

Get type and lookup sub-types recursively. This should allow callers to write a root type to one avsc schema file instead of scattering all named types to their own avsc files.

expand_type/3

expand_type(Type0::type_or_name(), Sc::avro:schema_all(), Style::compact | bloated) -> avro_type() | no_return()

Expand type in 2 different styles. compact: use type name reference if it is a type seen before while traversing the type tree. bloated: use type name reference only if a type is seen before in the current traversing stack. (to avoid stack overflow)

flatten_type/1

flatten_type(TypeName::type_or_name()) -> {type_or_name(), [avro_type()]}

Flatten out all named types, return the extracted format and all (recursively extracted) children types in a list. If the type is named (i.e. record type), the extracted format is its full name, and the type itself is added to the extracted list.

get_opt/2

get_opt(Key, Opts::[{Key, Value}]) -> Value | no_return()

Get prop from prop-list, 'error' exception if prop is not found.

get_opt/3

get_opt(Key, Opts::[{Key, Value}], Value) -> Value

Get prop from prop-list, return the given Default if prop is not found.

is_compatible/2

is_compatible(Reader::avro_type(), Writer::avro_type()) -> true | {false, {not_compatible, term(), term()}} | {false, {reader_missing_defalut_value, term()}}

Check if writer schema is compatible to reader schema.

make_lkup_fun/2

make_lkup_fun(AssignedName::name_raw(), Type::avro_type()) -> lkup()

Make a schema store (dict based) and wrap it in a lookup fun.

parse_defaults/2

parse_defaults(N::type_or_name(), ParseFun::avro_json_decoder:default_parse_fun()) -> type_or_name()

Decode default values into avro:out().

resolve_duplicated_refs/1

resolve_duplicated_refs(Type0::avro:type_or_name()) -> avro:type_or_name()

Resolve duplicated references to internal context. e.g. when two or more record fields share the same type and all are expanded, we should make reference for the later ones and only expand the first.

validate/2

validate(Type::avro:avro_type(), Opts::avro:schema_opts()) -> ok | no_return()

Validate type definitions according to given options.

verify_aliases/1

verify_aliases(Aliases::[name_raw()]) -> ok | no_return()

Assert validity of aliases.

verify_dotted_name/1

verify_dotted_name(Name::name_raw()) -> ok | no_return()

Assert validity of a fullname.

verify_names/1

verify_names(Names::[name_raw()]) -> ok | no_return()

Assert validity of a list of names.

verify_type/1

verify_type(Type::avro_type()) -> ok | no_return()

Assert that the given type names are not AVRO reserved names.


Generated by EDoc