Module gpb_compile

Description
Data Types
Function Index
Function Details

Compile protobuf definition files to a module that can encode and decode values to and from binaries.

Description

Compile protobuf definition files to a module that can encode and decode values to and from binaries.

The Erlang types for the values are as follows

Protobuf type	Erlang type
double, float	float() \| infinity \| '-infinity' \| nan When encoding, integers, too, are accepted
int32, int64 uint32, uint64 sint32, sint64 fixed32, fixed64 sfixed32, sfixed64	integer()
bool	true \| false When encoding, the integers 1 and 0, too, are accepted
enum	atom() unknown enums decode to integer()
message	record (thus tuple()) or map() if the maps (-maps) option is specified
string	unicode string, thus list of integers or binary() if the strings_as_binaries (-strbin) option is specified When encoding, iolists, too, are accepted
bytes	binary() When encoding, iolists, too, are accepted
oneof	`{ChosenFieldName, Value}` or `ChosenFieldName => Value` if the {maps_oneof,flat} (-maps_oneof flat) option is specified
map<_,_>	An unordered list of 2-tuples, `[{Key,Value}]` or a map(), if the maps (-maps) option is specified

Repeated fields are represented as lists.

Optional fields are represented as either the value or undefined if not set. However, for maps, if the option maps_unset_optional is set to omitted, then unset optional values are omitted from the map, instead of being set to undefined when encoding messages. When decoding messages, even with maps_unset_optional set to omitted, the default value will be set in the decoded map.

Data Types

arg()

arg() = term() | named_arg()

arg_template()

arg_template() = [arg()]

bin_ret()

bin_ret() = 
    {ok, module(), code()} | {ok, module(), code(), [warning()]}

boolean_opt()

boolean_opt(X) = X | {X, boolean()}

Just an option X means {X,true}

code()

code() = binary() | gpb_defs:defs() | [code_item()]

code_item()

code_item() = 
    {erl, ErlCode :: binary()} | {nif, NifCcText :: string()}

comp_ret()

comp_ret() = mod_ret() | bin_ret() | defs_ret() | error_ret()

defs_ret()

defs_ret() = 
    {ok, gpb_defs:defs()} | {ok, gpb_defs:defs(), [warning()]}

directory()

directory() = string()

error_ret()

error_ret() = 
    error | {error, reason()} | {error, reason(), [warning()]}

fetcher_ret()

fetcher_ret() = from_file | {ok, string()} | {error, term()}

field_path()

field_path() = [atom() | []]

filename()

filename() = string()

fn_name()

fn_name() = atom()

gpb_field_type()

gpb_field_type() = 
    int32 | int64 | uint32 | uint64 | sint32 | sint64 | fixed32 |
    fixed64 | sfixed32 | sfixed64 | bool | float | double |
    string | bytes |
    {enum, atom()} |
    {msg, atom()} |
    {group, atom()} |
    {map, gpb_map_key(), gpb_map_value()} |
    unknown

gpb_map_key()

gpb_map_key() = 
    int32 | int64 | uint32 | uint64 | sint32 | sint64 | fixed32 |
    fixed64 | sfixed32 | sfixed64 | bool | string

gpb_map_value()

gpb_map_value() = gpb_scalar() | {enum, atom()} | {msg, atom()}

gpb_scalar()

gpb_scalar() = 
    int32 | int64 | uint32 | uint64 | sint32 | sint64 | fixed32 |
    fixed64 | sfixed32 | sfixed64 | bool | float | double |
    string | bytes

import_fetcher_fun()

import_fetcher_fun() = fun((string()) -> fetcher_ret())

io_info_item()

io_info_item() = 
    {erl_output, filename()} |
    {hrl_output, filename()} |
    {nif_cc_output, filename()} |
    {sources, [source()]} |
    {missing, [source()]}

json_array_format()

json_array_format() = list | {atom(), list}

A list or a list in a tagged tuple.

json_format()

json_format() = jsx | mochijson2 | jiffy | maps

Convenience shorthand to specify object, key, array and string and null format.

json_key_format()

json_key_format() = atom | binary | string

json_object_format()

json_object_format() = 
    eep18 | {proplist} | {atom(), proplist} | map

eep18 means objects on format [{}] | proplist(), such as for instance for jsx.
{proplist} means a proplist() in a tuple.
{atom(),proplist} means a proplist() in a tagged tuple, such as {struct, proplist()} for instance for mochijson2.
map means as a map

json_string_format()

json_string_format() = binary | list

list_deps_format()

list_deps_format() = 
    makefile_format |
    {list_imports, newline_terminated} |
    {list_imports, null_terminated}

list_deps_makefile_target()

list_deps_makefile_target() = string() | {quote, string()}

mod_fn_argtemplate()

mod_fn_argtemplate() = {module(), fn_name(), arg_template()}

mod_ret()

mod_ret() = ok | {ok, [warning()]}

msg_name_change()

msg_name_change() = 
    name_change() | {prefix, {by_proto, prefix_by_proto()}}

msg_name_prefix()

msg_name_prefix() = name_part() | {by_proto, prefix_by_proto()}

name_change()

name_change() = 
    {prefix, name_part()} |
    {suffix, name_part()} |
    lowercase | uppercase | snake_case | dots_to_underscores |
    base_name

name_part()

name_part() = string() | atom()

named_arg()

named_arg() = '$1' | '$2' | '$errorf' | '$user_data' | '$op'

new_name()

new_name() = string() | atom()

opt()

opt() = 
    {i, directory()} |
    {o, directory()} |
    {o_erl, directory()} |
    {o_hrl, directory()} |
    {o_nif_cc, directory()} |
    binary | to_proto_defs | to_msg_defs |
    {import_fetcher, import_fetcher_fun()} |
    boolean_opt(ignore_wellknown_types_directory) |
    boolean_opt(strings_as_binaries) |
    boolean_opt(maps) |
    boolean_opt(msgs_as_maps) |
    boolean_opt(mapfields_as_maps) |
    {maps_unset_optional, omitted | present_undefined} |
    {maps_oneof, tuples | flat} |
    {maps_key_type, atom | binary} |
    boolean_opt(allow_preencoded_submsgs) |
    {verify, optionally | always | never} |
    boolean_opt(verify_decode_required_present) |
    boolean_opt(gen_verifiers) |
    {rename, renaming()} |
    {msg_name_prefix, msg_name_prefix()} |
    {msg_name_suffix, name_part()} |
    boolean_opt(msg_name_to_lower) |
    boolean_opt(msg_name_to_snake_case) |
    {module_name_prefix, name_part()} |
    {module_name_suffix, name_part()} |
    {module_name, new_name()} |
    boolean_opt(use_packages) |
    boolean_opt(descriptor) |
    boolean_opt(include_as_lib) |
    {include_mod_hrl_prepend, string()} |
    boolean_opt(bypass_wrappers) |
    {copy_bytes, true | false | auto | integer() | float()} |
    boolean_opt(type_specs) |
    boolean_opt(defaults_for_omitted_optionals) |
    boolean_opt(type_defaults_for_omitted_optionals) |
    {target_erlang_version, target_erlang_version()} |
    boolean_opt(preserve_unknown_fields) |
    boolean_opt(gen_enum_macros) |
    {erlc_compile_options, string()} |
    boolean_opt(gen_encoders) |
    boolean_opt(gen_decoders) |
    {proto_defs_version, gpb_defs:version()} |
    {introspect_proto_defs_version,
     gpb_defs:version() | preferably_1} |
    boolean_opt(introspect_get_proto_defs) |
    boolean_opt(defs_as_proplists) |
    boolean_opt(defs_as_maps) |
    boolean_opt(gen_introspect) |
    boolean_opt(json) |
    boolean_opt(json_always_print_primitive_fields) |
    boolean_opt(json_always_print_fields_with_no_presence) |
    boolean_opt(json_preserve_proto_field_names) |
    boolean_opt(json_case_insensitive_enum_parsing) |
    {json_format, json_format()} |
    {json_object_format, json_object_format()} |
    {json_key_format, json_key_format()} |
    {json_array_format, json_array_format()} |
    {json_string_format, json_string_format()} |
    {json_null, atom()} |
    boolean_opt(nif) |
    {load_nif, string()} |
    boolean_opt(gen_mergers) |
    {translate_type, {gpb_field_type(), [translation()]}} |
    {any_translate, [translation()]} |
    {translate_field, {field_path(), [translation()]}} |
    boolean_opt(epb_compatibility) |
    boolean_opt(epb_functions) |
    {list_deps, list_deps_format()} |
    {list_deps_dest_file, filename()} |
    boolean_opt(list_deps_missing_imports_are_generated) |
    boolean_opt(list_deps_makefile_phonies) |
    {list_deps_makefile_target, list_deps_makefile_target()} |
    boolean_opt(list_deps_and_generate) |
    return |
    boolean_opt(return_warnings) |
    boolean_opt(return_errors) |
    report |
    boolean_opt(report_warnings) |
    boolean_opt(report_errors) |
    boolean_opt(warnings_as_errors) |
    term()

opts()

opts() = [opt()]

prefix_by_proto()

prefix_by_proto() = 
    [{ProtoName :: atom(), Prefix :: name_part()}]

reason()

reason() = term()

renaming()

renaming() = 
    {pkg_name, name_change()} |
    {msg_name, msg_name_change()} |
    {msg_fqname, msg_name_change()} |
    {group_name, name_change()} |
    {group_fqname, name_change()} |
    {service_name, name_change()} |
    {service_fqname, name_change()} |
    {rpc_name, name_change()} |
    {msg_typename, name_change()} |
    {enum_typename, name_change()} |
    {enum_macro, name_change()}

source()

source() = 
    from_input_string |
    {from_fetched, Proto :: filename()} |
    filename()

target_erlang_version()

target_erlang_version() = integer() | current

eg: 23, 24, ...

translation()

translation() = 
    {encode, mod_fn_argtemplate()} |
    {decode, mod_fn_argtemplate()} |
    {decode_init_default, mod_fn_argtemplate()} |
    {decode_repeated_add_elem, mod_fn_argtemplate()} |
    {decode_repeated_finalize, mod_fn_argtemplate()} |
    {merge, mod_fn_argtemplate()} |
    {verify, mod_fn_argtemplate()} |
    {type_spec, string()}

warning()

warning() = term()

Function Index

c/0	Command line interface for the compiler.
c/1	This function is intended as a command line interface for the compiler.
c/2
file/1	Equivalent to `file(File, [])`.
file/2	Compile a .proto file to a .erl file and to a .hrl file.
format_error/1	Produce a plain-text error message from a reason returned by for instance `file/2` or `proto_defs/2`.
format_warning/1	Produce a plain-text error message from a reason returned by for instance `file/2` or `proto_defs/2`.
list_io/2	List inputs and file outputs, based on an input file.
locate_import/2	Locate an import target.
msg_defs/2	Deprecated, use proto_defs/2 instead.
msg_defs/3	Deprecated, use proto_defs/2 instead.
parse_opts_and_args/1
proto_defs/2	Equivalent to `proto_defs(Mod, Defs, [])`.
proto_defs/3	Compile a list of pre-parsed definitions to file or to a binary.
proto_defs/5	Compile a list of pre-parsed definitions to file or to a binary.
read_import/2	Read an import file.
show_args/0
show_version/0
string/2	Equivalent to `string(Mod, Str, [])`.
string/3	Compile a `.proto` file as string.
string_list_io/2	Equivalent to `string_list_io(Mod, Str, [])`.
string_list_io/3	List inputs and file outputs based on proto definitions in a string instead of in a file, similar to `string/3`.

Function Details

c/0

c() -> no_return()

Command line interface for the compiler. With no proto file to compile, print a help message and exit.

c/1

c(Files :: [string() | atom()]) -> no_return()

This function is intended as a command line interface for the compiler. Call it from the command line as follows:

     erl <erlargs> [gpb-opts] -s gpb_compile c File.proto ...
     erl <erlargs> -s gpb_compile c File.proto ... -extra [gpb-opts]

The <erlargs> can be -noshell -noinput +B -boot start_clean -pa SomeDir

If several files are specified, each is compiled individually, no checking is done for instance for multiply defined messages or fields across files, such as the protoc does.

The options below are supported. Dashes and underscores inside option names are equivalent, ie -o-erl and -o_erl are the same option.

Input files and output files and formats

-IDir -I Dir: Specify include directory. Option may be specified more than once to specify several include directories.
Corresponding Erlang-level option: i
-o Dir: Specify output directory for where to generate the ProtoFile.erl and ProtoFile.hrl
-o-erl Dir | -o-hrl Dir | -o-nif-cc Dir: Specify output directory for where to generate the ProtoFile.erl and ProtoFile.hrl respectively, and for the NIF C++ file, if the -nif option is specified. The -o-erl Dir option overrides any -o Dir option, and similarly for the other file-type specific output options.
Corresponding Erlang-level options: o, o_erl, o_hrl and o_nif_cc.
-ignore-priv-dir: Will stop gpb from looking for a well known types directory by trying to locate the priv directory of the gpb app.
Corresponding Erlang-level option: ignore_wellknown_types_directory

Format of the Erlang representation

-strbin

Specify that decoded strings should be returned as binaries, instead of as strings (lists).
Corresponding Erlang-level option: strings_as_binaries

-maps

This option expands to the following options:

-msgs-as-maps
-mapfields-as-maps
-defs-as-maps

See the maps option for the function file/2 for more info.
Corresponding Erlang-level options: maps

-msgs-as-maps

Specifies that messages should be maps. No .hrl file will be generated. Without this option, messages will be records.
Corresponding Erlang-level option: msgs_as_maps

-mapfields-as-maps

Specifies that fields of type map<_,_> should be maps. Otherwise, they will be 2-tuples.
Corresponding Erlang-level option: mapfields_as_maps

-maps_unset_optional omitted | present_undefined

Specifies the internal format for optional fields that are unset.
Corresponding Erlang-level option: maps_unset_optional

-maps_oneof tuples | flat

Specifies the internal format for oneof fields in maps. (Requires -maps and -maps_unset_optional omitted, of which the latter is default since 4.0.0.)
Corresponding Erlang-level option: maps_oneof

-maps_key_type atom | binary

Specifies the key type for maps.
Corresponding Erlang-level option: maps_key_type

-allow-preencoded-submsgs

Allow pre-encoded submsgs to save cpu during encoding
Corresponding Erlang-level option: allow_preencoded_submsgs

Verification of input

-v optionally | always | never: Specify how the generated encoder should verify the message to be encoded.
Corresponding Erlang-level option: verify
-vdrp: Specify if the generated decoder should verify presence of required fields.
Corresponding Erlang-level option: verify_decode_required_present
-no-gen-verifiers: Do not generate verify_msg functions. Implies -v never.
Corresponding Erlang-level option: gen_verifiers

Renaming for the Erlang side

-rename What:How

The following What values are available:

pkg_name: Modify the package name. Useful with the -pkgs option.
msg_name: Modify message names, but not the package part of them
msg_fqname: Modify message names, including their package parts. Useful with the -pkgs option.
group_name: Group names.
group_fqname: Group names including their package parts.
service_name: Service names.
service_fqname: Service names including their package parts.
rpc_name: The RPC name.
msg_typename: Erlang type names for messages and groups.
enum_typename: Erlang type names for enums.
enum_macro: Enum macros.

The following How values are available:

prefix=Prefix: Prepend the Prefix to the beginning of the name.
suffix=Suffix: Append the Suffix to the end of the name.
lowercase: Example: from MsgName to msgname
uppercase: Example: from MsgName to MSGNAME
snake_case: Example: from MsgName to msg_name
dots_to_underscores: Example: from Msg.Sub to Msg_Sub
base_name: Example: from Domain.Pkg.Msg to Msg
proto=Proto:prefix=Prefix[,...]: For each message belonging the the .proto Proto, without the .proto suffix, prepend Prefix to the beginning of the name. This only works for msg_name and msg_fqname.

It is possible to specify more than one -rename option, and they are applied in the order specified.
Corresponding Erlang-level option: rename

-msgprefix Prefix

Prefix each message with Prefix. This can be useful to when including different sub-projects that have colliding message names.
Corresponding Erlang-level option: msg_name_prefix

-msgsuffix Suffix

Suffix each message name with Suffix.
Corresponding Erlang-level option: msg_name_suffix

-msgtolower

ToLower each message. Any prefixes/suffixes are added after case modification.
Corresponding Erlang-level option: msg_name_to_lower

-modprefix Prefix

Prefix each module with Prefix. Normally the module name of the generated code is based on the name of the .proto file. This option prepends a prefix to the module name, which can be useful when including different sub-projects that have colliding proto file names.
Corresponding Erlang-level option: module_name_prefix

-modsuffix Suffix

Suffix each module name with Suffix.
Corresponding Erlang-level option: module_name_suffix

-modname Name

Specify the name of the generated module.
Corresponding Erlang-level option: module_name

What to generate and how

-pkgs: Prepend the name of a package to every message it contains. If no package is defined, nothing will be prepended. Default is to not prepend package names for backwards compatibility, but it is needed for some proto files.
Corresponding Erlang-level option: use_packages
-descr: Generate self-description information.
Corresponding Erlang-level option: descriptor
-il: Generate code that include gpb.hrl using -include_lib instead of -include, which is the default.
Corresponding Erlang-level option: include_as_lib
-include-mod-hrl-prepend string(): Specify a string to prepend to the generated -include("mod.hrl"). (when generating for records).
Corresponding Erlang-level option: include_mod_hrl_prepend
-bypass-wrappers: Bypass wrappers.
Corresponding Erlang-level option: bypass_wrappers
-c true | false | auto | integer() | float(): Specify how or when the generated decoder should copy fields of type bytes. See the copy_bytes option for the function file/2 for more info.
Corresponding Erlang-level option: copy_bytes
-type -no_type: Enables or disables ::Type() annotations in the generated code. Default is to enable if there are no cyclic dependencies.
Corresponding Erlang-level option: type_specs
-defaults-for-omitted-optionals: For optional fields not present on decoding, set the field to its default value, if any, instead of to undefined.
Corresponding Erlang-level option: defaults_for_omitted_optionals
-type-defaults-for-omitted-optionals: For optional fields not present on decoding, set the field to its type-default, instead of to undefined.
Corresponding Erlang-level option: type_defaults_for_omitted_optionals
-for-version N: Generate code for Erlang/OTP version N instead of current.
Corresponding Erlang-level option: target_erlang_version
-preserve-unknown-fields: Preserve unknown fields. An extra field, $unknowns, will be added to each record or map for storing unknown fields.
Corresponding Erlang-level option: preserve_unknown_fields
-gen-enum-macros: Generate macro definitions for enum symbols. Note that this causes a .hrl file to be generated even with the -maps option.
Corresponding Erlang-level option: gen_enum_macros
-erlc_compile_options Options: Specifies compilation options, in a comma separated string, to pass along to the -compile(...) directive on the generated code.
Corresponding Erlang-level option: erlc_compile_options
-no-gen-encoders: Do not generate encode_msg functions. Implies -no-gen-verifiers.
Corresponding Erlang-level option: gen_encoders
-no-gen-decoders: Do not generate decode_msg functions. Implies -no-gen-mergers.
Corresponding Erlang-level option: gen_decoders

Introspection of the proto definitions

-introspect-get_proto_defs: For introspection, generate a get_proto_defs/0 function\n" instead of get_msg_defs/0.
Corresponding Erlang-level option: introspect_get_proto_defs
-pldefs: Specify that introspection functions shall return proplists instead of #field{} records, to make the generated code completely free of even compile-time dependencies to gpb.
Corresponding Erlang-level option: defs_as_proplists
-defs-as-maps: Specifies that proto definitions from the generated code are to be returned as maps. Otherwise, they will be lists of tuples and records (or proplists if the -pldefs option is specified)
Corresponding Erlang-level option: defs_as_maps
-no-gen-introspect: Do not generate code for introspection.
Corresponding Erlang-level option: gen_introspect

JSON

-json

Generate functions for converting to and from a JSON representation.
Corresponding Erlang-level option: json

-json-always-print-primitive-fields

This is a synonym for -json-always-print-fields-with-no-presence
Corresponding Erlang-level option: json_always_print_primitive_fields

-json-always-print-fields-with-no-presence

Print also (proto3) fields whose value is omitted or have their type's default.
Corresponding Erlang-level option: json_always_print_fields_with_no_presence

-json-preserve-proto-field-names

Print the fields' names as in .proto file, not as lowerCamelCase.
Corresponding Erlang-level option: json_preserve_proto_field_names

-json-case-insensitive-enum-parsing

Make case insignificant when parsing enums in JSON. Also allow dash as alternative to undercore. Default is that case is significant when parsing enums.
Corresponding Erlang-level option: json_case_insensitive_enum_parsing

-json-format jsx | mochijson2 | jiffy | maps

Specify format for the JSON representation. maps is default if the -maps option is specified, otherwise the jsx format is default.
Corresponding Erlang-level option: json_format

-json-object-format eep18 | tpl | tpl:Tag | map

Specify JSON object format:

eep18 means [{}] | proplist(), this is the default.
tpl means {proplist()}.
tpl:Tag means {Tag, proplist()}.
map means map().

Corresponding Erlang-level option: json_object_format

-json-key-format binary | atom | string

Specify JSON object key format. binary is the default.
Corresponding Erlang-level option: json_key_format

-json-array-format list | tl:Tag

Specify JSON object array format.

list means list(), this is the default.
tl:Tag means {Tag, list()}.

Corresponding Erlang-level option: json_array_format

-json-string-format binary | list

Specify JSON string format. binary is the default.
Corresponding Erlang-level option: json_string_format

-json-null Null

Specify JSON null value, null is the default.
Corresponding Erlang-level option: json_null

NIF

-nif: Generate nifs for linking with the protobuf C(++) library.
Corresponding Erlang-level option: nif
-load_nif FunctionDefinition: Specify FunctionDefinition as the text that defines the function load_nif/0. This is called as the on_load hook for loading the NIF. See also the doc for the load_nif option in the file/2 function.
Corresponding Erlang-level option: load_nif
-no-gen-mergers: Do not generate code for merging of messages. This is only useful with the option -nif.
Corresponding Erlang-level option: gen_mergers

Translations

-translate_type TMsFs

Call functions in TMsFs to pack, unpack, merge and verify for the specified type. The TMsFs is a string on the following format: type=Type,e=Mod:Fn,d=Mod:Fn[,m=Mod:Fn][,V=Mod:Fn]. The Type and specified modules and functions are called and used as follows:

type=Type: Specifies that the translations apply to fields of type. The Type may be either of: msg:MsgName (after any renaming operations), enum:EnumName, int32, int64, uint32, uint64, sint32, sint64, fixed32, fixed64, sfixed32, sfixed64, bool, double, string, bytes or map<KeyType,ValueType>. The last may need quoting in the shell.
e=Mod:Fn: Call Mod:Fn(Term) to pack the Term to a value of type Type, ie to a value that gpb knows how to wire-encode.
d=Mod:Fn: Call Mod:Fn(Value) to unpack the just wire-decoded Value of type Type, to something of your choice.
m=Mod:Fn: Call Mod:Fn(Term1, Term2) -> Term3 to merge two unpacked terms to a resulting Term3. Note that this function is never called for scalar types.
V=Mod:Fn: Call Mod:Fn(Term) -> _ to verify an unpacked Term. If Term is valid, the function is expected to just return any value, which is ignored and discarded. If Term is invalid, the function is expected to not return anything, but instead either crash, call erlang:error/1, or throw/1 or exit/1 with the reason for error. If you want to use a verifier, this is the new preferred approach.
v=Mod:Fn: Call Mod:Fn(Term, ErrorF) -> _ to verify an unpacked Term. This exists for backwards compatibility, and its use is deprecated.

Corresponding Erlang-level option: translate_type

-any_translate MsFs

Call functions in MsFs to pack, unpack, merge and verify google.protobuf.Any messages. The MsFs is a string on the following format: e=Mod:Fn,d=Mod:Fn[,m=Mod:Fn][,V=Mod:Fn]. See the translate option for details on the string components.
Corresponding Erlang-level option: any_translate

-translate_field FMsFs

Call functions in FMsFs to pack, unpack, merge, and verify. This is similar to the -translate_type option, except that a message field is specified instead of a type. The FMsFs is a string on the following format: field=Path,e=...,d=...,m=...,V=...[,i=Mod:Fn][,a=Mod:Fn][,f=Mod:Fn] See the -translate_type option for info on e=, d=, m= and V= items. Additionally for this -translate_field option, these exist:

field=Path

The Path indicates the element to translate as follows:

MsgName for the message itself. (This is actually equivalent to -translate_type type=msg:MsgName,...)
MsgName.FieldName for fields generally
MsgName.OneofName.FieldName for oneof fields
MsgName.FieldName.[] for elements of repeated fields

i=Mod:Fn

For repeated fields, call Mod:Fn() on decoding to initialize the field to some value

a=Mod:Fn

For repeated fields, call Mod:Fn(Elem,S) on decoding to add an item)

f=Mod:Fn

For repeated fields, call Mod:Fn(S) on decoding to finalize the field

Corresponding Erlang-level option: translate_field

Compatibility with Erlang protobuffs

-epb

Enable compatibility with the Erlang Protobuffs library:

Implies the -epb-functions option
Implies the -defaults-for-omitted-optionals option
Implies the -modsuffix _pb option
Implies the -msgtolower option

Corresponding Erlang-level option: epb_compatibility

-epb-functions

For compatibility with the Erlang Protobuffs library, generate also the following functions: encode/1, decode/2, encode_MsgName/1 and decode_MsgName/1
Corresponding Erlang-level option: epb_functions

Querying dependencies

-M: Generate Makefile rule(s) to stdout for dependencies. No code is generated unless -MMD is specified.
Corresponding Erlang-level option: list_deps
-ML: Print imports, one per line instead of on Makefile format. No code is generated unless -MMD is specified.
Corresponding Erlang-level option: list_deps
-M0: Print imports, each terminated by a null character, instead of on Makefile format. No code is generated unless -MMD is specified.
Corresponding Erlang-level option: list_deps
-MF file: Specify a file to write dependency rules to, instead of printing them to stdout. No code is generated unless -MMD is specified.
Corresponding Erlang-level option: list_deps_dest_file
-MG: Consider missing imports to be generated and include\n" them in the listed dependencies or rules.
Corresponding Erlang-level option: list_deps_missing_imports_are_generated
-MP: Generate phony Makefile targets for dependencies.
Corresponding Erlang-level option: list_deps_makefile_phonies
-MT target: Override the Makefile rule target.
Corresponding Erlang-level option: list_deps_makefile_target
-MQ target: Same as -MT but quote characters special to make.
Corresponding Erlang-level option: list_deps_makefile_target
-MMD: List imports or Makefile rules and generate code. This option works like in erlc, which contrasts to gcc.
Corresponding Erlang-level option: list_deps_and_generate

Errors and warnings

-Werror, -W1, -W0, -W, -Wall: -Werror means treat warnings as errors
-W1 enables warnings, -W0 disables warnings.
-W and -Wall are the same as -W1
Corresponding Erlang-level option: warnings_as_errors

Help and info

--help or -h: Show help.
--version or -V: Show the version number of gpb.

c/2

c(Opts :: opts(), Args :: [ProtoFileName :: string()]) ->
     no_return()

file/1

file(File :: string()) -> comp_ret()

Equivalent to file(File, []).

file/2

file(File :: string(), Opts :: opts()) -> comp_ret()

Compile a .proto file to a .erl file and to a .hrl file.

The File argument must not include path to the .proto file. Example: "SomeDefinitions.proto" is ok, while "/path/to/SomeDefinitions.proto" is not ok.

Option overview

Options that are documented only as an atom can generally be specified either as the atom, or as {the_atom, boolean()}.

Input files and output files and formats: {i, string()}, {o_erl, string()}, {o_hrl, string()}, {o_nif_cc, string()}, binary, to_proto_defs, to_msg_defs, {import_fetcher, import_fetcher_fun()}, ignore_wellknown_types_directory
Format of the Erlang representation: strings_as_binaries, maps, msgs_as_maps, mapfields_as_maps, {maps_unset_optional, omitted|present_undefined}, {maps_oneof, tuples|flat}, {maps_key_type, atom|binary}, allow_preencoded_submsgs
See also use_packages.
Verification of input: {verify, always|never|optionally}, verify_decode_required_present, gen_verifiers
Renaming for the Erlang side: {rename, renaming()}, {msg_name_prefix, msg_name_prefix(), {msg_name_suffix, name_part()}, msg_name_to_lower, msg_name_to_snake_case, {module_name_prefix, name_part()}, {module_name_suffix, name_part()}, {module_name new_name()}
What to generate and how: use_packages, descriptor, include_as_lib, {include_mod_hrl_prepend, string()}, bypass_wrappers, {copy_bytes, false|true|auto|integer()|float()}, type_specs, defaults_for_omitted_optionals, type_defaults_for_omitted_optionals, {target_erlang_version, target_erlang_version()}, preserve_unknown_fields, gen_enum_macros, {erlc_compile_options, string()}, {gen_encoders,boolean()} {gen_decoders,boolean()}
See also gen_introspect and gen_verifiers
Introspection of the proto definitions: {proto_defs_version, gpb_defs:version()}, {introspect_proto_defs_version, gpb_defs:version()|preferably_1}, introspect_get_proto_defs, defs_as_proplists, defs_as_maps, gen_introspect
See also to_proto_defs.
JSON: json, json_always_print_primitive_fields, json_always_print_fields_with_no_presence, json_preserve_proto_field_names, json_case_insensitive_enum_parsing, {json_format, json_format()}, {json_object_format, json_object_format()}, {json_key_format, json_key_format()}, {json_array_format, json_array_format()}, {json_string_format, json_string_format()}, {json_null, atom()}
NIF: nif, {load_nif, string()}, gen_mergers
See also o_nif_cc
Translations: {translate_type, {gpb_field_type(),[translation()]}}, {any_translate, [translation()]}, {translate_field, {field_path(),[translation()]}}
Compatibility with Erlang protobuffs: epb_compatibility, epb_functions
Querying dependencies: {list_deps, list_deps_format()}, {list_deps_dest_file, string()}, list_deps_and_generate, list_deps_missing_imports_are_generated, list_deps_makefile_phonies, {list_deps_makefile_target, list_deps_makefile_target()}
Errors and warnings: report, report_errors, report_warnings,
return, return_errors, return_warnings,
warnings_as_errors

Input files and output files and formats

`{i, string()}`

The .proto file is expected to be found in a directories specified by an {i,Directory} option. It is possible to specify {i,Directory} several times, they will be searched in the order specified.

Corresponding command line option: -I.

`{o, string()}`
`{o_erl, string()}`
`{o_hrl, string()}`
`{o_nif_cc, string()}`

The {o,Directory} option specifies directory to use for storing the generated .erl and .hrl files. Default is the same directory as for the proto File.

The {o_erl,Directory}, {o_hrl,Directory}, {o_nif_cc,Directory}, options specify output directories for where to generate the .erl and .hrl files respectively, and for the NIF C++ file, if the nif option is specified. The {o_erl,Directory} option overrides any {o,Directory} option, and similarly for the other file-type specific output options.

Corresponding command line options: -o, -o-erl, -o-hrl, -o-nif-cc.

`binary`

The binary option will cause the generated and compiled code to be returned as a binary. No files will be written. The return value will be on the form {ok,Mod,Code} or {ok,Mod,Code,Warnings} if the compilation is successful. This option may be useful e.g. when generating test cases. In case the nif option is set, the Code will be a list of tuples: {erl,binary()} which contains the Erlang object byte code, and {nif,binary()} which contains the C++ code. You will have to compile the C++ code with a C++ compiler, before you can use the Erlang code.

`to_proto_defs`

The to_proto_defs option will result in {ok,Defs} or {ok,Defs,Warns} being returned if the compilation is successful. The returned message definitions can be used with the proto_defs/2 or proto_defs/3 functions.

The to_msg_defs option is a deprecated alias for to_proto_defs.

`{import_fetcher,import_fetcher_fun()}`

The import_fetcher option can be used to customize fetching of imports. The option value is be a function taking one argument, the name of the file to import. It must return either from_file, letting this file pass through the normal file import, or {ok,string()} if it has fetched the file itself, or {error,term()}. See the import_fetcher_fun().

`ignore_wellknown_types_directory`

The {ignore_wellknown_types_directory, true} option will stop gpb from looking for a well known types directory by trying to locate the priv directory of the gpb application. This can be used either when this directory is not available or to provide a custom set of well known types.

Corresponding command line option: -ignore-priv-dir.

Format of the Erlang representation

`strings_as_binaries`

The strings_as_binaries option specifies whether strings should be returned from decoding as strings (list of Unicode code points), or as binaries (UTF-8 encoded). The copy_bytes option applies to strings as well, when the strings_as_binaries option is set. Upon encoding, both binaries and iolists are accepted.

Corresponding command line option: -strbin.

`maps`
`msgs_as_maps`
`mapfields_as_maps`

The maps option will generate a protobuf encoder/decoder that uses maps instead of records. This option expands to the following options:

msgs_as_maps: No .hrl file will be generated, and the functions encode_msg, merge_msgs and verify_msg will take the message name as an additional parameter.
mapfields_as_maps: The value for fields of type map<_,_> will be a map instead of a list of 2-tuples.
defs_as_maps: The introspection will generate message field descriptions as maps, see the defs_as_maps. for further info.

Corresponding command line options: -maps, -msgs-as-maps, -mapfields-as-maps.

`{maps_unset_optional, omitted|present_undefined}`

For messages as maps, for optional fields, if not set, the maps_unset_optional option specifies the Erlang-internal representation; both how it is expected to be found at encoding, and how decoding will return it, for proto2 syntax:

omitted: This means it is not included in the map. This is the default. (since gpb version 4.0.0)
present_undefined: This means it is present and has the value undefined. This was the default before gpb version 4.0.0.

For proto3 syntax, the scene is a bit different. In proto3 all fields are kind-of optional, but omitted scalar fields, strings and bytes decode to their type-default. On encoding with proto3, a field that has its type-default value is not included in the encoded binary, so one could say that even though all fields are optional, in a conceptual way, all scalars, strings and bytes always have a value. The exceptions are sub-messages and oneof fields, and for these fields, this option has the meaning as indicated above.

Corresponding command line option: -maps_unset_optional.

`{maps_oneof, tuples|flat}`

The maps_oneof option can be used for messages as maps, and can only take effect if maps_unset_optional is omitted (default since 4.0.0). It changes the representation of oneof fields as described below, if we would have a oneof-field, xf with two alternatives a1 and a2:

{maps_oneof,tuples}: #{xf => {a1, Value}} or #{xf => {a2, Value}}
{maps_oneof,flat}: #{a1 => Value} or #{a2 => Value}

Corresponding command line option: -maps_oneof.

`{maps_key_type, atom|binary}`

For messages as maps, the maps_key_type option makes it possible to control whether keys should be atoms (default) or binaries.

Corresponding command line option: -maps_key_type.

`allow_preencoded_submsgs`

Allow pre-encoded submsgs to save cpu during encoding. A sub-message can then be specified as a binary. It is not possible to combine this option neither with the option nif nor with encoding to json.

Corresponding command line option: -allow-preencoded-submsgs.

Related options

use_packages

Verification of input

`{verify, always|never|optionally}`

The verify option specifies whether or not to generate code that verifies, during encoding, that values are of correct type and within range. The verify option can have the following values:

always: Generate code that unconditionally verifies values.
never: Generate code that never verifies values time. Encoding will fail if a value of the wrong type is supplied. This includes forgetting to set a required message field. Encoding may silently truncate values out of range for some types.
optionally: Generate an encode_msg/2 that accepts the run-time option verify or {verify,boolean()} for specifying whether or not to verify values.

Erlang value verification either succeeds or crashes with the error {gpb_type_error,Reason}. Regardless of the verify option, a function, verify_msg/1 is always generated.

Note that the verify_msg functions are still generated, even with {verify,never}. If needed, user code might want to call them explicitly. See the {gen_verifiers, false} option for a way to control this.

Corresponding command line option: -v.

`verify_decode_required_present`

The verify_decode_required_present option tells gpb to emit checks that on decoding, required fields are present in the binary to decode. If they are not present, decoding will fail with an error.

Corresponding command line option: -vdrp.

`gen_verifiers`

The {gen_verifiers,false} option tells gpb to not emit code that can verify type and range of the Erlang values before encoding. The main purpose is as a means to reduce the size of the generated code. Setting this option to false also implicitly sets the {verify, never} option.

Corresponding command line option: -no-gen-verifiers.

Renaming for the Erlang side

`{rename, renaming()}`

The {rename,{What,How}} can transform message names, package names, service and rpc names in various ways. This option supersedes the options {msg_name_prefix,Prefix}, {msg_name_suffix,Suffix}, msg_name_to_lower and msg_name_to_snake_case, while at the same time giving more fine-grained control. It is for example possible to apply snake_casing only to the message name, while keeping the package name, the service name and the rpc name intact. This can be useful with grpc, where these name components are exposed. The msg_fqname refers to the fully qualified message name, as in Package.MsgName, while the msg_name refers to just the message name without package. The service_fqname and service_name specifiers work analogously. The enum_typename and msg_typename operate on any enum or msg renamings already applied. The enum_macro operates on enum macros (see gen_enum_macros).

It is possible to stack rename options, and they will be applied in the order they are specified. So it is for example possible to snake_case a name, and then also prefix it.

Corresponding command line option: -rename.

`{msg_name_prefix, msg_name_prefix()}`
`{msg_name_suffix, name_part()}`

The {msg_name_prefix,Prefix} will add Prefix (a string or an atom) to each message. This might be useful for resolving colliding names, when incorporating several protocol buffer definitions into the same project. The {msg_name_suffix,Suffix} works correspondingly.

The {msg_name_prefix,Prefix} option expands to [{rename,{pkg_name,Prefix}},{rename,{msg_fqname,{prefix,Prefix}}}, {rename,{group_fqname,{prefix,Prefix}}}}], and ditto for suffixes.

For backwards compatibility, the {msg_name_prefix,{by_proto,PrefixList}} expands to just [{rename,{msg_fqname,{prefix,PrefixList}}}].

Corresponding command line options: -msgprefix and -msgsuffix.

`msg_name_to_lower`
`msg_name_to_snake_case`

The msg_name_to_lower and msg_name_to_snake_case options expands to [{rename,{pkg_name,X}},{rename,{service_fqname,X}}, {rename,{rpc_name,X}},{rename,{msg_fqname,X}}, {rename,{rpc_name,X}},{rename,{group_fqname,X}}] where X is lowercase or snake_case respectively.

`{module_name_prefix, name_part()}`
`{module_name_suffix, name_part()}`

The {module_name_prefix,Prefix} will add Prefix (a string or an atom) to the generated code and definition files. The {module_name_suffix,Suffix} works correspondingly. For the case of compatibility with Erlang Protobuffs, the epb_compatibility option implies {module_name_suffix,"_pb"}

Corresponding command line options: -modprefix and -modsuffix.

`{module_name, new_name()}`

The {module_name,Name} can be used to specify the module name of the generated code freely, instead of basing it on the proto file name. The name specified with module_name can be prefixed and suffixed with the module_name_prefix and module_name_suffix options.

Corresponding command line option: -modname.

What to generate and how

`use_packages`

The use_packages option instructs gpb to prepend the name of a package to every message it contains. If no package is defined, nothing will be prepended. This enables the reference of messages in other packages which would otherwise not be possible. However, for reasons of backward compatibility, this option is disabled by default.

Corresponding command line option: -pkgs.

`descriptor`

The descriptor option specifies whether or not to generate a function, descriptor/0, which returns a binary that describes the proto file(s) contents according to the protobuf's descriptor.proto. The default is to not generate such a description. The generated description binary is most likely not identical to what protoc would generate, but the contents is roughly equivalent.

Corresponding command line option: -descr.

`include_as_lib`

Generate code that includes gpb.hrl using -include_lib instead of -include, which is the default.

Corresponding command line option: -il.

`{include_mod_hrl_prepend, string()}`

Generate code that prepends the specified string before <output mod>.hrl. Default is the empty string. No slashes are added. If you need to prepend a directory, you also need to include the slash in the string to prepend.

Corresponding command line option: -include-mod-hrl-prepend.

`bypass_wrappers`

The bypass_wrappers option exposes the more-or-less internal top-level encode and decode functions without wrappers. The list below describe what functionality the wrappers provide. The main purpose of being able to bypass the wrappers is performance, especially when combined with the nif option. This option causes the following extra functions to be exported:

encode_msg_MsgName/1
encode_msg_MsgName/2 unless nif
decode_msg_MsgName/1
decode_msg_MsgName/2 unless nif

For encode, the wrapper takes care of:

Any calling of verifiers.
Reading of options for any translation user_data.

For decode, the wrapper takes care of:

Wrapping the decode in a try catch to provide a uniform error format when the binary to be decoded is invalid.
Reading of options for any translation user_data.

Corresponding command line option: -bypass-wrappers,

`{copy_bytes, false|true|auto|integer()|float()}`

The copy_bytes option specifies whether when decoding data of type bytes (or strings if the strings_as_binaries is set), the decoded bytes should be copied or not. Copying requires the binary module, which first appeared in Erlang R14A. When not copying decoded bytes, they will become sub binaries of the larger input message binary. This may tie up the memory in the input message binary longer than necessary after it has been decoded. Copying the decoded bytes will avoid creating sub binaries, which will in turn make it possible to free the input message binary earlier. The copy_bytes option can have the following values:

false: Never copy bytes/(sub-)binaries.
true: Always copy bytes/(sub-)binaries.
auto: Synonym for true. (This is the default)
integer() | float(): Copy the bytes/(sub-)binaries if the message this many times or more larger than the size of the bytes/(sub-)binary.

Corresponding command line option: -c.

`type_specs`

The type_specs option enables or disables ::Type() annotations in the generated .hrl file. Default is true. The default changed in gpb version 4.0.0. Previously, the default was false. If you have messages referencing other messages cyclically, and get into troubles when compiling the generated files, set this to false.

Corresponding command line options: -type. -no_type.

`defaults_for_omitted_optionals`
`type_defaults_for_omitted_optionals`

The defaults_for_omitted_optionals and type_defaults_for_omitted_optionals options generates code that set default values or type-defaults respectively, on decoding, if an optional field is not present in the binary to decode. Normally it would otherwise have been set to undefined. Note that with these options it is not possible to determine after decoding whether a field contained data in the binary message. Also note that these options are only applicable for proto2 syntax messages, and are ignored for proto3 syntax messages. (For proto3, it effectively must be ignored, since, on the wire, a field set to its type-default value is indistinguishable from an omitted value.)

Corresponding command line options: -defaults-for-omitted-optionals, -type-defaults-for-omitted-optionals.

`{target_erlang_version, target_erlang_version()}`

The target_erlang_version can be used to specify another major version of Erlang/OTP to generate code for. The default, current means that the generated code is expected to be compiled and run on the same major version as gpb runs on.

Corresponding command line option: -for-version,

`preserve_unknown_fields`

The preserve_unknown_fields option will add a field to records and maps. On decoding, info on unknown fields will be stored in here, such that the unknown fields can be preserved on encoding. An unknown field is a field with an unknown number. Without this option, such fields are skipped on decoding. There is no guarantee that a message with unknowns will be byte-by-byte identical when re-encoded.

Corresponding command line option: -preserve-unknown-fields.

`gen_enum_macros`

The gen_enum_macros option causes macros to be emitted on the form indicated by the following example:

     x.proto:
       syntax="proto3";
       message Msg {
         enum Status { NOT_SET = 0; FAILURE = 1; SUCCESS = 2; }
         Status f = 1;
       }
     x.hrl:
       -define('Msg.Status.NOT_SET', 'NOT_SET').
       -define('Msg.Status.FAILURE', 'FAILURE').
       -define('Msg.Status.SUCCESS', 'SUCCESS').

The intention is to make it possible to catch errors already at compile-time if any enum symbol would get renamed in a future version of the proto file. Note that this option will cause .hrl files to be generated, even with the maps option.

Corresponding command line option: -gen-enum-macros.

`{erlc_compile_options, string()}`

If the the {erlc_compile_options,string()} option is set, then the generated code will contain a directive -compile([String]).

Corresponding command line option: -erlc_compile_options.

`gen_encoders`

The {gen_encoders,false} option tells gpb to not emit code for encoding messages. This may be useful to reduce the size of the generated code in cases when no encoding is needed. The default is to generate encoders. Setting this option to false also implicitly sets the {gen_verifiers,false} option.

Corresponding command line option: -no-gen-encoders.

`gen_decoders`

The {gen_decoders,false} option tells gpb to not emit code for decoding messages. This may be useful to reduce the size of the generated code in cases when no decoding is needed. The default is to generate decoders. Setting this option to false also implicitly sets the {gen_mergers,false} option.

Corresponding command line option: -no-gen-decoders.

Introspection of the proto definitions

`defs_as_proplists`

The defs_as_proplists option changes the generated introspection functions find_msg_def, get_msg_defs and get_proto_defs to return the description of each message field as a proplist, instead of as a #field{} record. The purpose is to make the generated code completely independent of gpb, at compile-time (it is already independent at run-time). The keys of the proplist are the names of the record fields in the #field{} record. See also gpb:proplists_to_field_records() and related functions for conversion functions between these two formats.

Corresponding command line option: -pldefs.

`defs_as_maps`

The introspection will generate message field descriptions as maps instead of as #field{} records, unless, of course defs_as_proplists is specified, in which case they will be proplists instead.

Corresponding command line option: -defs-as-maps.

`{proto_defs_version, gpb_defs:version()}`
`{introspect_proto_defs_version, gpb_defs:version()|preferably_1}`

The proto_defs_version can be used to specify version of definitions returned with the to_proto_defs option. See the file doc/dev-guide/proto-defs-versions.md for some more info. Not all proto definitions may be expressible in all versions. In gpb-4.x.y it defaults to 1. The introspect_proto_defs_version can be used to specify the version returned by the generated introspection functions, default is 1 if possible, else 2.

`introspect_get_proto_defs`

When the introspect_get_proto_defs option is set, the introspection function will include get_proto_defs/0 instead of get_msg_defs/0. The get_msg_defs/0 returns a list of messages and enums, while the get_proto_defs/0 returns the same definitions returned when the to_proto_defs option is used.

Corresponding command line option: -introspect-get_proto_defs.

`gen_introspect`

The {gen_introspect,false} option will cause gpb to not generate code for introspection. One rationale for this is option is to reduce the size of the generated code.

Corresponding command line option: -no-gen-introspect.

Related options

to_proto_defs

JSON

`json`

The json option will cause gpb to also generate functions for converting between internal format and a JSON representation.

Note that gpb will not encode to an actual JSON text. Instead, it returns an Erlang structure that can be used with some other JSON library to turn it into actual JSON text. Ditto for decoding. It is possible to flexibly specify details of the JSON representation, with shorthand presets for some common libraries.

However, with the nif option, the generated code uses the Google C++ protobuf library, which produces already-formatted JSON text, as binaries, with no further processing required. When nif is specified, the various JSON format options are thus not used. The json_always_print_primitive_fields, the json_always_print_fields_with_no_presence, the json_preserve_proto_field_names and the json_case_insensitive_enum_parsing options are honoured with nif, though.

Corresponding command line option: -json.

`json_always_print_primitive_fields`

This is a synonym for json_always_print_fields_with_no_presence.

Corresponding command line option: -json-always-print-primitive-fields.

`json_always_print_fields_with_no_presence`

The json_always_print_fields_with_no_presence makes the generated to_json function always emit json key-value items also when the value is the type's default value. The default is to omit such values, as per the language guide. This holds for messages in files with proto3 syntax.

When the option nif is set, it sets the JSON print option always_print_fields_with_no_presence if the protobuf library is version 26.0 or newer. If the protobuf library is older, it sets the print option always_print_primitive_fields instead. These differ in whether omitted optional proto2 fields are printed.

Corresponding command line option: -json-always-print-fields-with-no-presence.

`json_preserve_proto_field_names`

The json_preserve_proto_field_names makes the generated to_json function always use the field name in the .proto file. The default is to use lowerCamelCase, as per the language guide.

Corresponding command line option: -json-preserve-proto-field-names.

`json_case_insensitive_enum_parsing`

If the the json_case_insensitive_enum_parsing option is specified, case is not significant when parsing json enums. Also, dashes instead of underscores are allowed. Default is that that case is significant.

Corresponding command line option: -json-case-insensitive-enum-parsing.

`{json_format, json_format()}`

The {json_format,Format} option is a convenience shorthand, and will expand as indicated below. If the json_format is not specified, it defaults to maps if the maps option is specified, and otherwise to eep18 when generating code for records.

jsx: [{json_object_format, eep18}, {json_key_format, binary}, {json_array_format, list}, {json_string_format, binary}, {json_null, null}]
mochijson2: [{json_object_format, {struct, proplist}}, {json_key_format, binary}, {json_array_format, list}, {json_string_format, binary}, {json_null, null}]
jiffy: [{json_object_format, {proplist}}, {json_key_format, binary}, {json_array_format, list}, {json_string_format, binary}, {json_null, null}]
maps: [{json_object_format, map}, {json_key_format, binary}, {json_array_format, list}, {json_string_format, binary}, {json_null, null}]

Corresponding command line option: -json-format.

`{json_object_format, json_object_format()}`

The {json_object_format,Format} option specifies the format of json object, as indicated below. (Note that the format of the keys is specified by the json_key_format option, see further below.)

eep18: The empty json object is represented as [{}].
Non-empty json objects are represented as proplists.
{proplist}: A json object is represented as a proplist in a tuple.
{atom(), proplist}: A json object is represented as a proplist in a tagged tuple, with the possibility to specify the tag.
map: The json object is represented as an Erlang map.

Corresponding command line option: -json-object-format.

`{json_key_format, json_key_format()}`

The {json_key_format,Format} option specifies the format of json object keys, as follows:

atom
binary (default)
string

Corresponding command line option: -json-key-format.

`{json_array_format, json_array_format()}`

The {json_array_format,Format} option specifies the format of json arrays, as follows:

list (default)
{atom(), list} A list in a tagged tuple, with the possibility to specify the tag.

Corresponding command line option: -json-array-format.

`{json_string_format, json_string_format()}`

The {json_string_format,Format} option specifies the format of json arrays, as follows:

list
binary (default)

Corresponding command line option: -json-string-format.

`{json_null, atom()}`

The {json_null,atom()} option specifies the atom to use for the JSON null value. The default is to use the atom null.

Corresponding command line option: -json-null.

NIF

`nif`

The nif option will cause the compiler to generate nif C++ code for encoding and decoding. The generated nif C++ code can be linked with the Google protobuf C++ library. Read the file README.nif-cc for more info.

Corresponding command line option: -nif.

`{load_nif, string()}`

The option {load_nif,FunctionDefinition} allows to specify FunctionDefinition as the text that defines the function load_nif/0. This is called as the -on_load. hook for loading the NIF.

The string can contain some special terms that will get replaced as follows:

{{nifbase}}: This will be replaced with the string "Mod.nif". where Mod is the name of the Erlang module.
{{loadinfo}}: This will be replaced with the version of gpb, as a list of integers and strings, see gpb:version_as_list/0.

Corresponding command line option: -load_nif.

`gen_mergers`

The {gen_mergers,false} option will cause gpb to not generate code for merging of messages. This is only useful with the option nif or with the option {gen_decoders,false}. One rationale for this is option is to reduce the size of the generated code.

Corresponding command line option: -no-gen-mergers.

Related options

{o_nif_cc,string()}

Translations

`{translate_type, {gpb_field_type(),[translation()]}}`

The translate_type option can be used to provide packer and unpacker functions for message fields of a certain type. For messages, the MsgName refers to a name after renaming has taken place. The merge translator is optional, and is called either via the merge_msgs function in the generated code, or when the decoder sees another field of the same type. The default merge operation is to let the second element overwrite previous elements. The verify translator is optional too, since verification can be disabled. The translation calls are specified as {Mod,Fn,ArgTemplate} where Mod,Fn is a module and function to call, ArgTemplate is a list of terms, containing markers, such as $1, $2 and so on, for where to place the actual args. This makes it possible to specify additional static argument terms, for instance. The translator functions are called as follows:

Encode (Packing): Call Mod:Fn(Term) to pack the Term ($1) to a value of the suitable for normal gpb encoding.
Decode (Unpacking): Call Mod:Fn(Any) to unpack the Any ($1) to unpack a normal gpb decoded value to a term.
Merge: Call Mod:Fn(Term1, Term2) -> Term3 to merge two unpacked terms to a resulting Term3. The $1 is the previously seen term (during decoding, on encountering a second field of the same type), or the first argument to the merge_msgs function. The $2 is the lastly seen term, or the second argument to the merge_msgs function.
Verify: Call Mod:Fn(Term) -> _ to verify an unpacked Term. If Term ($1) is valid, the function is expected to just return any value, which is ignored and discarded. If Term is invalid, the function is expected to not return anything, but instead either crash, call erlang:error/1, or throw/1 or exit/1. with the reason for error. (For backwards compatibility, it is also possible to have an error function as argument, using $errorf, but this is deprecated.)

There are additional translator argument markers:

$user_data: This will be replaced by the user_data option to the generated encode_msg, decode_msg, merge_msgs and verify_msg functions. If that option is not specified, the value undefined is used substituted for $user_data.
$op: This will be replaced by encode, decode, merge, verify, decode_init_default, decode_repeated_add_elem or decode_repeated_finalize, depending on from which context it is actually called. This can be useful because if the message is to be verified on encoding (see the verify option), then the same options, and thus the same user-data, are used for both encode_msg and for verify_msg. The $op marker makes it possible to tell these two call sites apart, if needed.

Corresponding command line option: -translate_type.

`{any_translate, [translation()]}}`

The option {any_translate,Translations} is retained for backwards compatibility, and expands to {translate_type, {'google.protobuf.Any',Translations}}.

Corresponding command line option: -any_translate.

`{translate_field, {field_path(),[translation()]}`

The translate_field option can be used to translate individual fields. The option format is {translate_field,{FieldPath,Translations}} where each Translation consists of {Op,{Mod,Fn,ArgTemplate}} elements, just as for translate_type. The FieldPath is a list on the following format:

[MsgName] for the message itself on the top-level
[MsgName,FieldName] for fields, generally
[MsgName,FieldName,[]] for elements of repeated fields
[MsgName,OneofFieldName,FieldName] for elements of oneof fields.

For repeated fields, the additional operations decode_init_default, decode_repeated_add_elem and decode_repeated_finalize also exist and must all be specified.

For translated proto3 message fields -- ie fields for messages in files with syntax="proto3"; -- the decode callback will be invoked also initially when the decoding of the message binary is about to start. This holds for non-repeated non-oneof fields of integer types, enums, strings and bytes. The decode callback will be invoked with the type's default value. This is because in proto3, a field with the type's default value is never included in the resulting wire octets, so on decoding, gpb initially assumes such fields have the type's default value, and the translator needs to be invoked accordingly.

Corresponding command line option: -translate_field.

Compatibility with Erlang protobuffs

These are for compatibility with Erlang protobuffs. library.

`epb_compatibility`
`epb_functions`

The epb_compatibility option is an umbrella-option for compatibility with the Erlang protobuffs library. It will expand to the options below. It will expand in-place, meaning any of these can be overridden if specified before the epb_compatibility option.

epb_functions
defaults_for_omitted_optionals
{module_name_suffix, "_pb"}
{msg_name_to_lower, true}

If the epb_functions option is specified, then for compatibility with Erlang protobuffs, the following functions will be generated:

encode/1
encode_<MsgName>/1
decode/2
decode_<MsgName>/1

Corresponding command line options: -epb, -epb-functions.

Querying dependencies

`{list_deps, list_deps_format()}`
`{list_deps_dest_file, string()}`
`list_deps_and_generate`
`list_deps_missing_imports_are_generated`
`list_deps_makefile_phonies`
`{list_deps_makefile_target, list_deps_makefile_target()}`

A set of options will cause file/2 and string/3 to list import dependencies as described below. To retrieve dependency information as Erlang terms, see list_io/2 and string_list_io/3.

{list_deps, Format} {list_deps_dest_file, Filename::string()}: Either or both of these two options without the list_deps_and_generate option described below, will cause dependencies to be listed, and no code to be generated. The default format is Makefile format, ie with only the list_deps_dest_file option. The default destination is to print to standard output, ie with only the list_deps option.
list_deps_and_generate: If specified, dependencies will be listed, and code will be generated. .

Formats:

{list_deps, makefile_format}: Makefile rules will be generated. The top-level .proto file will be the first dependency followed by imported .proto files, and the .erl file will be the target. Outdir options as well as module renaming options are considered for target. Include path options are considered for the dependencies.
{list_deps, {list_imports, newline_terminated}}: Imports of the .proto file will be listed, one per line.
{list_deps, {list_imports, null_terminated}}: Like newline_terminated but instead, each import is followed by a NUL character. This can be useful when paths have spaces or newline or other strange characters.
list_deps_missing_imports_are_generated: Consider missing imports to be generated, and include them in the dependency list

Some of the options apply only when the Target is makefile_format:

{list_deps_makefile_target, Target}

Override the default target for the Makefile rule, as follows:

Target :: string(): Use the specified value instead.
Target :: {quote, string()}: Same, but quote characters special to make.

list_deps_makefile_phonies

Generate phony Makefile targets for dependencies.

Corresponding command line options: -M, -ML, -M0, -MF, -MG, -MP, -MT, -MQ and -MMD.

Errors and warnings

`report`, `report_errors`, `report_warnings`
`return`, `return_errors`, `return_warnings`

report_errors/report_warnings: Causes errors/warnings to be printed as they occur.
report: This is a short form for both report_errors and report_warnings.
return_errors: If this flag is set, then {error,ErrorList,WarningList} is returned when there are errors.
return_warnings: If this flag is set, then an extra field containing WarningList is added to the tuples returned on success.
return: This is a short form for both return_errors and return_warnings.

`warnings_as_errors`

Setting the warnings_as_errors option will cause warnings to be treated as errors. If there are warnings but no errors, and return_warnings is not specified, then error will be returned.

See format_error/1 for a way to turn an error Reason to plain text.

Corresponding command line option: -Werror.

format_error/1

format_error(Err) -> iolist()

Err = 
    reason() | {error, reason()} | {error, reason(), [warning()]}

Produce a plain-text error message from a reason returned by for instance file/2 or proto_defs/2.

format_warning/1

format_warning(X :: warning()) -> iolist()

Produce a plain-text error message from a reason returned by for instance file/2 or proto_defs/2.

list_io/2

list_io(FileName :: filename(), Opts :: opts()) ->
           [io_info_item()]

List inputs and file outputs, based on an input file. The elements erl_output, hrl_output and nif_cc_output are present if output would be generated to those based on the options. The file is scanned for import "file.proto"; declarations recursively.

The imported files are located the same way file/2 would find them, using options such as {i,Directory}. For more info, See the inputs and outputs section.

If there is an error parsing the input proto file or an imported file, it is treated as it it was a file with no imports. Thus it is still included in sources. If a file in an import declaration cannot be found, it is included among the missing items.

The first element in sources is always the input file. The rest of the elements are any imported files in breadth-first order. An imported file is in sources only once, even if it is imported multiple times.

Any renaming options are used to compute the output file names. Any import fetcher options are used to retrieve files.

locate_import/2

locate_import(ProtoFileName :: string(), Opts :: opts()) ->
                 {ok, File :: string()} | {error, reason()}

Locate an import target. This function might be potentially useful for instance in an intercepting import_fetcher_fun() that just wants to record the accessed imports.

msg_defs/2

msg_defs(Mod :: module(), Defs :: gpb_defs:defs()) -> comp_ret()

Equivalent to msg_defs(Mod, Defs, []).

Deprecated, use proto_defs/2 instead.

msg_defs/3

msg_defs(Mod :: module(), Defs :: gpb_defs:defs(), Opts :: opts()) ->
            comp_ret()

Equivalent to proto_defs(Mod, Defs, Opts).

Deprecated, use proto_defs/2 instead.

parse_opts_and_args/1

parse_opts_and_args(Argv :: [string()]) ->
                       {ok, {opts(), Args :: [string()]}} |
                       {error, Reason :: string()}

proto_defs/2

proto_defs(Mod :: module(), Defs :: gpb_defs:defs()) -> comp_ret()

Equivalent to proto_defs(Mod, Defs, []).

proto_defs/3

proto_defs(Mod :: module(),
           Defs :: gpb_defs:defs(),
           Opts :: opts()) ->
              comp_ret()

Compile a list of pre-parsed definitions to file or to a binary. See file/2 for information on options and return values.

proto_defs/5

proto_defs(Mod, Defs, DefsNoRenamings, Renamings, Opts) -> any()

Compile a list of pre-parsed definitions to file or to a binary. This is useful when there are renamings, and one nifs or descriptors are to be generated, since these need the original definitions before any renamings. The renaming must be applied separately, see gpb_names:compute_renamings/2 and gpb_names:apply_renamings/2. See gpb_names:is_renaming_opt/1 for how to filter options. See file/2 for information on options and return values.

read_import/2

read_import(File :: string(), Opts :: opts()) ->
               {ok, string()} | {error, reason()}

Read an import file. This function might be potentially useful for instance in an intercepting import_fetcher fun that just wants to record the accessed imports.

show_args/0

show_args() -> term()

show_version/0

show_version() -> term()

string/2

string(Mod :: module(), Str :: string()) -> comp_ret()

Equivalent to string(Mod, Str, []).

string/3

string(Mod :: module(), Str :: string(), Opts :: opts()) ->
          comp_ret()

Compile a .proto file as string. See file/2 for information on options and return values.

string_list_io/2

string_list_io(Mod :: module(), Str :: string()) ->
                  [io_info_item()]

Equivalent to string_list_io(Mod, Str, []).

string_list_io/3

string_list_io(Mod :: module(), Str :: string(), Opts :: opts()) ->
                  [io_info_item()]

List inputs and file outputs based on proto definitions in a string instead of in a file, similar to string/3. See list_io/2 for further information.

Generated by EDoc

Module gpb_compile

Option overview

Input files and output files and formats

{i, string()}

{o, string()} {o_erl, string()} {o_hrl, string()} {o_nif_cc, string()}

binary

to_proto_defs

{import_fetcher,import_fetcher_fun()}

ignore_wellknown_types_directory

Format of the Erlang representation

strings_as_binaries

maps msgs_as_maps mapfields_as_maps

{maps_unset_optional, omitted|present_undefined}

{maps_oneof, tuples|flat}

{maps_key_type, atom|binary}

allow_preencoded_submsgs

Related options

Verification of input

{verify, always|never|optionally}

verify_decode_required_present

gen_verifiers

Renaming for the Erlang side

{rename, renaming()}

{msg_name_prefix, msg_name_prefix()} {msg_name_suffix, name_part()}

msg_name_to_lower msg_name_to_snake_case

{module_name_prefix, name_part()} {module_name_suffix, name_part()}

{module_name, new_name()}

What to generate and how

use_packages

descriptor

include_as_lib

{include_mod_hrl_prepend, string()}

bypass_wrappers

{copy_bytes, false|true|auto|integer()|float()}

type_specs

defaults_for_omitted_optionals type_defaults_for_omitted_optionals

{target_erlang_version, target_erlang_version()}

preserve_unknown_fields

gen_enum_macros

{erlc_compile_options, string()}

gen_encoders

gen_decoders

Related options

Introspection of the proto definitions

defs_as_proplists

defs_as_maps

{proto_defs_version, gpb_defs:version()} {introspect_proto_defs_version, gpb_defs:version()|preferably_1}

introspect_get_proto_defs

gen_introspect

Related options

JSON

json

json_always_print_primitive_fields

json_always_print_fields_with_no_presence

json_preserve_proto_field_names

json_case_insensitive_enum_parsing

{json_format, json_format()}

{json_object_format, json_object_format()}

{json_key_format, json_key_format()}

{json_array_format, json_array_format()}

{json_string_format, json_string_format()}

{json_null, atom()}

NIF

nif

{load_nif, string()}

gen_mergers

Related options

Translations

{translate_type, {gpb_field_type(),[translation()]}}

{any_translate, [translation()]}}

{translate_field, {field_path(),[translation()]}

Compatibility with Erlang protobuffs

epb_compatibility epb_functions

Querying dependencies

{list_deps, list_deps_format()} {list_deps_dest_file, string()} list_deps_and_generate list_deps_missing_imports_are_generated list_deps_makefile_phonies {list_deps_makefile_target, list_deps_makefile_target()}

Errors and warnings

report, report_errors, report_warnings return, return_errors, return_warnings

warnings_as_errors

`{i, string()}`

`{o, string()}`
`{o_erl, string()}`
`{o_hrl, string()}`
`{o_nif_cc, string()}`

`binary`

`to_proto_defs`

`{import_fetcher,import_fetcher_fun()}`

`ignore_wellknown_types_directory`

`strings_as_binaries`

`maps`
`msgs_as_maps`
`mapfields_as_maps`

`{maps_unset_optional, omitted|present_undefined}`

`{maps_oneof, tuples|flat}`

`{maps_key_type, atom|binary}`

`allow_preencoded_submsgs`

`{verify, always|never|optionally}`

`verify_decode_required_present`

`gen_verifiers`

`{rename, renaming()}`

`{msg_name_prefix, msg_name_prefix()}`
`{msg_name_suffix, name_part()}`

`msg_name_to_lower`
`msg_name_to_snake_case`

`{module_name_prefix, name_part()}`
`{module_name_suffix, name_part()}`

`{module_name, new_name()}`

`use_packages`

`descriptor`

`include_as_lib`

`{include_mod_hrl_prepend, string()}`

`bypass_wrappers`

`{copy_bytes, false|true|auto|integer()|float()}`

`type_specs`

`defaults_for_omitted_optionals`
`type_defaults_for_omitted_optionals`

`{target_erlang_version, target_erlang_version()}`

`preserve_unknown_fields`

`gen_enum_macros`

`{erlc_compile_options, string()}`

`gen_encoders`

`gen_decoders`

`defs_as_proplists`

`defs_as_maps`

`{proto_defs_version, gpb_defs:version()}`
`{introspect_proto_defs_version, gpb_defs:version()|preferably_1}`

`introspect_get_proto_defs`

`gen_introspect`

`json`

`json_always_print_primitive_fields`

`json_always_print_fields_with_no_presence`

`json_preserve_proto_field_names`

`json_case_insensitive_enum_parsing`

`{json_format, json_format()}`

`{json_object_format, json_object_format()}`

`{json_key_format, json_key_format()}`

`{json_array_format, json_array_format()}`

`{json_string_format, json_string_format()}`

`{json_null, atom()}`

`nif`

`{load_nif, string()}`

`gen_mergers`

`{translate_type, {gpb_field_type(),[translation()]}}`

`{any_translate, [translation()]}}`

`{translate_field, {field_path(),[translation()]}`

`epb_compatibility`
`epb_functions`

`{list_deps, list_deps_format()}`
`{list_deps_dest_file, string()}`
`list_deps_and_generate`
`list_deps_missing_imports_are_generated`
`list_deps_makefile_phonies`
`{list_deps_makefile_target, list_deps_makefile_target()}`

`report`, `report_errors`, `report_warnings`
`return`, `return_errors`, `return_warnings`

`warnings_as_errors`