Multibase v0.0.1 Multibase View Source

This module provides encoding, decoding, and convenience functions for working with Multibase.

Multibase is a protocol for distinguishing base encodings and other simple string encodings, and for ensuring full compatibility with program interfaces. It answers the question:

  Given data d encoded into string s, how can I tell what base d is encoded with?

Base encodings exist because transports have restrictions, use special in-band sequences, or must be human-friendly. When systems chose a base to use, it is not always clear which base to use, as there are many tradeoffs in the decision. Multibase is here to save programs and programmers from worrying about which encoding is best. It solves the biggest problem: a program can use multibase to take input or produce output in whichever base is desired. The important part is that the value is self-describing, letting other programs elsewhere know what encoding it is using.

Use-Cases

The following are some typical use-cases for Multibase:

Abstracting encoding and decoding
Tagging encoded data with the encoding type
IPFS
Processing multiple types of data in the same API, stream, code etc. where the encoding can vary or is not always known
Avoiding heuristics and other inaccurate methods of determining an encoding type
Streamlining an encoding interfaces
- Simplifying patterns where there can be many options such as padding and case

Format

The Format is:

<varint-base-encoding-code><base-encoded-data>

Where <varint-base-encoding-code> is used according to the multibase table. Note that varints (bases above 127) are not yet supported, but planned.

Multibase By Example

Consider the following encodings of the same binary string:

4D756C74696261736520697320617765736F6D6521205C6F2F # base16 (hex)
JV2WY5DJMJQXGZJANFZSAYLXMVZW63LFEEQFY3ZP           # base32
YAjKoNbau5KiqmHPmSxYCvn66dA1vLmwbt                 # base58
TXVsdGliYXNlIGlzIGF3ZXNvbWUhIFxvLw==               # base64

And consider the same encodings with their multibase prefix

F4D756C74696261736520697320617765736F6D6521205C6F2F # base16 F
BJV2WY5DJMJQXGZJANFZSAYLXMVZW63LFEEQFY3ZP           # base32 B
zYAjKoNbau5KiqmHPmSxYCvn66dA1vLmwbt                 # base58 z
MTXVsdGliYXNlIGlzIGF3ZXNvbWUhIFxvLw==               # base64 M

The base prefixes used are: F, B, z, M.

Encodings

The following table lists the currently supported Multibase encodings. Each encoding has an accompanying prefix code. An upper-case code signifies upper-encoding/decoding, and a lower-case code signifies a lower-case encoding/decoding.

encoding	code	name	encoding ids
identity	0x00	8-bit binary (encoder and decoder keeps data unmodified)	`:identity`
base1	1	unary tends to be 11111	`:base1`
base2	0	binary has 1 and 0	`:base2`
base8	7	highest char in octal	`:base8`
base10	9	highest char in decimal	`:base10`
base16	F,f	highest char in hex	`:base16_upper`, `:base16_lower`
base32hex	V,v	rfc4648 no padding - highest char	`:base32_hex_upper`, `:base32_hex_lower`
base32hexpad	T,t	rfc4648 with padding	`:base32_hex_pad_upper`, `:base32_hex_pad_lower`
base32	B,b	rfc4648 no padding	`:base32_upper`, `:base32_lower`
base32pad	C,c	rfc4648 with padding	`:base32_pad_upper`, `:base32_pad_lower`
base32z	h	z-base-32 - used by Tahoe-LAFS - highest letter	`:base32_z`
base58flickr	Z	highest letter	`:base58_flickr`
base58btc	z	highest letter	`:base58_btc`
base64	m	rfc4648 no padding	`:base64`
base64pad	M	rfc4648 with padding - MIME encoding	`:base64_pad`
base64url	u	rfc4648 no padding	`:base64_url`
base64urlpad	U	rfc4648 with padding	`:base64_url_pad`

All encodings above are fully supported at present.

Resources

Link to this section Summary

Types

encoding_family_id()

A family of encoding_id values that are related but differ by options or alphabet typically

encoding_id()

An encoding supported by Multibase

multibase_binary()

A binary encoded and prefixed according to Multibase

prefix()

A Multibase prefix added to encoded binaries

Functions

codec!(data)

Returns the encoding_id used to encode a Multibase encoded binary

codec(data)

Returns the encoding_id used to encode a Multibase encoded binary

codec_decode!(data)

Decodes a Multibase-encoded binary, returning the encoding_id used to encode the Multibase binary

codec_decode(data)

Decodes a Multibase-encoded binary, returning the encoding_id used to encode the Multibase binary

decode!(string)

Decodes a Multibase-encoded binary

decode(string)

Decodes a Multibase-encoded binary

encode!(data, encoding_id)

Encodes a binary using Multibase in the given encoding as specified by encoding_id

encode(data, encoding_id)

Encodes a binary using Multibase in the given encoding as specified by encoding_id

encoded?(data)

Returns true if a binary is probably Multibase encoded

encoding_families()

Returns all encoding families that are available for use with Multibase

encoding_family!(encoding_id)

Returns the encoding family for the given encoding_id

encoding_family(encoding_id)

Returns the encoding family for the given encoding_id

encodings()

Returns all the encodings that are available for use with Multibase

encodings_for!(encoding_family_id)

Returns all the encodings that are available for use with the given encoding_family_id

encodings_for(encoding_family_id)

Returns all the encodings that are available for use with the given encoding_family_id

multibase!(data, encoding_id)

Prefixes an encoded binary with the appropriate multibase prefix for the given encoding as specified by encoding_id

multibase(data, encoding_id)

Prefixes an encoded binary with the appropriate multibase prefix for the given encoding as specified by encoding_id

prefix!(encoding_id)

Returns the prefix that is used with a given encoding_id

prefix(encoding_id)

Returns the prefix that is used with a given encoding_id

Link to this section Types

encoding_family_id()

encoding_family_id() ::
  :identity
  | :base1
  | :base2
  | :base8
  | :base10
  | :base16
  | :base32
  | :base58
  | :base64

A family of encoding_id values that are related but differ by options or alphabet typically.

encoding_id()

encoding_id() ::
  :identity
  | :base1
  | :base2
  | :base8
  | :base10
  | :base16_upper
  | :base16_lower
  | :base32_hex_upper
  | :base32_hex_lower
  | :base32_hex_pad_upper
  | :base32_hex_pad_lower
  | :base32_upper
  | :base32_lower
  | :base32_pad_upper
  | :base32_pad_lower
  | :base32z
  | :base58_flickr
  | :base58_btc
  | :base64
  | :base64_pad
  | :base64_url
  | :base64_url_pad

An encoding supported by Multibase.

multibase_binary()

multibase_binary() :: binary()

A binary encoded and prefixed according to Multibase.

prefix()

prefix() :: <<_::8>>

A Multibase prefix added to encoded binaries.

Link to this section Functions

codec!(data)

codec!(binary()) ::
  {:ok, encoding_id()} | {:error, :missing_encoding | :unsupported_encoding}

Returns the encoding_id used to encode a Multibase encoded binary.

Returns an error if the encoding is not supported or the binary is not a valid multibase encoded binary.

Examples

iex> Multibase.codec!("731672555304674403446254332270145")
:base8

iex> Multibase.codec!("UZ3VtYm8gcmVjaXBl")
:base64_url_pad

iex> Multibase.encode!("gumbo recipe", :base58_btc) |> Multibase.codec!()
:base58_btc

codec(data)

codec(binary()) ::
  {:ok, encoding_id()} | {:error, :missing_encoding | :unsupported_encoding}

Returns the encoding_id used to encode a Multibase encoded binary.

Returns an error if the encoding is not supported or the binary is not a valid multibase encoded binary.

Examples

iex> Multibase.codec!("731672555304674403446254332270145")
:base8

iex> Multibase.codec!("UZ3VtYm8gcmVjaXBl")
:base64_url_pad

iex> Multibase.encode!("gumbo recipe", :base58_btc) |> Multibase.codec!()
:base58_btc

codec_decode!(data)

codec_decode!(binary()) :: {binary(), encoding_id()}

Decodes a Multibase-encoded binary, returning the encoding_id used to encode the Multibase binary.

Raises an ArgumentError if the input is not a valid Multibase binary.

Examples

iex> Multibase.codec_decode!("hqpts13mqcp11n")
{"science!", :base32_z}

iex> Multibase.codec_decode!("VEDHMIPBECDII2")
{"science!", :base32_hex_upper}

iex> Multibase.codec_decode!("vedhmipbecdii2")
{"science!", :base32_hex_lower}

codec_decode(data)

codec_decode(binary()) :: {:ok, {binary(), encoding_id()}} | {:error, term()}

Decodes a Multibase-encoded binary, returning the encoding_id used to encode the Multibase binary.

Raises an ArgumentError if the input is not a valid Multibase binary.

Examples

iex> Multibase.codec_decode("UZnVuZCBpdA==")
{:ok, {"fund it", :base64_url_pad}}

iex> Multibase.codec_decode("uZnVuZCBpdA")
{:ok, {"fund it", :base64_url}}

iex> Multibase.codec_decode("-ZnVuZCBpdA")
:error

decode!(string)

decode!(binary()) :: binary()

Decodes a Multibase-encoded binary.

Raises an ArgumentError if the input is not a valid Multibase binary.

Examples

iex> Multibase.decode!("z42vFWDGstskJy3N74kMHfTKEA")
"I distribute files"

iex> Multibase.decode!("Z42VfvdgSTSKiY3n74KmhEsjea")
"I distribute files"

iex> Multibase.decode!("MZGlzb3JnYW5pemVkIGRlc2sgZGF5")
"disorganized desk day"

iex> Multibase.decode!("731064563336711473026715136462544100621453466544031060571")
"disorganized desk day"

decode(string)

decode(binary()) :: {:ok, binary()} | {:error, term()}

Decodes a Multibase-encoded binary.

Returns an error if the input is not a valid Multibase binary.

Examples

iex> Multibase.decode("BNFZSA5DINFZSA5DINFXGOIDPNY")
{:ok, "is this thing on"}

iex> Multibase.decode("bnfzsa5dinfzsa5dinfxgoidpny")
{:ok, "is this thing on"}

iex> Multibase.decode("-nfzsa5dinfzsa5dinfxgoidpny")
:error

encode!(data, encoding_id)

encode!(binary(), encoding_id()) :: binary()

Encodes a binary using Multibase in the given encoding as specified by encoding_id

Raises an ArgumentError if the encoding type is not supported.

Examples

iex> Multibase.encode!("hello world", :base32_lower)
"bnbswy3dpeb3w64tmmq"

iex> Multibase.encode!("hello world", :base32_pad_lower)
"cnbswy3dpeb3w64tmmq======"

iex>  Multibase.encode!(<<0, "love's got the world in motion">>, :base58_btc)
"z1NjCQeEgij5WQXS5UXtL8yiTLPkGRYohc5MopavNUd"

encode(data, encoding_id)

encode(binary(), encoding_id()) ::
  {:ok, binary()} | {:error, :unsupported_encoding}

Encodes a binary using Multibase in the given encoding as specified by encoding_id.

Returns an {:error, :unsupported_encoding} if the given encoding id is not implemented or supported.

Examples

iex> Multibase.encode("hello world", :base32_lower)
{:ok, "bnbswy3dpeb3w64tmmq"}

iex> Multibase.encode("hello world", :base32_pad_lower)
{:ok, "cnbswy3dpeb3w64tmmq======"}

iex>  Multibase.encode(<<0, "love's got the world in motion">>, :base58_btc)
{:ok, "z1NjCQeEgij5WQXS5UXtL8yiTLPkGRYohc5MopavNUd"}

iex> Multibase.encode("hello world", :nonsense_codec)
{:error, :unsupported_encoding}

encoded?(data)

encoded?(binary()) :: boolean()

Returns true if a binary is probably Multibase encoded.

Note that this check only scans for a valid prefix. A non-encoded binary may have a valid Multibase prefix.

Examples

iex> Multibase.encoded?("f7468652063757265")
true

iex> Multibase.encoded?("~f7468652063757265")
false

iex> Multibase.encoded?(<<>>)
false

encoding_families() (macro)

encoding_families() :: [encoding_family_id()]

Returns all encoding families that are available for use with Multibase.

Examples

iex> Multibase.encoding_families()
[:base1, :base10, :base16, :base2, :base32, :base58, :base64, :base8, :identity]

encoding_family!(encoding_id)

encoding_family!(encoding_id()) ::
  {:ok, encoding_family_id()} | {:error, :unsupported_encoding}

Returns the encoding family for the given encoding_id.

Raises an ArgumentError if the given encoding_id is not supported.

Examples

iex> Multibase.encoding_family!(:base64_url_pad)
:base64

iex> Multibase.encoding_family!(:base8)
:base8

iex>  Multibase.encoding_family!(:base1)
:base1

encoding_family(encoding_id)

encoding_family(encoding_id()) ::
  {:ok, encoding_family_id()} | {:error, :unsupported_encoding}

Returns the encoding family for the given encoding_id.

Returns an error if the given encoding_id is not supported.

Examples

iex> Multibase.encoding_family(:base16_lower)
{:ok, :base16}

iex> Multibase.encoding_family(:base32_pad_upper)
{:ok, :base32}

iex> Multibase.encoding_family(:bbq)
{:error, :unsupported_encoding}

encodings() (macro)

encodings() :: [encoding_id()]

Returns all the encodings that are available for use with Multibase.

Examples

iex> Multibase.encodings()
[:identity, :base1, :base2, :base8, :base10, :base16_upper, :base16_lower,
:base32_hex_upper, :base32_hex_lower, :base32_hex_pad_upper,
:base32_hex_pad_lower, :base32_upper, :base32_lower, :base32_pad_upper,
:base32_pad_lower, :base32_z, :base58_flickr, :base58_btc, :base64,
:base64_pad, :base64_url, :base64_url_pad]

encodings_for!(encoding_family_id)

encodings_for!(encoding_family_id()) ::
  {:ok, [encoding_id()]} | {:error, :unsupported_encoding}

Returns all the encodings that are available for use with the given encoding_family_id.

Useful you auditing the state of Multibase, debugging, or encoding by group.

Raises an ArgumentError if the given encoding_family_id is not supported.

Examples

iex> Multibase.encodings_for!(:base16)
[:base16_upper, :base16_lower, :base32_hex_upper, :base32_hex_lower]

iex>  Multibase.encodings_for!(:base8)
[:base8]

iex> Multibase.encodings_for!(:base1)
[:base1]

encodings_for(encoding_family_id)

encodings_for(encoding_family_id()) ::
  {:ok, [encoding_id()]} | {:error, :unsupported_encoding}

Returns all the encodings that are available for use with the given encoding_family_id.

Useful you auditing the state of Multibase, debugging, or encoding by group.

Returns an error if the given encoding_family_id is not supported.

Examples

iex> Multibase.encodings_for(:base32)
{:ok,
[:base32_hex_pad_upper, :base32_hex_pad_lower, :base32_upper, :base32_lower,
:base32_pad_upper, :base32_pad_lower, :base32_z]}

iex> Multibase.encodings_for(:base64)
{:ok, [:base64, :base64_pad, :base64_url, :base64_url_pad]}

iex> Multibase.encodings_for(:base58)
{:ok, [:base58_flickr, :base58_btc]}

multibase!(data, encoding_id)

multibase!(binary(), encoding_id()) ::
  {:ok, multibase_binary()} | {:error, :unsupported_encoding}

Prefixes an encoded binary with the appropriate multibase prefix for the given encoding as specified by encoding_id.

An ArgumentError is raised if the encoding_id is not supported.

Examples

iex> Base.encode64("high attendance party", padding: false) |> Multibase.multibase!(:base64)
"maGlnaCBhdHRlbmRhbmNlIHBhcnR5"

iex> Base.encode64("high attendance party", padding: true) |> Multibase.multibase!(:base64_pad)
"MaGlnaCBhdHRlbmRhbmNlIHBhcnR5"

iex> Base.encode16("high attendance party", case: :lower) |> Multibase.multibase!(:base16_lower)
"f6869676820617474656e64616e6365207061727479"

multibase(data, encoding_id)

multibase(binary(), encoding_id()) ::
  {:ok, multibase_binary()} | {:error, :unsupported_encoding}

Prefixes an encoded binary with the appropriate multibase prefix for the given encoding as specified by encoding_id.

An error is returned if the encoding_id is not supported.

Examples

iex> Base.encode32("absent forever", case: :lower) |> Multibase.multibase(:base32_lower)
{:ok, "bmfrhgzlooqqgm33smv3gk4q="}

iex> Base.encode32("absent forever", case: :upper) |> Multibase.multibase(:base32_upper)
{:ok, "BMFRHGZLOOQQGM33SMV3GK4Q="}

iex> Base.encode32("absent forever", case: :upper) |> Multibase.multibase(:base32_deepfried)
{:error, :unsupported_encoding}

prefix!(encoding_id)

prefix!(encoding_id()) :: {:ok, prefix()} | {:error, :unsupported_encoding}

Returns the prefix that is used with a given encoding_id

Raises an ArgumentError if the given encoding_id is not supported.

Examples

iex>  Multibase.prefix!(:base32_hex_lower)
"v"

iex> Multibase.prefix!(:base58_flickr)
"Z"

iex> Multibase.prefix!(:base58_btc)
"z"

prefix(encoding_id)

prefix(encoding_id()) :: {:ok, prefix()} | {:error, :unsupported_encoding}

Returns the prefix that is used with a given encoding_id

Returns an error if the given encoding_id is not supported.

Examples

iex>  Multibase.prefix(:base1)
{:ok, "1"}

iex> Multibase.prefix(:base32_hex_pad_upper)
{:ok, "T"}

iex> Multibase.prefix(:base32_hex_pad_lower)
{:ok, "t"}

Multibase.prefix(:quest_for_chips)
{:error, :unsupported_encoding}