Once is an Ecto type for locally unique 64-bits IDs generated by multiple Elixir nodes. Locally unique IDs make it easier to keep things separated, simplify caching and simplify inserting related things (because you don't have to wait for the database to return the ID). A Once can be generated in multiple ways:

• counter (default): really fast to generate, predictable, works well with b-tree indexes
• encrypted: unique and unpredictable, like a UUIDv4 but shorter
• sortable: time-sortable like a Snowflake ID

A Once can look however you want, and can be stored in multiple ways as well. By default, in Elixir it's a url64-encoded 11-char string, and in the database it's a signed bigint. By using the :ex_format and :db_format options, you can choose both the Elixir and storage format out of format/0. You can pick any combination and use to_format/2 to transform them as you wish!

Because a Once fits into an SQL bigint, they use little space and keep indexes small and fast. Because of their structure they have counter-like data locality, which helps your indexes perform well, unlike UUIDv4s. If you don't care about that and want unpredictable IDs, you can use encrypted IDs that seem random and are still unique.

The actual values are generated by NoNoncense, which performs incredibly well, hitting rates of tens of millions of nonces per second, and it also helps you to safeguard the uniqueness guarantees.

The library has only Ecto and its sibling NoNoncense as dependencies.

Usage

To get going, you need to set up a NoNoncense instance to generate the base unique values. Follow its documentation to do so. Once expects an instance with its own module name by default, like so:

# application.ex (read the NoNoncense docs!)
machine_id = NoNoncense.MachineId.id!(opts)
NoNoncense.init(name: Once, machine_id: machine_id)

In your Ecto schemas, you can then use the type:

schema "things" do
  field :id, Once
end

And that's it!

Options

The Ecto type takes a few optional parameters:

• :no_noncense name of the NoNoncense instance used to generate new IDs (default Once)
• :ex_format what an ID looks like in Elixir, one of format/0 (default :url64). Be sure to read the caveats.
• :db_format what an ID looks like in your database, one of format/0 (default :signed)
• :nonce_type how the nonce is generated, one of nonce_type/0 (default :counter)
• :get_key a zero-arity getter for the 192-bits encryption key, required if encryption is enabled
• :encrypt? deprecated, use nonce_type: :encrypted (default false).

Data formats

There's a drawback to having different data formats for Elixir and SQL: it makes it harder to compare the two. The following are all the same ID:

-1
<<255, 255, 255, 255, 255, 255, 255, 255>>
"__________8"
18_446_744_073_709_551_615
"ffffffffffffffff"

If you use the defaults :url64 as the Elixir format and :signed in your database, you could see "AAAAAACYloA" in Elixir and 10_000_000 in your database. The reasoning behind these defaults is that the encoded format is readable, short, and JSON safe by default, while the signed format means you can use a standard bigint column type.

The negative integers will not cause problems with Postgres and MySQL, they both happily swallow them. Also, negative integers will only start to appear after ~70 years of usage.

If you don't like the formats, it's really easy to change them! The Elixir format especially, which can be changed at any time.

The supported formats are:

• :url64 a url64-encoded string of 11 characters, for example "AAjhfZyAAAE"
• :hex a hex-encoded string of 16 characters, for example "E010831058218A39"
• :raw a bitstring of 64 bits, for example <<0, 8, 225, 125, 156, 128, 0, 2>>
• :signed a signed 64-bits integer, like -12345, between -(2^63) and 2^63-1
• :unsigned an unsigned 64-bits integer, like 67890, between 0 and 2^64-1

Elixir format caveats

Some caveats apply to the :ex_format options.

While JSON does not impose a precision limit on numbers, JavaScript can't deal with >= 2^53 numbers. That means the first 11 nonce bits can't be used, so the first 11 timestamp bits can't be used, which leaves 33 timestamp bits, which will run out after exactly 24 days, so let's say immediately. If you want to use integers, convert them to strings.

That's because we can't disambiguate some binaries that are valid hex, url64 and raw binaries and also valid numeric strings. An example is "12345678901", which is either integer 12_345_678_901 or url64-encoded <<215, 109, 248, 231, 174, 252, 247, 77>> (a.k.a. quite a different number).

By treating all incoming binaries as either a valid numeric string or invalid when using an integer Elixir format, this ambiguity is resolved at the cost of some flexibility. Note that to_format/2 does not support numeric strings, but that does mean it converts reliably between formats once values have been cast/dumped/loaded.

On local uniqueness

By locally unique, we mean unique within your domain or application. UUIDs are globally unique across domains, servers and applications. A Once is not, because 64 bits is not enough to achieve that. It is enough for local uniqueness however: you can generate 8 million IDs per second on 512 machines in parallel for 140 years straight before you run out of bits, by which time your great-grandchildren will deal with the problem. Even higher burst rates are possible and you can use separate NoNoncense instanses for every table if you wish.

Encrypted IDs

By default, IDs are generated using a machine init timestamp, machine ID and counter (although they should be considered to be opague). This means they leak a little information and are somewhat predictable. If you don't like that, you can use encrypted IDs by passing options nonce_type: :encrypted and get_key: fn -> <<_::192>> end. Note that encrypted IDs will cost you the data locality and decrease index performance a little. The encryption algorithm is 3DES and that can't be changed. If you want to know why, take a look at NoNoncense.