

Understanding OTP releases

Fundamentally, OTP releases are the means by which one or more applications are bundled up together for execution as a single unit. In terms of Elixir projects, an OTP release is the same whether the underlying project is an umbrella or not, in both cases, the application or applications the project defines (and their associated dependencies), are compiled, bundled up together in a single package, and upon startup, loaded and then started in dependency order.

Why releases?

Releases enable simplified deployment: they are self-contained, and provide everything needed to boot the release; they are easily administered via the provided shell script to open up a remote console, start/stop/restart the release, start in the background, send remote commands, and more. In addition, they are archivable artifacts, meaning you can restore an old release from its tarball at any point in the future (barring incompatibilities with the underlying OS or system libraries). The use of releases is also a prerequisite of performing hot upgrades and downgrades, one of the most powerful features of the Erlang VM.

Unlike source code, which requires that you have the build tools, fetch dependencies, and compile a new version (which may differ from the last time it was compiled, due to different tools or dependency resolution resolving newer versions of your deps), OTP releases disconnect development and deployment in such a way that you only need to build the artifact once, and can then deploy it many times; rather than build the artifact each time.

That latter bit may make you think that OTP releases aren’t necessary when building with Docker, but they still have considerable value even there: release packages are stripped down, and so produce smaller images; releases are also configured to be automatically set up for distribution, which means they can be easily configured into a cluster, but more importantly they are ready to be remotely administered via remote shell if needed. When deploying source and running via Mix, you have to take extra steps to make sure the node is properly configured for this.

Now that we’ve looked at what releases are, and why they are worth using, let’s dig in to how they work in more detail. If you are not interested in this right now, and would rather see how to get started with a walkthrough, take a look at the Phoenix Walkthrough.

Startup and lifecycle

Mix itself loads/starts things a little differently than OTP releases, in order to provide Mix.Config, as well as its task infrastructure. For example, when using mix run to start your project, Mix compiles, then loads and starts the :kernel, :stdlib, :compiler, :elixir, and :mix applications, at which point Mix will use the project definition to load and start applications as needed. Before it does this though, it will evaluate the config.exs file in order to ensure the application configuration is set up before your application starts.

OTP releases are very similar, except the runtime itself knows what to load and start, and the order in which to do so, based on something called a boot script. This script instructs the runtime exactly how to boot everything. So in a release, we use a custom instruction to ensure that configuration providers, like Mix.Config have a chance to run before your application is started, but Mix itself is not part of this process. As a result, there are some small differences in how your application will run under Mix and under a release, but for the most part, there should be no noticeable differences.

To give you an example of what a boot script looks like, let’s take a quick look at one generated by Distillery:

%% script generated at {2017,2,22} {12,8,28}
{script,
    {"test","0.1.0"},
    [{preLoaded,
         [erl_prim_loader,erl_tracer,erlang,erts_code_purger,
          erts_dirty_process_code_checker,erts_internal,
          erts_literal_area_collector,init,otp_ring0,prim_eval,prim_file,
          prim_inet,prim_zip,zlib]},
     {progress,preloaded},
     {path,["$ROOT/lib/kernel-5.1.1/ebin","$ROOT/lib/stdlib-3.2/ebin"]},
     {primLoad,
         [error_handler,application,application_controller,application_master,
          code,code_server,erl_eval,erl_lint,erl_parse,error_logger,ets,file,
          filename,file_server,file_io_server,gen,gen_event,gen_server,heart,
          kernel,lists,proc_lib,supervisor]},
     {kernel_load_completed},
     {progress,kernel_load_completed},
     ..snip..
     {path,["$ROOT/lib/test-0.1.0/ebin"]},
     {primLoad,
         ['Elixir.Test,'Elixir.Test.Server', ...]},
     {kernelProcess,heart,{heart,start,[]}},
     {kernelProcess,error_logger,{error_logger,start_link,[]}},
     {kernelProcess,application_controller,
         {application_controller,start,
             [{application,kernel,
                  ..snip.. }]}},
     {progress,init_kernel_started},
     ..snip..
     {apply,
         {application,load,
             [{application,test,
                  [{description,"test"},
                   {vsn,"0.1.0"},
                   {id,[]},
                   {modules,
                       ['Elixir.Test','Elixir.Test.Server',
                        'Elixir.Test.ServerB','Elixir.Test.ServerC',
                        'Elixir.Test.Supervisor']},
                   {registered,[]},
                   {applications,[kernel,stdlib,poison,logger,elixir]},
                   {included_applications,[]},
                   {env,[]},
                   {maxT,infinity},
                   {maxP,infinity},
                   {mod,{'Elixir.Test',[]}}]}]}},
     ..snip..
     {progress,applications_loaded},
     {apply,{application,start_boot,[kernel,permanent]}},
     ..snip..
     {apply,{application,start_boot,[test,permanent]}},
     {apply,{c,erlangrc,[]}},
     {progress,started}]}.

As you can see this script operates at a very low level, instructing the runtime what modules to load, checkpointing major events, using apply instructions to load and start applications, and more that is not shown here for brevity.

Tip

In an OTP release, this script is actually converted to a binary form, which is stored with a .boot extension, but all this file is, is the result of calling :erlang.term_to_binary/1 on the data structure in the .script file. If you ever need to see exactly what is in the .boot, just run :erlang.binary_to_term/1 on the contents to see the data structure itself.

Release descriptor

An OTP release at its most basic level describes the applications and their versions, that it needs to run. The file in which this is described is stored with a .rel extension, and looks like so:

{release,{"test","0.1.0"},
         {erts,"8.2"},
         [{kernel,"5.1.1"},
          {stdlib,"3.2"},
          {poison,"3.1.0"},
          {logger,"1.4.1"},
          {compiler,"7.0.3"},
          {elixir,"1.4.1"},
          {test,"0.1.0"},
          {iex,"1.4.1"},
          {sasl,"3.0.2"}]}.

The second element of the release tuple is another tuple which describes the release name and version. The third element is a tuple which describes the version of ERTS (the Erlang Runtime System) which the release is targeting. The final element is a list of the applications and versions of those applications which are required to run the release.

This release descriptor (as I have come to call it), is also what is used to generate the boot script I showed you in the last section! The OTP module for constructing releases is able to convert this high level description into all of the required low-level instructions needed to boot the described release.

Given the release descriptor, and the boot script, a release is packaged by gathering all of the compiled .beam files required by the applications contained in the release, the target ERTS (if included), and any supporting files - such as config.exs, sys.config, vm.args, as well as the shell script used to set up the environment and run the release - into a gzipped tarball for easy deployment.

Next steps

You may want to take a look at one of the following next:

• See a walkthrough of deploying an application in Walkthrough
• See a walkthrough of deploying a Phoenix application in detail in Phoenix Walkthrough
• Compare releases against other deployment tools you may know in Comparisons
• Check out one of our in-depth deployment guides by looking under Guides in the side bar