Inline NIF support for Zig

Motivation

Zig is a general-purpose programming language designed for robustness, optimality, and maintainability.

The programming philosophy of Zig matches up nicely with the programming philosophy of the BEAM VM and in particular its emphasis on simplicity and structure should very appealing to the practitioners of Elixir.

The following features make Zig extremely amenable to inline language support in a BEAM language:

• simplicity. Zig's syntax is definable in a simple YACC document and Zig takes a stance against making its featureset more complex (though it may evolve somewhat en route to 1.0)
• Composability. Zig is unopinionated about how to go about memory allocations. Its allocator interface is very easily able to be backed by the BEAM's, which means that you have access to generic memory allocation strategies through its composable allocator scheme.
• C integration. It's very easy to design C-interop between Zig and C. Zigler has been designed to make it easier to use Zigler to build C libraries than to use C directly see Easy C.

Guides

Please consult the following guides for detailed topics:

• Using Nifs
• Collection datatypes
• Allocator strategies
• Nif options
• Resources
• C integration
• Concurrency strategies
• Global module options
• Raw calling
• Module callbacks

Zig version support

although the large-scale archictecture of zigler is settled, zigler features may break backwards compatibility until zig reaches 1.0

Nerves Support

Nerves is supported out of the box, and Zigler will be able to seamlessly detect the cross-compilation information (os, architecture, runtime) and build correctly for that target.

Basic NIFs

In the BEAM, you can define a NIF by consulting the following document and implementing the appropriate shared object/DLL callbacks. However, Zigler will take care of all of this for you.

Simply use Zig in your module, providing the otp_app name as an option.

Then, use the sigil_Z/2 macro and write inline zig code. To present a function as a nif in your module, simply export it from your code namespace by making it a pub function in your zig code.

Example

defmodule BasicModule do
  use Zig, otp_app: :zigler

  ~Z"""
  pub fn add_one(number: i64) i64 {
      return number + 1;
  }
  """
end

test "basic module with nif" do
  assert 48 = BasicModule.add_one(47)
end

otp_app setting

You should replace :zigler in the following example with the name of your own app. If no such app exists (e.g. you are using livebook or are in the terminal or escript), you can use :zigler as a fallback.

Zigler will automatically fill out the appropriate NIF C template, compile the shared object, and bind it into the module pre-compilation. In the above example, there will be a BasiceModule.add_one/1 function call created.

Zigler will also make sure that your statically-typed Zig data are guarded when you marshal it from the dynamically-typed BEAM world. However, you may only pass in and return certain types. As an escape hatch, you may use the beam.term type which is a wrapped ERL_NIF_TERM type. See erl_nif.

test "argument error when types are mismatched" do
  assert_raise ArgumentError, fn -> BasicModule.add_one("not a number") end
end

I don't want to use inline Zig

\\ .noinline.zig
pub fn add_one(number: i64) i64 {
    return number + 1;
}

defmodule NoInline do
  use Zig, otp_app: :zigler, zig_code_path: ".noinline.zig"
end

test "non-inline zig" do
  assert 48 = NoInline.add_one(47)
end

Advanced usage: Unsupported erl_nif functions

the beam import does not comprehensively provide support for all functions in erl_nif.h. If you need access to a function in erl_nif.h that isn't provided by zigler, you would do it in the following fashion:

• import erl_nif into your zig code, typically under the e namespace.
• retrieve beam.context.env and use that as your ErlNifEnv pointer.
• use beam.term for function return types, which is a struct with a single field, v, of type ERL_NIF_TERM.

Example

defmodule WithErlNif do
  use Zig, otp_app: :zigler

  ~Z"""
  const e = @import("erl_nif");
  const beam = @import("beam");

  pub fn add_one(number: u64) beam.term {
      return .{.v = e.enif_make_uint64(beam.context.env, number + 1)};
  }
  """
end

test "raw erl_nif_function" do
  assert 48 = WithErlNif.add_one(47)
end

beam.context.env is a threadlocal

beam.context.env is a threadlocal variable, and is not available when calling functions using raw mode. See Raw mode calling for more information.

Advanced usage: Manual marshalling

If you need to marshal your own data, you may use the beam.get and beam.make functions to marshal data to and from the BEAM world.

Example

defmodule ManualMarshalling do
  use Zig, otp_app: :zigler, nifs: [add_one: [spec: false]]

  @spec add_one(integer) :: integer

  ~Z"""
  const beam = @import("beam");

  pub fn add_one(val: beam.term) !beam.term {
      const number = try beam.get(i64, val, .{});
      return beam.make(number + 1, .{});
  }
  """
end

test "manual marshalling" do
  assert 48 = ManualMarshalling.add_one(47)
end

For more details on get and make functions see the beam documentation.

Manual Term marshalling

If you don't use automatic marshalling, Zigler will not be able to provide the following conveniences:

• argument error details. The zig code will raise a generic BEAM ArgumentError but it won't have specific details about what the expected type was and which argument was in error.

• dialyzer type information for your function. You will have to supply that type information outside ~Z block, as shown in the example.

Importing external files

If you need to write zig code outside of the module, just place it in the same directory as your module.

You may either call imported functions from the external file, or forward a function from the external file, either strategy will work correctly.

Example

\\ .extra_code.zig
pub fn add_one(number: u64) u64 {
    return number + 1;
}

defmodule ExternalImport do
  use Zig, otp_app: :zigler

  ~Z"""
  const extra_code = @import(".extra_code.zig");
  pub fn add_one(number: u64) u64 {
      return extra_code.add_one(number);
  }

  pub const forwarded_add_one = extra_code.add_one;
  """
end

test "external imports by calling" do
  assert 48 = ExternalImport.add_one(47)
end

test "external imports by forwarding" do
  assert 48 = ExternalImport.forwarded_add_one(47)
end

Advanced Usage: Custom source location

By default, Zigler places generated source code in the same directory as the module that uses Zigler, however, you may specify a different directory:

defmodule CustomSourceLocation do
  use Zig, otp_app: :zigler, dir: "test/.custom_location"
  
  ~Z"""
  pub fn add_one(number: u64) u64 {
      return number + 1;
  }
  """
end

test "custom_location is built" do
  assert File.dir?("test/custom_location")
  assert File.exists?("test/.custom_location/.Elixir.CustomSourceLocation.zig")
end

Advanced usage: change staging directory location

By default, zigler stages files in /tmp/{modulename} directory. In some cases this will cause user collisions and permissions errors when trying to build modules on multitenant systems. If you need to change the staging directory, set the ZIGLER_STAGING_ROOT environment variable to the desired directory. The recommended staging directory is ~/.cache/zigler. NB: In the future, this may become the default staging directory.