Building Nerves Systems using nerves_systems repository

This guide provides instructions for building custom Nerves systems using the nerves_systems repository.

The nerves_systems repository offers an alternative way to build Nerves systems, designed for scenarios where the standard mix-based approach may be slower, such as when working extensively with Buildroot or maintaining multiple systems. While this method is faster and more efficient, it requires some setup and familiarity with system configuration and build processes.

By following this guide, you’ll gain the ability to create and customize Nerves systems for your hardware platform, contributing valuable improvements to the Nerves community.

Prerequisites

The nerves_systems build process only works on Linux systems with x86_64 or aarch64 architectures. Non-Linux users must set up a Linux environment, such as a virtual machine (VM) or a container.

General Requirements

• Basic familiarity with the Nerves project and embedded systems development.
• Access to a Linux environment:
  • Native Linux Machine: Best for performance and simplicity.
  • macOS Users: Install a Linux VM (e.g., via UTM) to create an Ubuntu environment.
  • Windows Users: Use WSL2 with an Ubuntu distribution.
• At least 128GB of free disk space: Building Nerves systems can require significant disk space, depending on the components included in the system.

Linux Environment Requirements

Install the following packages in your Linux environment:

Apt based distributions (Ubuntu, Debian...)

sudo apt update && sudo apt install -y git build-essential bc cmake cvs wget curl mercurial python3 python3-aiohttp python3-flake8 python3-ijson python3-nose2 python3-pexpect python3-pip python3-requests rsync subversion unzip gawk jq squashfs-tools libssl-dev automake autoconf libncurses5-dev

Arch linux

sudo pacman -Syu git base-devel bc cmake cvs wget curl mercurial python python-aiohttp flake8 python-ijson python-nose2 python-pexpect python-pip python-requests rsync subversion unzip gawk jq squashfs-tools openssl automake autoconf ncurses

RPM based distributions (Red Hat, Fedora...)

sudo dnf install -y git @development-tools @c-development kernel-devel cvs wget curl mercurial python3 python3-aiohttp python3-flake8 python3-ijson python3-nose2 python3-pexpect python3-pip python3-requests rsync  unzip gawk jq squashfs-tools openssl-devel ncurses-devel

macOS Setup

• Install UTM to set up a Linux VM.
• Follow the Linux Environment Requirements above inside the VM.

Windows Setup

• Install WSL2.
• Set up an Ubuntu distribution and follow the Linux Environment Requirements above within WSL2.

Install Erlang and Elixir

If you've already followed the Nerves Installation guide, Erlang and Elixir should be installed on your system. If not, refer to the installation instructions provided in the guide for your operating system.

Install Nerves Archives

If you've completed the Nerves Installation guide, the Nerves bootstrap archive and local rebar should already be set up. If not, you can install them with:

mix archive.install hex nerves_bootstrap
mix local.rebar

Cloning the nerves_systems Repository

To begin working with Nerves systems, you’ll need to clone the nerves_systems repository from GitHub. This repository contains the necessary scripts and configurations for building and maintaining custom Nerves systems.

  git clone https://github.com/nerves-project/nerves_systems.git
  cd nerves_systems

Step 1: Configuring the Build Environment

Copy the Starter Configuration

To begin configuring the environment for building Nerves systems, you need to create a configuration file. This file specifies which systems to build. Use the provided starter configuration as a template:

cp config/starter-config.exs config/config.exs

The starter-config.exs file includes example configurations for common hardware platforms.

Modify the Configuration File

Open the newly created config/config.exs file in a text editor. Review the listed systems and customize the configuration to include only the systems you want to build. For example:

Download the Necessary Systems

After finalizing the configuration file, use the ns.clone mix task to download the repositories for the specified systems into the src directory. This command automates the cloning process:

mix ns.clone

The directory structure after running the command will look something like this:

src/
    nerves_system_br
    nerves_system_rpi0
    nerves_system_rpi3
    nerves_system_bbb
    ...

Step 2: Building Your Nerves Systems

The nerves_systems repository simplifies building custom systems by automating most of the setup. Follow these steps to build your systems:

1. Start the Build Process

Run the ns.build Mix task to build all systems listed in your configuration file. This task generates Buildroot .config files and compiles the systems.

mix ns.build

2. Check Build Output

Once the build completes, system outputs will be located in the o/ directory. For example:

o/
  rpi0/
  rpi3/
  bbb/

Each directory contains:

• .config: The Buildroot configuration file.
• build/: Compiled binaries and intermediate files.
• nerves.env.sh: Script for setting environment variables.

3. Handle Build Failures

If the ns.build task fails, use the following steps to debug:

1. Locate the Failing System: Navigate to the output directory of the system that failed:

   cd o/<system name>
   

2. Rebuild Manually: Run the Buildroot make process to identify issues:

   make
   

3. Review Logs: Examine error messages or logs for missing dependencies or configuration issues.

4. Retry a Clean Build

If issues persist, clean the system's output directory and rebuild:

rm -rf o/<system name>
mix ns.build

5. (Optional) Preload Build Dependencies

To speed up builds, you can preload dependencies for a system by running:

cd o/<system name>
make source

This downloads all required files in advance, making subsequent builds faster.

Step 3: Using Your Custom Nerves System

After successfully building the Nerves system, you need to set up your environment to use it in your Nerves project. This involves loading environment variables and specifying the target system for your project.

1. Source the Environment Script

   Each built system includes a nerves.env.sh script in the corresponding output directory (e.g., o/rpi0/nerves.env.sh). This script sets the necessary environment variables for your custom-built system.

   Open a new terminal session dedicated to working with your custom-built system, and source the script:

   . ~/path/to/nerves_systems/o/rpi0/nerves.env.sh
   

   Replace rpi0 with the short name of your target system (e.g., rpi3, bbb) and adjust the path as needed.

   Warning

   Each time you start a new terminal session for your Nerves project, you must source the script again to ensure the custom-built system is correctly configured.

2. Set the Target System Nerves uses the MIX_TARGET environment variable to identify the hardware target for your project. Set this variable to the short name of your target system. For example:

   export MIX_TARGET=rpi0
   

3. Build Your Nerves Project Navigate to your Nerves project directory and build it using mix. The environment variables and target settings will ensure that the project uses your custom-built Nerves system:

   mix deps.get
   mix firmware
   

4. Verify the Custom System is in Use Check that your project is using the custom-built system by running:

   mix nerves.info
   

   Look for the output indicating that the system is being sourced from your custom-built location (e.g., o/rpi0).

   Troubleshooting

   • If the custom-built system isn’t being used, double-check that:
     • The nerves.env.sh script was sourced correctly.
     • The MIX_TARGET environment variable matches your intended target system.
   • Verify the o/<system short name> directory contains the required build artifacts.

Step 4: Customizing the Build (Optional)

Customizing your Nerves system is an advanced but powerful way to tailor the system to your needs. For comprehensive details on customizing systems, refer to the official Customizing Your Nerves System document. This guide provides deeper insights into topics such as Buildroot configurations, kernel adjustments, and integrating additional features.

Customizing the build allows you to tailor the Nerves system to meet specific requirements for your hardware or application. This involves modifying Buildroot configurations and applying changes to the Nerves system.

Modify Buildroot Package Configuration

Navigate to the output directory of the system you wish to modify.

cd o/<system short name>

The workflow to customize a Nerves system is the same as when using Buildroot outside of Nerves, using make menuconfig and make savedefconfig.

If you wish to make configuration changes to your system's kernel, then you will need to use make linux-menuconfig

It is also possible to change the busybox configuration using make busybox-menuconfig

When you quit one of these menuconfig interfaces, the changes are stored in your buildroot folder and not reflected in your nerves system directory. For this to happen, you will need to run the appropriate command, depending on what you just modified.

1. After make menuconfig:

   Run make savedefconfig to update the nerves_defconfig file in your System.

2. After make linux-menuconfig:

   Once done with configuring the kernel, you can save the Linux config to the default configuration file using make linux-update-defconfig. The destination file is linux-4.9.defconfig in your project's root (or whatever the kernel version is you're working with).

   NOTE: If your system doesn't contain a custom Linux configuration yet, you'll need to update the Buildroot configuration (using make menuconfig) to point to the new Linux defconfig in your system directory. The path is usually something like $(NERVES_DEFCONFIG_DIR)/linux-x.y_defconfig.

3. After make busybox-menuconfig:

   Unfortunately, there's not currently an easy way to save a BusyBox defconfig. What you have to do instead is save the full BusyBox config and configure it to be included in your nerves_defconfig.

    cp build/busybox-1.27.2/.config ../src/<full system name>/busybox.config
   

   Your Buildroot configuration will need to be updated to point to your busybox custom config. This can be done by typing make menuconfig. Go to Target Packages and under the Busybox line, change the path that is inside Busybox configuration file to use? to the one saved in your nerves system directory ${NERVES_DEFCONFIG_DIR}/busybox.config.

   NOTE: If your system uses Additional BusyBox configuration fragment files option under it needs to be disabled for make busybox-update-config to work.

   Run make busybox-update-config to update busybox.config in your system.

   NOTE: Since this method uses full buysbox configuration file if upstream busybox configuration in nerves_system_br changes it will not apply automatically.

The Buildroot user manual can be very helpful, especially if you need to add a package. The various Nerves system repositories have examples of many common use cases, so check them out as well.

Rebuild the System

To apply your changes, clean the output directory for the system and rebuild:

rm -rf o/<system short name>
mix ns.build

This ensures a fresh build with your updated configuration.

Test the Custom Build

After rebuilding, test the custom firmware on your hardware to ensure it meets your requirements. If issues arise:

• Review the Buildroot logs in o/<system short name>/build/.
• Iterate on the configuration as needed.

Version Control Your Changes

If your customizations are for long-term use, consider committing your changes to version control. This is especially useful for:

• Collaborating with other developers.
• Reproducing builds in the future.

Let's say that you want to version control your customized rpi3 system

cd src
cp -r nerves_system_rpi3 custom_rpi3

The name of the system directory is up to you, but we will call it custom_rpi3 in this example. It's recommended that you check your custom system into your version control system before making changes. This makes it easier to merge in upstream changes from the official systems later. For example, assuming you're using GitHub:

# After creating an empty custom_rpi3 repository in your GitHub account

cd custom_rpi3
git remote rename origin upstream
git remote add origin git@github.com:YourGitHubUserName/custom_rpi3.git
git checkout -b main
git push origin main

Next, tweak the metadata of your Mix project by updating your mix.exs with the following:

• The module name of the mix project at the top of the file
• the value of @app to custom_rpi3
• the value of @github_organization to your GitHub user name or organization

See the Official Mix.Project document for the structure of this file.

# custom_rpi3/mix.exs

# defmodule NervesSystemRpi3.MixProject do
defmodule CustomRpi3.MixProject do
  #      =^^^^^^^^^^= Rename `NervesSystemRpi3` to `CustomRpi3`
  use Mix.Project

  # @github_organization "nerves-project"
  @github_organization "YourGitHubUserOrOrganizationName"
  #                    =^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^=
  #                    Rename `"nerves-project"` here to your GitHub user or organization name

  # @app :nerves_system_rpi3
  @app :custom_rpi3
  #    =^^^^^^^^^^^= Rename `nerves_system_rpi3` here to `custom_rpi3`
end

# =^^^= The rest of this file remains the same

# Commit and push your changes.

git add mix.exs
git commit -m "Change project info"
git push origin main

Now you can go to your nerves_systems/config/config.exs and add it to your systems.

You can also use it in your nerves project as :github dependency now.

# Add system into targets
@alltargets [
  :custom_rpi3,
  :rpi,
  ...
]

# Update the `custom_rpi3` dep in your `deps/0` function.
{:custom_rpi3, github: "YourGitHubUserName/custom_rpi3", runtime: false, targets: :custom_rpi3}

Adding a custom Buildroot Package

If you have a non-Elixir program that's too complicated to compile with elixir_make and not included in Buildroot, you'll need to add instructions for how to build it to your system. This is called a "custom Buildroot package" and the process to add one in a Nerves System is nearly the same as in Buildroot. This is documented in the Adding new package chapter of the Buildroot manual. The main difference with Nerves is the directory.

As you go through this process, please consider whether it makes sense to contributor your package upstream to Buildroot.

A Nerves System will need the following files in the root of the custom system directory src/<nerves_system_name>:

1. Config.in - Includes each package's Config.in file
2. external.mk - Includes each package's <package-name>.mk file
3. packages - Directory containing your custom package directories

Each directory inside the packages directory should contain two things:

1. Config.in - Defines package information
2. <package-name>.mk - Defines how a package is built.

So if you wanted to build a package libfoo, first create the Config.in and external.mk files at the base directory of your system.

/Config.in:

menu "Custom Packages"

source "$NERVES_DEFCONFIG_DIR/packages/libfoo/Config.in"

endmenu

/external.mk:

include $(sort $(wildcard $(NERVES_DEFCONFIG_DIR)/packages/*/*.mk))

Then create the package directory and package files:

mkdir -p packages/libfoo
touch packages/libfoo/Config.in
touch packages/libfoo/libfoo.mk

At this point, you should follow the Official Buildroot documentation for what should be added to these files. Often the easiest route is to find a similar package in Buildroot and copy/paste the contains with appropriate renaming.