Service mesh distribution module.

MeshxNode implements custom carrier protocol for the Erlang distribution as module, designed to work with service mesh adapter implementing Meshx.ServiceMesh behavior.

Standard Erlang distribution depends on two components: Erlang Port Mapper Daemon (EPMD) and distribution module which by default is :inet_tcp_dist. Traffic generated by both components is unencrypted. Distribution module :inet_tcp_dist can be replaced with :inet_tls_dist offering TLS. There is no out-of-the-box solution to secure EPMD.

MeshxNode can be considered as alternative to EPMD and :inet_tcp_dist. MeshxNode by using service mesh provides following benefits:

• automatic traffic encryption with service mesh data plane mTLS,
• "node services" registration and management with external service mesh application control plane.

MeshxNode distribution module cannot be started during Beam VM boot process, iex command line arguments --name and --sname are not supported.

Requirements

MeshxNode depends on service mesh adapter. Meshx is released with MeshxConsul service mesh adapter, required adapter configuration steps are described in package documentation.

Installation

Add :meshx_consul and :meshx_node to application dependencies:

# mix.exs
def deps do
  [
    {:meshx_consul, "~> 0.1.0"},
    {:meshx_node, "~> 0.1.0"}
  ]
end

Usage

Start non-distributed node using MeshxNode as distribution module:

iex -pa _build/dev/lib/meshx_node/ebin --erl "-proto_dist Elixir.MeshxNode" -S mix

Used command arguments:

• -pa - adds specified directory to the code path (dev environment path is used here),
• -proto_dist - specifies custom distribution module (physical file name is: Elixir.MeshxNode_dist.beam).

Transform node into distributed using Node.start/3:

iex(1)> Node.start(:node1@myhost)
{:ok, #PID<0.248.0>}
iex(node1@myhost)2>
# [node1@myhost][stdout]: ==> Consul Connect proxy starting...
# [node1@myhost][stdout]:     Configuration mode: Agent API
# [node1@myhost][stdout]:         Sidecar for ID: node1@myhost
# [node1@myhost][stdout]:     Proxy ID: node1@myhost-sidecar-proxy
# [node1@myhost][stdout]: ==> Log data will now stream in as it occurs:
iex(node1@myhost)3> Node.self()
:node1@myhost

Node.start/3 command starts "node service" using MeshxConsul.start/4. Lines marked with # are stdout output of command running sidecar proxy binary, by default Consul Connect proxy.

Using second terminal, start another node:

iex(1)> Node.start(:node2@myhost)
{:ok, #PID<0.258.0>}

Consul UI screenshot showing both "node services" registered:

Use Consul UI to allow/deny connection between nodes with Consul intentions:

MeshxNode is fully compatible with standard Erlang distribution. After connecting both nodes with Node.connect/1, one can for example spawn new process on remote node using Node.spawn/4. When connecting to other node MeshxConsul.connect/3 function is executed.

Other erl command line options that might be helpful:

• -start_epmd false - do not start EPMD,
• -kernel inet_dist_use_interface {127,0,0,1} - limit Erlang listen interface to loopback,
• -connect_all false - do not maintain a fully connected network of distributed Erlang nodes,
• -no_epmd - specifies that the distributed node does not need epmd at all (OTP24+).

Configuration options

• :mesh_adapter - Required. Specifies service mesh adapter module. Example: mesh_adapter: MeshxConsul.
• :service_params - 2-arity function executed when distribution is started. First function argument is node "short name", second argument is host name. For example: :mynode@myhost translates to (:mynode, 'myhost'). Function should return first argument params passed to c:Meshx.ServiceMesh.start/4, for example: "mynode@myhost". Default: &MeshxNode.Default.service_params/2.
• :service_reg - service registration template passed as second argument to c:Meshx.ServiceMesh.start/4. Default: [].
• :upstream_params - 1-arity function executed when connection between nodes is setup. Function argument is remote node name: running Node.connect(:node1@myhost) will invoke function with (:node1@myhost). Function should return first argument params passed to c:Meshx.ServiceMesh.connect/3, for example: "node1@myhost". Default: &MeshxNode.Default.upstream_params/1.
• :upstream_reg - upstream registration template passed as second argument to c:Meshx.ServiceMesh.connect/3. Default: nil.
• :upstream_proxy - 2-arity function executed when connection between nodes is setup. Function arguments are (:remote_node_name, :local_node_name). Function should return third argument proxy passed to c:Meshx.ServiceMesh.connect/3, for example: {"node1@myhost", "node1@myhost"}. Default: &MeshxNode.Default.upstream_proxy/2.
• :force_registration? - boolean passed as third argument to c:Meshx.ServiceMesh.start/4. Default: false.
• :timeout - timeout value passed as fourth argument to c:Meshx.ServiceMesh.start/4. Default: 5000.

Credits

MeshxNode distribution module is based on example code by Jérôme de Bretagne.