This clustering strategy works by loading all endpoints in the current Kubernetes namespace with the configured label. It will fetch the addresses of all endpoints with that label and attempt to connect. It will continually monitor and update its connections every 5s. Alternatively the IP can be looked up from the pods directly by setting kubernetes_ip_lookup_mode to :pods.

In order for your endpoints to be found they should be returned when you run:

kubectl get endpoints -l app=myapp

In order for your pods to be found they should be returned when you run:

kubectl get pods -l app=myapp

It assumes that all nodes share a base name, are using longnames, and are unique based on their FQDN, rather than the base hostname. In other words, in the following longname, <basename>@<domain>, basename would be the value configured in kubernetes_node_basename.

domain would be the value configured in mode and can be either of type :ip (the pod's ip, can be obtained by setting an env variable to status.podIP), :hostname or :dns, which is the pod's internal A Record. This A Record has the format <ip-with-dashes>.<namespace>.pod.cluster.local, e.g. 1-2-3-4.default.pod.cluster.local.

Getting :dns to work requires setting the POD_A_RECORD environment variable before the application starts. If you use Distillery you can set it in your pre_configure hook:

# deployment.yaml
command: ["sh", "-c"]
args: ["POD_A_RECORD"]
args: ["export POD_A_RECORD=$(echo $POD_IP | sed 's/\./-/g') && /app/bin/app foreground"]

# vm.args
-name app@<%= "${POD_A_RECORD}.${NAMESPACE}.pod.cluster.local" %>

If you use mix releases instead, you can configure the required options in rel/env.sh.eex. Doing so will append a -name option to the start command directly and requires no further changes to the vm.args:

# rel/env.sh.eex
export POD_A_RECORD=$(echo $POD_IP | sed 's/\./-/g')
export RELEASE_DISTRIBUTION=name
export RELEASE_NODE=<%= @release.name %>@${POD_A_RECORD}.${NAMESPACE}.pod.cluster.local

To set the NAMESPACE and POD_IP environment variables you can configure your pod as follows:

# deployment.yaml
env:
- name: NAMESPACE
  valueFrom:
    fieldRef:
      fieldPath: metadata.namespace
- name: POD_IP
  valueFrom:
    fieldRef:
      fieldPath: status.podIP

The benefit of using :dns over :ip is that you can establish a remote shell (as well as run observer) by using kubectl port-forward in combination with some entries in /etc/hosts.

Using :hostname is useful when deploying your app to K8S as a stateful set. In this case you can set your erlang name as the fully qualified domain name of the pod which would be something similar to my-app-0.my-service-name.my-namespace.svc.cluster.local. e.g.

# vm.args
-name app@<%=`(hostname -f)`%>

In this case you must also set kubernetes_service_name to the name of the K8S service that is being queried.

mode defaults to :ip.

An example configuration is below:

config :libcluster,
  topologies: [
    k8s_example: [
      strategy: #{__MODULE__},
      config: [
        mode: :ip,
        kubernetes_node_basename: "myapp",
        kubernetes_selector: "app=myapp",
        kubernetes_namespace: "my_namespace",
        polling_interval: 10_000]]]