View Source The Operator

Run in Livebook 
Mix.install([:kino, :bonny])

defmodule MyApp.API.V1.TestResource do
  use Bonny.API.Version
  def manifest(), do: defaults()
end

defmodule MyApp.Controller.GenericController do
  use Bonny.ControllerV2

  step :handle_event
  def handle_event(axn, _), do: axn
end

Logger.configure(level: :info)

About this Livebook

This livebook connects to a kubernetes cluster. Please define here the connection to a cluster you have access to:

{:ok, conn} = K8s.Conn.from_file("~/.kube/config", context: "k3d-bonny-ex")

The Operator

The operator defines custom resources, watch queries and their controllers and serves as the entry point to the watching and handling processes.

Overall, an operator has the following responsibilities:

  • to provide a wrapper for starting and stopping the operator as part of a supervision tree
  • To define the resources to be watched together with the controllers which handle action events on those resources.
  • to define an initial pluggable pipeline containing the step :delegate_to_controller for all action events to pass through
  • To define any custom resources ending up in the manifest generated by mix bonny.gen.manifest
defmodule MyApp.Operator do
  use Bonny.Operator, default_watch_namespace: "default"

  @impl Bonny.Operator
  def crds() do
    [
      %Bonny.API.CRD{
        group: "example.com",
        scope: :Namespaced,
        names: Bonny.API.CRD.kind_to_names("MyCustomResource"),
        versions: [MyApp.API.V1.TestResource]
      }
    ]
  end

  step :delegate_to_controller
  step Bonny.Pluggable.ApplyStatus
  step Bonny.Pluggable.ApplyDescendants

  @impl Bonny.Operator
  def controllers(watching_namespace, _opts) do
    [
      %{
        query:
          K8s.Client.watch("example.com/v1", "MyCustomResource", namespace: watching_namespace),
        controller: MyApp.Controller.GenericController
      },
      %{
        query: K8s.Client.watch("apps/v1", "Deployment", namespace: watching_namespace),
        controller: MyApp.Controller.GenericController
      }
    ]
  end
end

The crds/0 Callback

By implementing the crds/0 callback, you tell Bonny what custom resources your operator defines. It is read only when running mix bonny.gen.manifest in order to generate the operator manifest:

crds =
  [MyApp.Operator]
  |> Bonny.Mix.Operator.crds()
  |> Ymlr.documents!()

IO.puts(crds)

In order to run the operator in this livebook, we have to apply the CRD to the cluster. This step has nothing to do with the operator directly. We just do it in order to run the operator.

crds
|> YamlElixir.read_all_from_string!()
|> Bonny.Resource.apply_async(conn, field_manager: "livebook")
|> Enum.each(fn {_, {:ok, applied_crd}} -> dbg(applied_crd) end)

The controllers/2 Callback

In controllers/2 we define the queries and their event handlers, i.e. controllers. function should return a list where each element of the list is a map with these 2 keys:

  • :query - A list operation of type K8s.Operation.t(). Bonny will watch the cluster with this operation and forward all events to the :controller.
  • :controller - A controller (See the controller guide) or any other Pluggable step. Accepts a module or a {controller :: module(), init_opts :: keyword()} tuple. If a tuple is given, the init_opts are passed to the controller's init/1 function.

If you managed to define a valid conn above, you can now run the operator defined above in this livebook. The code below starts the operator and shows the supervision tree. Note how the operator starts an EventRecorder and two proceses for each controller defined in controllers/2. These two processes are the Watcher and the Reconciler. The Watcher watches for ADD, MODIFY and DELETE events in the cluster. The Reconciler regularly creates :reconcile events for each resource found in the cluster.

{:ok, supervisor} = Supervisor.start_link([{MyApp.Operator, conn: conn}], strategy: :one_for_one)
Kino.Process.render_sup_tree(supervisor)

Adding the Operator to your Supervisor

Once your operator is implemented, you need to add it to your application supervision tree. You can pass :conn and :watch_namespace as init arguments if you like. If you don't pass them, :conn will be retrieved from the callback in your config.exs and :watch_namespace will fall back to the :default_watch_namespace you configured your operator with.

defmodule MyOperator.Application do
  use Application

  def start(_type, env: env) do
    children = [
      {MyOperator.Operator, conn: MyOperator.K8sConn.get!(env), watch_namespace: :all}
    ]

    opts = [strategy: :one_for_one, name: MyOperator.Supervisor]
    Supervisor.start_link(children, opts)
  end
end

Leader Election - Running Multiple Replicas

From https://docs.okd.io/latest/operators/operator_sdk/osdk-leader-election.html:

During the lifecycle of an Operator, it is possible that there may be more than one instance running at any given time, for example when rolling out an upgrade for the Operator. In such a scenario, it is necessary to avoid contention between multiple Operator instances using leader election. This ensures only one leader instance handles the reconciliation while the other instances are inactive but ready to take over when the leader steps down.

In other words, if you want to run multiple replicas of your oprator, you need to turn on leader election.

Leader election is enabled per default. To disable it, you have to pass the enable_leader_election: false option when adding the operator to your Supervisor:

defmodule MyOperator.Application do
  use Application

  def start(_type, env: env) do
    children = [
      {MyOperator.Operator,
       conn: MyOperator.K8sConn.get!(env),
       watch_namespace: :all,
       enable_leader_election: false} # <-- disables the leader elector
    ]

    opts = [strategy: :one_for_one, name: MyOperator.Supervisor]
    Supervisor.start_link(children, opts)
  end
end

Pluggable Pipeline and Steps

The operator implements a Pluggable pipeline, the controller represents one step in this pipeline but can contain sub-steps as well.

Bonny comes with a few steps to your convenience. In most caes it makes sense to add at least Bonny.Pluggable.ApplyDescendants and Bonny.Pluggable.ApplyStatus to the end of your operator pipeline.