Electric.StackSupervisor (electric v1.6.2)

Copy Markdown View Source

Root supervisor that starts a stack of processes to serve shapes.

Full supervision tree looks roughly like this:

First, we start 2 registries, Electric.ProcessRegistry, and a registry for shape subscriptions. Both are named using the provided stack_id variable.

  1. Electric.Postgres.Inspector.EtsInspector is started with a pool name as a config option, module that is passed from the base config is ignored

  2. Electric.Connection.Supervisor takes a LOT of options to configure replication and start the rest of the tree. It starts (3) and then (4) in rest-for-one mode

  3. Electric.Connection.Manager takes all the connection/replication options and starts the db pools. It goes through the following steps:

    • start_admin_pool Starts an admin connection pool (3.1) using replication credentials before the lock is acquired. This pool is available for operations like lock breaking, shutdown cleanup, and ETS inspector cache population.
    • start_replication_client This starts a replication client (3.2) with no auto-reconnection, because manager is expected to restart this client in case something goes wrong. The streaming of WAL does not start automatically and has to be started explicitly by the manager. The replication client acquires the advisory lock, creates/verifies the publication and slot.
    • start_snapshot_pool Starts a snapshot connection pool (3.3) using pooled credentials after the replication client is configured. If it's ok, we then do a bunch of checks, then ask (3) to finally start (4), and start streaming.
    1. Postgrex admin pool - small pool (1-4 connections) for metadata/admin operations
    2. Electric.Postgres.ReplicationClient - connects to PG in replication mode, sets up slots, does not start streaming until requested
    3. Postgrex snapshot pool - larger pool for querying initial snapshots & info about the DB
  4. Electric.Shapes.Supervisor is a supervisor responsible for taking the replication log from the replication client and shoving it into storage appropriately. It starts 3 things in one-for-all mode:

    1. Electric.Shapes.DynamicConsumerSupervisor is DynamicSupervisor. It oversees various per-shape processes
      1. Electric.Shapes.Consumer is a consumer subscribing to LogCollector, which acts a shared producer for all shapes. It passes any incoming operation along to the storage.
      2. Electric.Shapes.Consumer.Snapshotter is a temporary GenServer that executes initial snapshot query and writes that to storage
      3. Electric.Shapes.Consumer.Materializer monitors a sub-shape in order to invalidate dependent shapes
    2. Electric.Replication.PublicationManager manages all filters on the publication for the replication
    3. Electric.Replication.ShapeLogCollector collects transactions from the replication connection, fanning them out to Electric.Shapes.Consumer (4.1.1.2)
    4. Electric.ShapeCache coordinates shape creation and handle allocation, shape metadata

Summary

Functions

build_shared_opts(opts)

child_spec(init_arg)

Returns a specification to start this module under a supervisor.

See Supervisor.

dispatch_stack_event(registry \\ Electric.stack_events_registry(), stack_id, event)

opts_schema()

registry_name(stack_id)

start_link(opts)

subscribe_to_shape_events(stack_id, handle, ref \\ make_ref())

subscribe_to_stack_events(registry \\ Electric.stack_events_registry(), stack_id, ref \\ make_ref())