Overview

Module mnesia_to_khepri

• Description
• Data Types
• Function Index
• Function Details

Description

Tools to migrate from Mnesia to Khepri.

1. the cluster membership
2. the tables content

Both parts can be used independently.

Cluster membership synchronization

For the first part, sync_cluster_membership/0 and sync_cluster_membership/1 ensure the default/specified Khepri store has the same cluster members as Mnesia.

All "instances" of the Khepri store on unclustered nodes will be reset except one. The selected surviving Khepri store is determined using several heuristics which are explained in sync_cluster_membership/1.

Tables copy

For the second part, copy_tables/3, copy_tables/4, copy_all_tables/2 and copy_all_tables/3 take care of copying records to the designated Khepri store.

The functions take a converter module which is responsible for actually writing each record to wherever it wants in the Khepri store. This allows the caller to filter and convert the records. The converter module interface is defined by the mnesia_to_khepri_converter behavior. There is an example converter module called mnesia_to_khepri_example_converter provided.

The copy works while Mnesia tables are still being used and updated. To allow that, the copy follows several stages which are explained in copy_tables/4.

The functions is_migration_finished/1, is_migration_finished/2, wait_for_migration/2 and wait_for_migration/3 can be used to follow an on-going copy of the tables.

Data Types

converter_mod()

converter_mod() = module() | {module(), any()}

A converter module, possibly with a private term to initliaze it.

migration_id()

migration_id() = binary()

MigrationId of a migration.

This is used to semantically identify a migration that covers a set of Mnesia tables and an associated converter module.

mnesia_table()

mnesia_table() = atom()

The name of a Mnesia table.

Function Index

Function Details

sync_cluster_membership/0

Ensures the default Khepri store has the same members as the Mnesia cluster.

See also: sync_cluster_membership/1.

sync_cluster_membership/1

Ensures the Khepri store named StoreId has the same members as the Mnesia cluster.

1. Mnesia is queried to list all its members. We ensure that all Mnesia members are running at this point: if some members are down, there is little chance that Khepri will work on those anyway.
2. All Mnesia members are queried to learn about the membership status of all "instances" of the Khepri store named StoreId.
3. Among all instances of the Khepri store, one instance is selected to be the one that will be joined by other nodes. The selection process is described below.
4. Other nodes are reset, meaning their data is dropped, and they are added to the selected Khepri store.

The synchronization process has to select a store instance that will grow and other store instances that will be reset. The criterias below are evaluated to sort the list of store instances, then the first instance in that list is selected as the winning instance.

Criterias are evaluated in order. For a given criteria, if both sides are equal, the next criteria is evaluated until one criteria gives an instance "greater" than another.

1. A Khepri store instance with more members gets precedence.
2. Then, a Khepri store instance with more tree nodes (more records) gets precedence.
3. Then, a Khepri store instance where a member's Erlang node runs for the longest time gets precedence.
4. Then, a Khepri store instance where its members list sorts before another members list gets precedence.

• A Khepri store instance with 3 members will be selected before another Khepri store instance with 1 member.
• If they have the same number of members, a Khepri store instance with 125 tree nodes will be selected before another Khepri store instance with 34 tree nodes.
• If they have the same number of members and the same number of tree nodes, a Khepri store instance where the oldest Erlang node runs for 67 days will be selected before another Khepri store instance where the oldest Erlang node runs for 3 days.

copy_tables/3

Copies records from Mnesia tables Tables to the default Khepri store.

See also: copy_tables/3.

copy_tables/4

Copies records from Mnesia tables Tables to the Khepri store named StoreId.

The converter module Mod is responsible for storing each Mnesia record in the Khepri store. How it is called is described below. mnesia_to_khepri_example_converter can be used as the default converter module or as an example to write a new one.

1. The PID of the Erlang process working on the copy is stored in the Khepri store. This serves as an indicator that tables are being copied and this prevents concurrent copy from happening.
2. The converter module state is initialized using its Mod:init_copy_to_khepri/3 function, or Mod:init_copy_to_khepri/4 if ModArgs is set.
3. The copy process subscribes to changes from Mnesia.
4. The copy process copies records from Mnesia using its Backup & Restore API. The process uses Mod:copy_to_khepri/3 for each Mnesia record.
5. The copy process marks all involved Mnesia tables as read-only.
6. The copy process consumes all Mnesia events it received during the initial copy and calls Mod:copy_to_khepri/3 and Mod:delete_from_khepri/3 to update the Khepri store accordingly.
7. The copy process calls Mod:finish_copy_to_khepri/1 to let the converter module do any cleanups.
8. The "migration in progress" marker is removed from the Khepri store.

copy_all_tables/2

Copies records from all Mnesia tables to the default Khepri store.

See also: copy_all_tables/3.

copy_all_tables/3

Copies records from all Mnesia tables to the Khepri store named StoreId.

See also: copy_tables/4.

is_migration_finished/1

Returns the migration status of the specified migration identifier.

See also: is_migration_finished/2.

is_migration_finished/2

Returns the migration status of the specified migration identifier.

wait_for_migration/2

Waits for migration MigrationId to be finish.

See also: wait_for_migration/3.

wait_for_migration/3

Waits for migration MigrationId to be finish.

If the migration has not started, it returns false immediately.

If the migration is finished, it returns true immediately.

If the migration is in progress or the status is undefined (see is_migration_finished/3), it waits until the status is known to be "finished" or "not started" or until Timeout milliseconds.

cleanup_after_table_copy/1

Performs any cleanups after a migration has finished.

See also: cleanup_after_table_copy/2.

cleanup_after_table_copy/2

Performs any cleanups after a migration has finished.

Currently this includes the deletion of the copied Mnesia tables.

rollback_table_copy/1

Rolls back a migration.

See also: rollback_table_copy/2.

rollback_table_copy/2

Rolls back a migration.

This function puts Mnesia tables back to read-write mode and deletes the "migration in progress" marker.

handle_fallback/3

Runs MnesiaFun or evaluates KhepriFunOrRet depending on the status of the migration.

See also: handle_fallback/4.

handle_fallback/4

Runs MnesiaFun or evaluates KhepriFunOrRet depending on the status of the migration.

If the migration is finished it executes KhepriFunOrRet if it is a function with an arity of 0 or returns the term directly otherwise.

If the migration is not finished, the function tries to execute MnesiaFun. It should fail by returning or raising {aborted, {no_exists, Table}}. When this happens, the function waits for the migration to finish using wait_for_table_migration/3, then it starts again.

Overview

Generated by EDoc