Nasty.Semantic.Coreference.Neural.Trainer (Nasty v0.3.0)

View Source

Training pipeline for neural coreference resolution.

Trains mention encoder and pair scorer models end-to-end using binary cross-entropy loss with early stopping on dev set.

Example

# Train models
{:ok, models, history} = Trainer.train(
  training_data,
  dev_data,
  vocab,
  epochs: 20,
  batch_size: 32,
  learning_rate: 0.001
)

# Save trained models
Trainer.save_models(models, "priv/models/en/coref_neural")

Summary

Types

history()
models()
params()
training_data()

Functions

evaluate(models, params, data, vocab)

Evaluate models on dataset.

load_models(base_path)

Load trained models from disk.

save_models(models, params, vocab, base_path)

Save trained models to disk.

train(training_data, dev_data, vocab, opts \\ [])

Train neural coreference models.

Types

history()

@type history() :: %{
  train_loss: [float()],
  train_acc: [float()],
  dev_loss: [float()],
  dev_acc: [float()],
  best_epoch: pos_integer()
}

models()

@type models() :: %{encoder: Axon.t(), scorer: Axon.t()}

params()

@type params() :: %{encoder: map(), scorer: map()}

training_data()

@type training_data() :: [
  {Nasty.AST.Semantic.Mention.t(), Nasty.AST.Semantic.Mention.t(), 0 | 1}
]

Functions

evaluate(models, params, data, vocab)

@spec evaluate(models(), params(), training_data(), map()) :: %{
  loss: float(),
  accuracy: float()
}

Evaluate models on dataset.

Parameters

  • models - Trained models
  • params - Model parameters
  • data - Evaluation data
  • vocab - Vocabulary map

Returns

Map with loss and accuracy

load_models(base_path)

@spec load_models(Path.t()) :: {:ok, models(), params(), map()} | {:error, term()}

Load trained models from disk.

Parameters

  • base_path - Base path without extension

Returns

  • {:ok, models, params, vocab} - Loaded models
  • {:error, reason} - Load error

save_models(models, params, vocab, base_path)

@spec save_models(models(), params(), map(), Path.t()) :: :ok | {:error, term()}

Save trained models to disk.

Parameters

  • models - Models to save
  • params - Model parameters
  • vocab - Vocabulary
  • base_path - Base path without extension

Example

Trainer.save_models(models, params, vocab, "priv/models/en/coref")
# Creates:
#   priv/models/en/coref_encoder.axon
#   priv/models/en/coref_scorer.axon
#   priv/models/en/coref_vocab.etf

train(training_data, dev_data, vocab, opts \\ [])

@spec train(training_data(), training_data(), map(), keyword()) ::
  {:ok, models(), params(), history()} | {:error, term()}

Train neural coreference models.

Parameters

  • training_data - List of {mention1, mention2, label} tuples
  • dev_data - Development set for early stopping
  • vocab - Vocabulary map
  • opts - Training options

Options

  • :epochs - Number of training epochs (default: 20)
  • :batch_size - Batch size (default: 32)
  • :learning_rate - Learning rate (default: 0.001)
  • :hidden_dim - LSTM hidden dimension (default: 128)
  • :dropout - Dropout rate (default: 0.3)
  • :patience - Early stopping patience (default: 3)
  • :clip_norm - Gradient clipping norm (default: 5.0)

Returns

  • {:ok, models, params, history} - Trained models and history
  • {:error, reason} - Training error