Nasty.Translation.TokenTranslator (Nasty v0.3.0)

Translates individual tokens using bilingual lexicons.

Provides intelligent token-level translation with:

POS-aware selection (nouns, verbs, adjectives, etc.)
Morphological feature preservation (gender, number, tense, person)
Unknown word handling with transliteration fallback
Context-aware translation selection

Usage

alias Nasty.AST.Token
alias Nasty.Translation.TokenTranslator

# Translate a single token
token = %Token{text: "cat", pos: :NOUN, lemma: "cat"}
{:ok, translated} = TokenTranslator.translate_token(token, :en, :es)
# => %Token{text: "gato", pos: :NOUN, lemma: "gato", language: :es}

# Translate a list of tokens
tokens = [%Token{text: "the"}, %Token{text: "cat"}]
{:ok, translated_tokens} = TokenTranslator.translate_tokens(tokens, :en, :es)

Summary

Functions

translate_token(token, source_lang, target_lang)

Translates a single token from source language to target language.

translate_tokens(tokens, source_lang, target_lang)

Translates a list of tokens from source language to target language.

Functions

translate_token(token, source_lang, target_lang)

@spec translate_token(Nasty.AST.Token.t(), atom(), atom()) ::
  {:ok, Nasty.AST.Token.t()} | {:error, term()}

Translates a single token from source language to target language.

Returns {:ok, translated_token} or {:error, reason}.

Examples

iex> token = %Token{text: "cat", pos: :NOUN, lemma: "cat"}
iex> TokenTranslator.translate_token(token, :en, :es)
{:ok, %Token{text: "gato", pos: :NOUN, lemma: "gato", language: :es}}

translate_tokens(tokens, source_lang, target_lang)

@spec translate_tokens([Nasty.AST.Token.t()], atom(), atom()) ::
  {:ok, [Nasty.AST.Token.t()]} | {:error, term()}

Translates a list of tokens from source language to target language.

Returns {:ok, translated_tokens} or {:error, reason}.

Examples

iex> tokens = [%Token{text: "the"}, %Token{text: "cat"}]
iex> TokenTranslator.translate_tokens(tokens, :en, :es)
{:ok, [%Token{text: "el"}, %Token{text: "gato"}]}