mix ash_phoenix_translations.extract (ash_phoenix_translations v1.0.0)

Extracts translatable strings from Ash resources to Gettext POT/PO files.

This task scans your Ash resources for translatable attributes and generates or updates POT (Portable Object Template) and PO (Portable Object) files that can be used with Gettext for professional translation management.

Features

Automatic String Discovery: Scans resources for translatable attributes
POT/PO Generation: Creates standard Gettext files
Multi-Format Support: Generate POT, PO, or both
Domain Filtering: Extract from specific Ash domains
Resource Selection: Choose specific resources to process
Merge Capability: Merge with existing translation files
Verbose Mode: Detailed extraction logging

Basic Usage

# Extract all resources to POT files
mix ash_phoenix_translations.extract

# Extract for specific domain
mix ash_phoenix_translations.extract --domain MyApp.Shop

# Extract with custom output directory
mix ash_phoenix_translations.extract --output priv/gettext

# Generate PO files for specific locales
mix ash_phoenix_translations.extract --locales en,es,fr --format po

# Extract with verbose logging
mix ash_phoenix_translations.extract --verbose

Options

--domain - Extract from a specific Ash domain (e.g., MyApp.Shop)
--resources - Comma-separated list of resource modules
--output - Output directory for files (default: priv/gettext)
--locales - Comma-separated list of locales for PO generation
--merge - Merge with existing POT/PO files
--verbose - Show detailed extraction information
--format - Output format: pot, po, or both (default: pot)

Generated File Structure

POT Format (Translation Templates)

priv/gettext/
└── default.pot           # Contains all translatable strings

PO Format (Locale-Specific)

priv/gettext/
├── en/
│   └── LC_MESSAGES/
│       └── default.po
├── es/
│   └── LC_MESSAGES/
│       └── default.po
└── fr/
    └── LC_MESSAGES/
        └── default.po

Both Formats

priv/gettext/
├── default.pot           # Template
├── en/
│   └── LC_MESSAGES/
│       └── default.po    # English translations
├── es/
│   └── LC_MESSAGES/
│       └── default.po    # Spanish translations
└── fr/
    └── LC_MESSAGES/
        └── default.po    # French translations

POT File Example

Generated POT files follow standard Gettext format:

# Attribute name for MyApp.Product.name
#: MyApp.Product:name
msgid "product.name"
msgstr ""

# Description for MyApp.Product.description
#: MyApp.Product:description
msgid "product.description"
msgstr ""

# Validation message for MyApp.Product
#: MyApp.Product:validation
msgid "must be at least 3 characters"
msgstr ""

PO File Example

Generated PO files include locale-specific headers:

msgid ""
msgstr ""
"Language: es\n"
"MIME-Version: 1.0\n"
"Content-Type: text/plain; charset=UTF-8\n"
"Content-Transfer-Encoding: 8bit\n"

# Attribute name for MyApp.Product.name
#: MyApp.Product:name
msgid "product.name"
msgstr "nombre del producto"

Extraction Scope

The task extracts the following from resources:

Translatable Attributes

translations do
  translatable_attribute :name, locales: [:en, :es]
  translatable_attribute :description, locales: [:en, :es]
end

Generates:

product.name msgid
product.description msgid

Validation Messages

validations do
  validate string_length(:name, min: 3, message: "must be at least 3 characters")
end

Generates:

"must be at least 3 characters" msgid

Action Descriptions

actions do
  create :create do
    description "Creates a new product"
  end
end

Generates:

product.actions.create.description msgid

Workflow Examples

Initial Setup for New Project

# 1. Extract strings to POT template
mix ash_phoenix_translations.extract --verbose

# 2. Generate PO files for your locales
mix ash_phoenix_translations.extract --locales en,es,fr --format both

# 3. Edit PO files manually or send to translators
# priv/gettext/es/LC_MESSAGES/default.po

# 4. Compile translations
mix compile.gettext

Incremental Updates

# 1. Extract new strings and merge with existing
mix ash_phoenix_translations.extract --merge

# 2. Merge into existing PO files
mix gettext.merge priv/gettext

# 3. Find untranslated strings
msggrep --no-wrap -T priv/gettext/es/LC_MESSAGES/default.po | grep 'msgstr ""'

# 4. Translate and recompile
mix compile.gettext

Domain-Specific Extraction

# Extract only shop-related resources
mix ash_phoenix_translations.extract --domain MyApp.Shop

# Extract only catalog resources
mix ash_phoenix_translations.extract       --resources MyApp.Product,MyApp.Category,MyApp.Brand

CAT Tool Integration

# 1. Extract to POT template
mix ash_phoenix_translations.extract --format pot

# 2. Upload priv/gettext/default.pot to CAT tool
# (memoQ, Trados, Smartcat, etc.)

# 3. Download translated PO files from CAT tool

# 4. Place in correct locale directories
# priv/gettext/es/LC_MESSAGES/default.po

# 5. Compile
mix compile.gettext

Integration with Gettext Tools

Standard Gettext Workflow

# After extraction, use standard Gettext commands:

# Merge new strings into existing PO files
mix gettext.merge priv/gettext

# Extract strings from templates (complementary to this task)
mix gettext.extract

# Compile PO to MO files
mix compile.gettext

Tools Compatibility

Generated files are compatible with:

GNU gettext utilities: msgmerge, msgfmt, msginit
CAT tools: memoQ, Trados, Smartcat, OmegaT
Online platforms: Lokalise, Crowdin, POEditor
Editors: Poedit, Virtaal, Lokalize

CI/CD Integration

Automated Extraction on Merge

# .github/workflows/extract-translations.yml
name: Extract Translations

on:
  push:
    branches: [main]
    paths:
      - 'lib/**/resources/**'

jobs:
  extract:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      - uses: erlef/setup-beam@v1
        with:
          elixir-version: '1.17'
          otp-version: '27'

      - name: Install dependencies
        run: mix deps.get

      - name: Extract strings
        run: |
          mix ash_phoenix_translations.extract                 --format both                 --locales en,es,fr                 --merge

      - name: Commit updated POT/PO files
        run: |
          git config user.name "Translation Bot"
          git config user.email "bot@example.com"
          git add priv/gettext/
          git commit -m "Update translation templates" || true
          git push

Pre-commit Hook

# .git/hooks/pre-commit
#!/bin/sh

# Extract translations before each commit
mix ash_phoenix_translations.extract --merge --quiet

# Check if gettext files changed
if git diff --quiet priv/gettext/; then
  echo "No translation changes"
else
  echo "Translation templates updated"
  git add priv/gettext/
fi

Advanced Use Cases

Multi-Domain Extraction

# Extract each domain to separate files
defmodule MyApp.ExtractAll do
  def run do
    domains = [MyApp.Shop, MyApp.Catalog, MyApp.Content]

    Enum.each(domains, fn domain ->
      domain_name =
        domain
        |> Module.split()
        |> List.last()
        |> Macro.underscore()

      output_dir = "priv/gettext/#{domain_name}"

      System.cmd("mix", [
        "ash_phoenix_translations.extract",
        "--domain", inspect(domain),
        "--output", output_dir,
        "--verbose"
      ])
    end)
  end
end

Selective Resource Extraction

# Extract only resources with changes since last extraction
defmodule MyApp.IncrementalExtract do
  def run do
    # Get resources modified in last 24 hours
    modified_resources =
      get_recently_modified_resources()
      |> Enum.map(&inspect/1)
      |> Enum.join(",")

    if modified_resources != "" do
      System.cmd("mix", [
        "ash_phoenix_translations.extract",
        "--resources", modified_resources,
        "--merge"
      ])
    end
  end

  defp get_recently_modified_resources do
    # Implementation to detect modified resource files
    []
  end
end

Translation Coverage Report

# Generate report of extraction coverage
defmodule MyApp.TranslationCoverage do
  def report do
    # Extract to temporary directory
    temp_dir = System.tmp_dir!() <> "/extract_9729"

    {output, 0} = System.cmd("mix", [
      "ash_phoenix_translations.extract",
      "--output", temp_dir,
      "--verbose"
    ])

    # Parse output for statistics
    resources = extract_resource_count(output)
    strings = extract_string_count(output)

    IO.puts("Extraction Coverage Report")
    IO.puts("==========================")
    IO.puts("Resources processed: #{resources}")
    IO.puts("Strings extracted: #{strings}")
    IO.puts("Average strings/resource: #{div(strings, max(resources, 1))}")

    # Cleanup
    File.rm_rf!(temp_dir)
  end

  defp extract_resource_count(output) do
    case Regex.run(~r/Found (+) resources/, output) do
      [_, count] -> String.to_integer(count)
      _ -> 0
    end
  end

  defp extract_string_count(output) do
    case Regex.run(~r/Extracted (+) unique strings/, output) do
      [_, count] -> String.to_integer(count)
      _ -> 0
    end
  end
end

Security Considerations

Atom Safety

The extract task uses String.to_existing_atom/1 when processing format parameter to prevent atom exhaustion:

format =
  case opts[:format] || "pot" do
    format when format in ["pot", "po", "both"] ->
      String.to_existing_atom(format)  # Safe conversion
    invalid ->
      Mix.raise("Invalid format: #{invalid}")
  end

File Security

Directory Validation: Ensures output paths are within project
Safe File Operations: Uses File.mkdir_p!/1 with validated paths
Merge Safety: Preserves existing content when merging

Troubleshooting

No Resources Found

Problem: "No resources found with translations"

Solution:

Verify resources use extensions: [AshPhoenixTranslations]
Check that resources are compiled: mix compile
Use --domain flag to specify domain explicitly
Use --verbose to see which resources are scanned

POT File Empty

Problem: Generated POT file has no msgid entries

Solution:

Ensure resources have translatable_attribute definitions
Check for validation messages and action descriptions
Use --verbose to see extraction process

Format Validation Error

Problem: "Invalid format: xyz"

Solution:

Use only: pot, po, or both for --format
Check for typos in format flag

Merge Conflicts

Problem: Merged POT file loses translations

Solution:

Use proper Gettext merge: mix gettext.merge priv/gettext
Back up PO files before merging
Use version control to track changes

Performance Considerations

Large Codebases

For projects with many resources:

Use domain filtering: Extract one domain at a time
Resource selection: Process specific resources with --resources
Caching: Merge mode is faster for incremental updates

Optimization Tips

# Fast extraction for large projects
mix ash_phoenix_translations.extract       --domain MyApp.Shop       --merge       --output priv/gettext/shop

mix ash_phoenix_translations.install - Initial setup with Gettext backend
mix gettext.extract - Extract strings from templates
mix gettext.merge - Merge POT templates into PO files
mix compile.gettext - Compile PO files to MO format

Examples

Complete Gettext Workflow

# 1. Install with Gettext backend
mix ash_phoenix_translations.install --backend gettext

# 2. Extract resource strings to POT
mix ash_phoenix_translations.extract --verbose

# 3. Generate PO files for locales
mix ash_phoenix_translations.extract       --locales en,es,fr       --format both

# 4. Merge POT into existing PO files
mix gettext.merge priv/gettext

# 5. Edit translations manually
# vim priv/gettext/es/LC_MESSAGES/default.po

# 6. Compile to binary MO files
mix compile.gettext

# 7. Test translations
iex -S mix
iex> Gettext.put_locale(MyAppWeb.Gettext, "es")
iex> Gettext.gettext(MyAppWeb.Gettext, "product.name")

Professional Translation Service Integration

# 1. Extract to POT template
mix ash_phoenix_translations.extract --format pot

# 2. Send priv/gettext/default.pot to translation agency

# 3. Receive translated PO files for each locale

# 4. Place in locale directories
# priv/gettext/es/LC_MESSAGES/default.po
# priv/gettext/fr/LC_MESSAGES/default.po

# 5. Validate and compile
mix compile.gettext

Resource-Specific Extraction

# Extract only product-related resources
mix ash_phoenix_translations.extract       --resources MyApp.Shop.Product,MyApp.Shop.ProductVariant       --output priv/gettext/products       --verbose

Development Workflow

# During development, frequently update POT
mix ash_phoenix_translations.extract --merge

# Check what's new
git diff priv/gettext/default.pot

# Merge into locales
mix gettext.merge priv/gettext

# Compile and test
mix compile.gettext
mix test

mix ash_phoenix_translations.extract (ash_phoenix_translations v1.0.0)

Features

Basic Usage

Options

Generated File Structure

POT Format (Translation Templates)

PO Format (Locale-Specific)

Both Formats

POT File Example

PO File Example

Extraction Scope

Translatable Attributes

Validation Messages

Action Descriptions

Workflow Examples

Initial Setup for New Project

Incremental Updates

Domain-Specific Extraction

CAT Tool Integration

Integration with Gettext Tools

Standard Gettext Workflow

Tools Compatibility

CI/CD Integration

Automated Extraction on Merge

Pre-commit Hook

Advanced Use Cases

Multi-Domain Extraction

Selective Resource Extraction

Translation Coverage Report

Security Considerations

Atom Safety

File Security

Troubleshooting

No Resources Found

POT File Empty

Format Validation Error

Merge Conflicts

Performance Considerations

Large Codebases

Optimization Tips

Related Tasks

Examples

Complete Gettext Workflow

Professional Translation Service Integration

Resource-Specific Extraction

Development Workflow