Prosody (Prosody v1.1.0)

Copy Markdown View Source

Prosody is a content analysis library that measures reading flow and cognitive load for mixed text and code content.

The library provides a three-stage processing pipeline:

  1. Parsing: Format-specific parsers convert content into interleaved content blocks
  2. Analysis: Block analyzers summarize each content block using configurable algorithms
  3. Summarization: Results are aggregated into final metrics including reading time

Prosody comes with:

  • Prosody.MDExParser: a markdown parser based on MDEx

  • Prosody.TextParser: a plain text parser

  • Prosody.CodeAnalyzer: a code block analyzer that attempts to apply cognitive load adjustments that aren't captured by simple word counting

  • Prosody.TextAnalyzer: an implementation that emulates word processor counting algorithms based on configuration. There are three short-hand configurations:

    • :balanced: The default algorithm, which splits words in a way that matches human intution. Hyphenated words (fast-paced) and alternating words (and/or) are counted as separate words. Formatted numbers (1,234) are counted as single words. This is similar to what Apple Pages does.

    • :minimal: This splits words on spaces, so that fast-paced and and/or are one word, but and / or is two words. This is most like Microsoft Word or LibreOffice Writer.

    • :maximal: This splits words on space and punctuation, resulting in the highest word count.

    The algorithm results are sometimes surprising, but are consistent:

    Example:balanced:minimal:maximal
    two words222
    and/or212
    and / or222
    fast-paced212
    1,234.56113
    www.example.com113
    bob@example.com113

    A longer result on the sentence:

    The CEO's Q3 buy/sell analysis shows revenue increased 23.8% year-over-year, reaching $4.2M through our e-commerce platform at shop.company.co.uk. Email investors@company.com for the full profit/loss report.

    • :balanced produces 30 words
    • :minimal produces 25 words
    • :maximal produces 37 words

    For details, see Prosody.TextAnalyzer.

Example Usage

content = "# Hello World

This is some text.

IO.puts("Hello")


# Separated pipeline
with {:ok, blocks} <- Prosody.MDExParser.parse(content),
     {:ok, results} <- Prosody.analyze_blocks(blocks),
     {:ok, summary} <-  Prosody.summarize(results) do
  render(:analysis, content: content, summary: summary)
end

# Convenience wrapper
render(:analysis, content: content, summary: Prosody.analyze!(content, parser: :markdown))

Summary

Types

analysis()

Analysis result from processing a content block.

block()

A content block represents a segment of content with type and metadata.

summary()

Final summary of content analysis.

Functions

analyze(content, opts \\ [])

Produces summary/0 output with one function call, returning as {:ok, summary} or {:error, reason}.

analyze!(content, opts \\ [])

Produces summary/0 output with one function call, or raises an error.

analyze_blocks(blocks, opts \\ [])

Analyze content blocks using configured analyzers. Returns {:ok, result} or {:error, reason}.

analyze_blocks!(blocks, opts \\ [])

Analyze content blocks using configured analyzers (bang version). Returns the result or raises an error.

parse(content, opts \\ [])

Parse content blocks from content using parser-specific parsers. Returns {:ok, blocks} or {:error, reason}.

parse!(content, opts \\ [])

Parse content blocks from content using parser-specific parsers. Returns blocks or raises an error.

summarize(analysis, opts \\ [])

Summarize analysis results into final metrics. Returns {:ok, summary} or {:error, reason}.

summarize!(analysis_results, opts \\ [])

Summarize analysis results into final metrics. Returns summary or raises an error.

Types

analysis()

@type analysis() :: %{
  optional(:lines) => nil | non_neg_integer(),
  words: non_neg_integer(),
  reading_words: non_neg_integer(),
  metadata: map()
}

Analysis result from processing a content block.

  • words: Actual word count
  • reading_words: Words adjusted for cognitive load (may differ from words)
  • lines: Optional number of lines (relevant for code blocks)
  • metadata: Analyzer-specific metadata

block()

@type block() :: %{
  type: :text | :code,
  content: String.t(),
  language: nil | String.t(),
  metadata: map()
}

A content block represents a segment of content with type and metadata.

  • type: The content type (:text or :code)
  • content: The actual content string
  • language: Optional language hint (code block language, if available, or an ISO 3166-1 alpha-2 language code)
  • metadata: Parser-specific metadata

summary()

@type summary() :: %{
  words: non_neg_integer(),
  reading_time: nil | non_neg_integer(),
  code: nil | %{words: non_neg_integer(), lines: non_neg_integer()},
  text: nil | %{words: non_neg_integer()},
  metadata: map()
}

Final summary of content analysis.

  • words: Total reading word count (may include cognitive load adjustments), this is reading_words from analysis/0.
  • reading_time: Estimated reading time in minutes
  • code: Code block metrics with words and lines (nil if no code blocks)
  • text: Text block metrics with words (nil if no text blocks)
  • metadata: Summary-specific metadata

Functions

analyze(content, opts \\ [])

@spec analyze(
  String.t(),
  keyword()
) :: {:ok, summary()} | {:error, String.t()}

Produces summary/0 output with one function call, returning as {:ok, summary} or {:error, reason}.

Options

  • parser (default: :text): Parser configuration for parser/2.
  • analyzers: Analyzer configuration for analyze_blocks/2.
  • words_per_minute: Reading speed for summarize/2.
  • min_reading_time: Minimum reading time for summarize/2.

All options are passed to each pipeline step, except for the four noted above.

Examples

# Text parser
{:ok, summary} = Prosody.analyze(content)

# Explicit parser
{:ok, summary} = Prosody.analyze(content, parser: :markdown)

# Full configuration
{:ok, summary} = Prosody.analyze(content,
  parser: {:markdown, strip_frontmatter: false},
  analyzers: [MermaidAnalyzer, :default],
  words_per_minute: 250
)

analyze!(content, opts \\ [])

@spec analyze!(
  String.t(),
  keyword()
) :: summary()

Produces summary/0 output with one function call, or raises an error.

Options

  • parser (default: :text): Parser configuration for parser/2.
  • analyzers: Analyzer configuration for analyze_blocks/2.
  • words_per_minute: Reading speed for summarize/2.
  • min_reading_time: Minimum reading time for summarize/2.

All options are passed to each pipeline step, except for the four noted above.

Examples

# Text parser
summary = Prosody.analyze!(content)

# Explicit parser
summary = Prosody.analyze!(content, parser: :markdown)

# Full configuration
summary = Prosody.analyze!(content,
  parser: {:markdown, strip_frontmatter: false},
  analyzers: [MermaidAnalyzer, :default],
  words_per_minute: 250
)

analyze_blocks(blocks, opts \\ [])

@spec analyze_blocks(
  block() | [block()],
  keyword()
) :: {:ok, [analysis()]} | {:error, String.t()}

Analyze content blocks using configured analyzers. Returns {:ok, result} or {:error, reason}.

Options

  • analyzers: List of analyzers to run over the blocks

analyze_blocks!(blocks, opts \\ [])

@spec analyze_blocks!(
  block() | [block()],
  keyword()
) :: [analysis()]

Analyze content blocks using configured analyzers (bang version). Returns the result or raises an error.

Options

  • analyzers: List of analyzers to run over the blocks

parse(content, opts \\ [])

@spec parse(
  String.t(),
  keyword()
) :: {:ok, [block()]} | {:error, String.t()}

Parse content blocks from content using parser-specific parsers. Returns {:ok, blocks} or {:error, reason}.

Options

  • parser (default :text): Content parser. Must be :markdown, :text, module/0, or {parser, opts}.
  • Other options are passed to the parser unless the parser is provided as {parser, opts}

Examples

{:ok, blocks} = Prosody.parse(content, parser: :markdown)
{:ok, blocks} = Prosody.parse(content, parser: {:markdown, strip_frontmatter: false})
{:ok, blocks} = Prosody.parse(content, parser: {MyCustom.Parser, custom_opt: true})

parse!(content, opts \\ [])

@spec parse!(
  String.t(),
  keyword()
) :: [block()]

Parse content blocks from content using parser-specific parsers. Returns blocks or raises an error.

Options

  • parser (default :text): Content parser. Must be :markdown, :text, module/0, or {parser, opts}.
  • Other options are passed to the parser unless the parser is provided as {parser, opts}

Examples

{:ok, blocks} = Prosody.parse(content, parser: :markdown)
{:ok, blocks} = Prosody.parse(content, parser: {:markdown, strip_frontmatter: false})
{:ok, blocks} = Prosody.parse(content, parser: {MyCustom.Parser, custom_opt: true})

summarize(analysis, opts \\ [])

@spec summarize(
  analysis() | [analysis()],
  keyword()
) :: {:ok, summary()} | {:error, String.t()}

Summarize analysis results into final metrics. Returns {:ok, summary} or {:error, reason}.

Options

  • words_per_minute: Reading speed for time calculation (default: 200)
  • min_reading_time: Minimum reading time in minutes (default: 1)

Examples

{:ok, summary} = Prosody.summarize(analysis, words_per_minute: 250)

summarize!(analysis_results, opts \\ [])

@spec summarize!(
  analysis() | [analysis()],
  keyword()
) :: summary()

Summarize analysis results into final metrics. Returns summary or raises an error.

Options

  • words_per_minute: Reading speed for time calculation (default: 200)
  • min_reading_time: Minimum reading time in minutes (default: 1)

Examples

summary = Prosody.summarize!(analysis, words_per_minute: 250)