ExNlp.Filter.Case (ex_nlp v0.1.0)

View Source

Case conversion filtering for tokens.

Converts token text to uppercase or lowercase while preserving metadata.

Summary

Types

token()

A token struct

Functions

lowercase(tokens)

Converts all tokens to lowercase.

uppercase(tokens)

Converts all tokens to uppercase.

Types

token()

@type token() :: ExNlp.Token.t()

A token struct

Functions

lowercase(tokens)

@spec lowercase([token() | String.t()]) :: [token() | String.t()]

Converts all tokens to lowercase.

Examples

iex> tokens = [%ExNlp.Token{text: "The"}, %ExNlp.Token{text: "QUICK"}]
iex> ExNlp.Filter.Case.lowercase(tokens)
[%ExNlp.Token{text: "the"}, %ExNlp.Token{text: "quick"}]

uppercase(tokens)

@spec uppercase([token()]) :: [token()]

Converts all tokens to uppercase.

Examples

iex> tokens = [%ExNlp.Token{text: "hello"}, %ExNlp.Token{text: "world"}]
iex> ExNlp.Filter.Case.uppercase(tokens)
[%ExNlp.Token{text: "HELLO"}, %ExNlp.Token{text: "WORLD"}]