essence v0.2.0 Essence.Token

This module contains helpful functions around Tokens

Summary

Calculates the average word length over the given list of tokens

Count the number of syllabels in a token

Determines whether a given token is a polysyllabic, containing more than n syllables, or not

Determines whether a given token is punctuation or not. Returns true if the token is punctuation, false otherwise

Determines whether a given token is a word or not. Returns true if the token is a word, false otherwise

Calculates the length of a token as the number of graphemes in that token

avg_word_length(tokens)

Calculates the average word length over the given list of tokens.

count_syllabels(token)

Count the number of syllabels in a token.

is_polysyllabic?(token, n \\ 1)

Determines whether a given token is a polysyllabic, containing more than n syllables, or not.

is_punctuation?(token)

Determines whether a given token is punctuation or not. Returns true if the token is punctuation, false otherwise.

is_word?(token)

Determines whether a given token is a word or not. Returns true if the token is a word, false otherwise.

token_length(token)

Calculates the length of a token as the number of graphemes in that token.