Essence.Vocabulary (essence v0.3.0)
This module exports helpful methods around Vocabularies.
Link to this section Summary
Functions
The freq_dist method calculates the frequency distribution
of tokens in the given text.
The lexical_richness method computes the lexical richness of a given
text.
Return a list of {int, token} pairs, ordered by their token frequency in the given Essence.Document.
Optionally supply a filter function such as Essence.Token.is_word?/1 to exclude unwanted tokens from the calculation.
The vocabulary method computes the vocabulary of a given
Essence.Document. The vocabulary is the unique set of dictionary words in
that text.
Link to this section Functions
freq_dist(d)
The freq_dist method calculates the frequency distribution
of tokens in the given text.
lexical_richness(d)
The lexical_richness method computes the lexical richness of a given
text.
top_tokens(doc, filter_fun \\ &always_true/1)
Return a list of {int, token} pairs, ordered by their token frequency in the given Essence.Document.
Optionally supply a filter function such as Essence.Token.is_word?/1 to exclude unwanted tokens from the calculation.
vocabulary(d)
Specs
vocabulary(any()) :: List.t()
The vocabulary method computes the vocabulary of a given
Essence.Document. The vocabulary is the unique set of dictionary words in
that text.