Markov.TextUtil (markov v4.1.3)
Link to this section Summary
Functions
Strips textual tokens of preceding and trailing non-word characters and downcases them
Link to this section Functions
Link to this function
sanitize_token(tok)
Strips textual tokens of preceding and trailing non-word characters and downcases them
iex> Markov.TextUtil.sanitize_token(:atom)
:atom
iex> Markov.TextUtil.sanitize_token("test")
"test"
iex> Markov.TextUtil.sanitize_token(" !!!???/// tEsT >>>")
"test"