token

Types

PositionedToken

A token with source position information.

pub type PositionedToken {
  PositionedToken(token: Token, line: Int, column: Int)
}

Constructors

PositionedToken(token: Token, line: Int, column: Int)

Token

</>

A token produced by the tokenizer.

pub type Token {
  KeywordExpectations
  KeywordUnmeasured
  KeywordMeasured
  KeywordBy
  KeywordExtends
  KeywordRequires
  KeywordProvides
  KeywordIn
  KeywordX
  KeywordString
  KeywordInteger
  KeywordFloat
  KeywordBoolean
  KeywordList
  KeywordDict
  KeywordOptional
  KeywordDefaulted
  KeywordType
  KeywordURL
  KeywordPercentage
  LiteralString(String)
  LiteralInteger(Int)
  LiteralFloat(Float)
  LiteralPercentage(Float)
  LiteralTrue
  LiteralFalse
  SymbolLeftBrace
  SymbolRightBrace
  SymbolLeftParen
  SymbolRightParen
  SymbolLeftBracket
  SymbolRightBracket
  SymbolColon
  SymbolComma
  SymbolStar
  SymbolPipe
  SymbolEquals
  SymbolDotDot
  WhitespaceNewline
  WhitespaceIndent(Int)
  CommentLine(String)
  CommentSection(String)
  Identifier(String)
  EOF
}

Constructors

```
KeywordExpectations
```
```
KeywordUnmeasured
```
```
KeywordMeasured
```
```
KeywordBy
```
```
KeywordExtends
```
```
KeywordRequires
```
```
KeywordProvides
```
```
KeywordIn
```
```
KeywordX
```
```
KeywordString
```
```
KeywordInteger
```
```
KeywordFloat
```
```
KeywordBoolean
```
```
KeywordList
```
```
KeywordDict
```
```
KeywordOptional
```
```
KeywordDefaulted
```
```
KeywordType
```
```
KeywordURL
```
```
KeywordPercentage
```
```
LiteralString(String)
```
```
LiteralInteger(Int)
```
```
LiteralFloat(Float)
```
```
LiteralPercentage(Float)
```
```
LiteralTrue
```
```
LiteralFalse
```
```
SymbolLeftBrace
```
```
SymbolRightBrace
```
```
SymbolLeftParen
```
```
SymbolRightParen
```
```
SymbolLeftBracket
```
```
SymbolRightBracket
```
```
SymbolColon
```
```
SymbolComma
```
```
SymbolStar
```
```
SymbolPipe
```
```
SymbolEquals
```
```
SymbolDotDot
```
```
WhitespaceNewline
```
```
WhitespaceIndent(Int)
```
```
CommentLine(String)
```
```
CommentSection(String)
```
```
Identifier(String)
```
```
EOF
```

Values

to_string

</>

pub fn to_string(tok: Token) -> String

Convert token to string for error messages.