Module lexer
Expand description
The lexer module handles the lexical analysis of the source code and then passes the results (a vector of tokens)
to the parser.
Structs§
- Token
- A token produced by lexical analysis.
Enums§
- Float
Literal Suffix - A floating-point literal suffix.
- Integer
Literal Suffix - An integer literal suffix.
- Numeric
Literal Base - The base of an integer or floating-point literal.
- Token
Type - Types of tokens produced by lexical analysis.
Functions§
- lex
- Performs lexical analysis of the source translation unit (preprocessed file) and generates a stream of tokens, and then passes ownership of the token stream to the parser stage.
- lex_
buf_ reader - Performs lexical analysis on the given
BufReader<T>.