Enum TokenType

pub enum TokenType {
    OpenParen,
    CloseParen,
    OpenBrace,
    CloseBrace,
    OpenSqBracket,
    CloseSqBracket,
    Semicolon,
    Colon,
    Comma,
    BitwiseAnd,
    BitwiseOr,
    BitwiseNot,
    BitwiseXor,
    Plus,
    Minus,
    Multiply,
    Divide,
    Remainder,
    Assignment,
    LogicalNot,
    LessThan,
    GreaterThan,
    Ternary,
    BitwiseAndAssignment,
    BitwiseOrAssignment,
    BitwiseXorAssignment,
    LeftShiftAssignment,
    RightShiftAssignment,
    Increment,
    Decrement,
    AdditionAssignment,
    SubtractionAssignment,
    MultiplyAssignment,
    DivideAssignment,
    RemainderAssignment,
    LeftShift,
    RightShift,
    LogicalAnd,
    LogicalOr,
    EqualTo,
    NotEqualTo,
    LessThanOrEqualTo,
    GreaterThanOrEqualTo,
    Identifier(String),
    CharLiteral {
        literal: String,
        is_multichar: bool,
        value: i32,
    },
    IntegerLiteral {
        literal: String,
        base: NumericLiteralBase,
        suffix: Option<IntegerLiteralSuffix>,
    },
    FloatLiteral {
        literal: String,
        base: NumericLiteralBase,
        suffix: Option<FloatLiteralSuffix>,
    },
    StringLiteral {
        literal: String,
        ascii: Vec<String>,
    },
}

Types of tokens produced by lexical analysis.

Variants

OpenParen

CloseParen

OpenBrace

CloseBrace

OpenSqBracket

CloseSqBracket

Semicolon

Colon

Comma

BitwiseAnd

BitwiseOr

BitwiseNot

BitwiseXor

Plus

Minus

Multiply

Divide

Remainder

Assignment

LogicalNot

LessThan

GreaterThan

Ternary

BitwiseAndAssignment

BitwiseOrAssignment

BitwiseXorAssignment

LeftShiftAssignment

RightShiftAssignment

Increment

Decrement

AdditionAssignment

SubtractionAssignment

MultiplyAssignment

DivideAssignment

RemainderAssignment

LeftShift

RightShift

LogicalAnd

LogicalOr

EqualTo

NotEqualTo

LessThanOrEqualTo

GreaterThanOrEqualTo

Identifier(String)

CharLiteral

Fields

literal: String

is_multichar: bool

value: i32

IntegerLiteral

Fields

literal: String

base: NumericLiteralBase

suffix: Option<IntegerLiteralSuffix>

FloatLiteral

Fields

literal: String

base: NumericLiteralBase

suffix: Option<FloatLiteralSuffix>

StringLiteral

Fields

literal: String

ascii: Vec<String>

Implementations

impl TokenType

pub fn new_identifier(id: &str) -> TokenType

Makes an identifier token type with the given name.

Trait Implementations

impl Clone for TokenType

fn clone(&self) -> TokenType

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for TokenType

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Display for TokenType

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl From<AstAssignmentOp> for TokenType

fn from(op: AstAssignmentOp) -> Self

Converts to this type from the input type.

impl From<AstBinaryOp> for TokenType

fn from(op: AstBinaryOp) -> Self

Converts to this type from the input type.

impl From<AstUnaryOp> for TokenType

fn from(op: AstUnaryOp) -> Self

Converts to this type from the input type.

impl PartialEq for TokenType

fn eq(&self, other: &TokenType) -> bool

Tests for self and other values to be equal, and is used by ==.

fn ne(&self, other: &Rhs) -> bool

Tests for !=. The default implementation is almost always sufficient, and should not be overridden without very good reason.

impl Eq for TokenType

impl StructuralPartialEq for TokenType