Lexer

A lexer for Lua

final

class Lexer {

Nullable!Token last;

this(string code, Diagnostic[] diagnostics);

string source();

void restore();

void discard();

Token next();

Token peek();

Tuple!(bool, Token) consume(TokenType type, string value);

bool isNext(TokenType type, string value);

bool tryConsume(TokenType type, string value);

}

Constructors

this this(string code, Diagnostic[] diagnostics): Create a new lexer from a piece of code

Members

Functions

consume Tuple!(bool, Token) consume(TokenType type, string value): Attempt to consume the next token. If the token is missing and toConsume is Yes, the following token will be consumed regardless of whether it matches the given pattern. Returns (false, Token) if the token is not of the required type; returns (true, Token) otherwise
discard void discard(): Discard the previously-saved lexer state
eof bool eof(): Check if the next token is EOF
isNext bool isNext(TokenType type, string value): Check if the next token matches the given pattern
next Token next(): Consume the next token and return it
peek Token peek(): Peek the next token and return it, leaving the lexer in the same state as it was before
restore void restore(): Restore the previously-saved lexer state
save void save(): Save the current state of the lexer
source string source(): Get the source code that this Lexer is lexing
tryConsume bool tryConsume(TokenType type, string value): Attempt to consume the next token if it matches the given pattern. Has no effect on the state of the lexer if the token is not found.

Variables

last Nullable!Token last;: The last consumed token

Meta

Source

See Implementation

zua parser lexer aliases

classes

Lexer

enums

TokenType

structs

Token