The Parle\Lexer class

(PECL parle >= 0.5.1)

Introduction

Single state lexer class. Lexemes can be defined on the fly. If the particular lexer instance is meant to be used with Parle\Parser , the token IDs need to be taken from there. Otherwise, arbitrary token IDs can be supplied. This lexer can give a certain performance advantage over Parle\RLexer , if no multiple states are required. Note, that Parle\RParser is not compatible with this lexer.

Class synopsis

class Parle\Lexer {

/* Constants */

const int ICASE = 1;

const int DOT_NOT_LF = 2;

const int DOT_NOT_CRLF = 4;

const int SKIP_WS = 8;

const int MATCH_ZERO_LEN = 16;

/* Properties */

public bool $bol = false ;

public int $flags = 0;

public int $state = 0;

public int $marker = 0;

public int $cursor = 0;

/* Methods */

public advance (): void

public build (): void

public callout (int $id, callable $callback): void

public consume (string $data): void

public dump (): void

public getToken (): Parle\Token

public insertMacro (string $name, string $regex): void

public push (string $regex, int $id): void

public reset (int $pos): void

}

Predefined Constants

Parle\Lexer::ICASE
Parle\Lexer::DOT_NOT_LF
Parle\Lexer::DOT_NOT_CRLF
Parle\Lexer::SKIP_WS
Parle\Lexer::MATCH_ZERO_LEN

Properties

bol: Start of input flag.
flags: Lexer flags.
state: Current lexer state, readonly.
marker: Position of the latest token match, readonly.
cursor: Current input offset, readonly.

Parle\Lexer::advance — Process next lexer rule
Parle\Lexer::build — Finalize the lexer rule set
Parle\Lexer::callout — Define token callback
Parle\Lexer::consume — Pass the data for processing
Parle\Lexer::dump — Dump the state machine
Parle\Lexer::getToken — Retrieve the current token
Parle\Lexer::insertMacro — Insert regex macro
Parle\Lexer::push — Add a lexer rule
Parle\Lexer::reset — Reset lexer

Found A Problem?

Learn How To Improve This Page • Submit a Pull Request • Report a Bug

+add a note

User Contributed Notes

There are no user contributed notes for this page.