Class Token

Index

Constructors

Properties

alias? content length type

Methods

Constructors

constructor

new Token(
    type: TokenName,
    content: string | TokenStream,
    alias?: TokenName | TokenName[],
    matchedStr?: string,
): Token
Creates a new token.
Parameters
- type: TokenName
  See Token#type
- content: string | TokenStream
  See Token#content
- Optionalalias: TokenName | TokenName[]
  The alias(es) of the token.
- matchedStr: string = ''
  A copy of the full string this token was created from.
Returns Token
- Defined in core/token.ts:44

Properties

`Optional`alias

alias?: TokenName | TokenName[]

The alias(es) of the token.

See

GrammarToken#alias

content

content: string | TokenStream

The strings or tokens contained by this token.

This will be a token stream if the pattern matched also defined an inside grammar.

`Internal`length

length: number

Length of the full string this token was created from.

Only used internally. The API does not guarantee that this field has any particular value or meaning.

type

type: TokenName

The type of the token.

This is usually the key of a pattern in a Grammar.

See

Methods

addAlias

addAlias(alias: TokenName): void
Adds the given alias to the list of aliases of this token.
Parameters
- alias: TokenName
Returns void
- Defined in core/token.ts:59