Interface TokenNoLocation

Token without location tracking.

This is the optimized token format where only field tokens are emitted. The next property indicates what delimiter follows this field.

interface TokenNoLocation {
    delimiter: Delimiter;
    delimiterLength: number;
    value: string;
}

Properties

delimiter: Delimiter

What delimiter follows this field

delimiterLength: number

Length of the delimiter in characters (1 for comma/LF/CR, 2 for CRLF, 0 for EOF)

value: string

The field value