Provides some predefined tokens and Tokenizer implementations.
See: Description
| Class Summary | |
|---|---|
| NamedToken | A token that has a name. |
| Tokenizers | The facade class for all pre-built Tokenizer implementations. |
| TokenQuoted | represents a string that is quoted by a open and close string. |
| Tokens | This class create some basic tokens. |
| TokenType | This represents all pre-built token types. |
| TypedToken<Type> | Represents a token associated with a token type. |