jfun.parsec

Interface Tokenizer

public interface Tokenizer extends Serializable

A tokenizer is used by a scanner object to interpret the output of a Scanner.

Author: Ben Yu 2004-11-7

Method Summary
Object	toToken(CharSequence cs, int from, int len) interprets the matched input range to a token object.

Method Detail

public Object toToken(CharSequence cs, int from, int len)

interprets the matched input range to a token object.

Parameters: cs - the input from - the starting point len - the length of the range

Returns: - the token object.