Tokenizer typedef Null safety

Tokenizer = Future<Iterable<Token>> Function(SourceText source, [Zone? zone])

Type definition of a function that returns a collection of Token from the source text.

Optional parameter zone is the name of the zone in a document in which the term is located.

Implementation

typedef Tokenizer = Future<Iterable<Token>> Function(SourceText source,
    [Zone? zone]);