tokenFilter property Null safety
final
A filter that returns a subset of tokens.
Provide a tokenFilter if you want to manipulate tokens or restrict tokenization to tokens that meet criteria for either index or count.
Implementation
@override
final TokenFilter? tokenFilter;