tokenFilter property Null safety
A filter that returns a subset of tokens.
Provide a tokenFilter if you want to manipulate tokens or restrict tokenization to tokens that meet criteria for either index or count.
Implementation
TokenFilter? get tokenFilter;