defaultTokenFilter method Null safety

Future<List<Token>> defaultTokenFilter(
  1. List<Token> tokens
)

The default TokenFilter used by TextAnalyzer.

Returns tokens with Token.term stemmed using the Porter2Stemmer.

Implementation

static Future<List<Token>> defaultTokenFilter(List<Token> tokens) async =>
    tokens
        .map((e) => Token(e.term.stemPorter2(), e.termPosition, e.zone))
        .toList();