Name: filter

Type: command

Syntax: filter [{lines | items | keys | elements} of] <filterSource> {with| without | [not] matching} [wildcard] [pattern] <wildcardPattern> [into <targetContainer>]

Syntax: filter [{lines | items | keys | elements} of] <filterSource> {with| without | [not] matching} regex [pattern] <regexPattern> [into <targetContainer>]

Syntax: filter [{lines | items | keys | elements} of] <filterSource> where <whereExpression> [into <targetContainer>]

Summary:
Filters each line, item, key or element in a source container or expression,
removing the lines, items, keys or elements that do or don't match a pattern.

Introduced: 1.0

OS: mac, windows, linux, ios, android

Platforms: desktop, server, mobile

Example:
local tVar
put the propertyNames into tVar
filter tVar with "[az]*"
-- tVar contains all property names beginning with a or z

Example:
-- Filtering a string literal causes the filtered string to be placed in the it variable
filter items of "apple,banana,cherry" with regex pattern "b.*"
-- it contains "banana"

Example:
local tSource, tBackscriptObjects
put the backscripts into tSource
filter lines of tSource without regex pattern "^stack.*" into tBackscriptObjects
-- tBackscriptObjects contains a list of non-stack backscripts

Example:
create field "Numbers"
put "123,234,345,456,567,678,789" into field "Numbers"
filter items of field "Numbers" matching "*[!35-9]"
-- field contains "234"

Example:
local tArray
put true into tArray["foo"]
put false into tArray["bar"]
filter keys of tArray with "f*"
put the keys of tArray is "foo"

Example:
local tArray
put true into tArray["foo"]
put false into tArray["bar"]
filter elements of tArray without "f*"
put the keys of tArray is "foo"

Example:
create field "Numbers"
put "123,234,345,456,567,678,789" into field "Numbers"
filter items of field "Numbers" where each >= 300 and each <= 500
-- field contains "345,456,567"

Example:
command FilterIt pValue
   filter lines of pValue where checkLine(each)
end FilterIt

function checkLine pLine
   return item 1 of pLine > item 2 of pLine
end checkLine

Parameters:
filterSource:
An expression that evaluates to a string, array, or a container.

wildcardPattern:
An expression used to match certain lines, items, keys or elements.
See description.

regexPattern:
An regular expression used to match certain lines, items, keys or elements.
See [Microsoft's RegEx Quick Reference](https://docs.microsoft.com/en-us/dotnet/standard/base-types/regular-expression-language-quick-reference)
guide for more info on regex usage.

targetContainer:
An expression that evaluates to a container.

whereExpression:
An expression that evaluates to a boolean.

It:
If the filter command is used on a <filterSource> which is not
a <container>, and no <targetContainer> is specified, the filtered string
or array will be placed in the <it> variable.

Description:
Use the <filter> command to pick specific lines, items, keys or elements
in a <container> or <expression>.

The `filter...with` form and the `filter...matching` form retain the
lines, items, keys or elements that contain a match for the specified
<wildcardPattern> or <regexPattern>.

The `filter...without` form and the `filter...not matching` form discard
the lines, items, keys or elements that do not contain a match for the
specified <wildcardPattern> or <regexPattern>.

If you don't specify lines, items, keys or elements then lines are
filtered by default.

>**Note:** If neither `wildcard pattern` or `regex pattern` forms are explicitly
specified then the wildcard pattern form is assumed.

If the `regex pattern` form is specified, the <regexPattern> should
be formatted as a <regular expression>. There are [many websites](https://www.google.com/search?regex)
that provide examples and tutorials of RegEx expressions.

If the 'wildcard pattern' form is to be used, the <wildcardPattern> should
consists of a string of characters to match, which may be combined with any
number of the following special characters:

- `*` : Matches zero or more of any character. The filterPattern `A*C`
    matches "AC", "ABC", or "ADZXC".
- `?` : Matches exactly one character. The filterPattern `A?C` matches
  "ABC", but not "AC" or "ADZXC".
- `[chars]` : Matches any one of the characters inside the brackets. The
  filterPattern `A[BC]D` matches "ABD" or "ACD", but not "AD" or "ABCD".
- `[!chars]` : Matches any character which is not one of the characters
  inside the brackets.
- `[char-char]` : Matches any character whose unicode codepoint is between
  the first character and the second character, such as `[a-y]` any character
  between "a" and "y" but not "z"


You can match instances of special chars as follows:

- `?` with `[?]`
- `*` with `[*]`
- `[` with `[[]`
- `-` with `-`
- `!` with `!`


For example, the <wildcardPattern> `[[]A]*` will match any
string beginning with `[A]`. Broken down, there is `[[]` which equates
to an open square bracket `[` followed by `A]*` as the closing square
bracket is not a special character.

The three bracketed forms can be combined to create more complex
character classes, for example the pattern [!abcA-C] matches any
character which is not a, b or c (upper or lower case)

If no <targetContainer> is specified, and you filter a <container>, 
the <container> contents will be replaced by the filtered result.

When using the where form of the <filter> command the <each> keyword may
be used in the <whereExpression> for the value to be filtered.

Changes:
The <filter>...without form was added in version 2.1.1. In previous
versions, the <filter> command could be used only to retrieve lines that
matched a wildcard expression. The <filter> items... form was added in
version 6.1. In previous versions, the <filter> command could be used
only to retrieve lines. The ability to filter using a regular expression
was added in version 6.1. In previous versions, the <filter> command
only supported wildcard expressions. The ability to filter an expression
was added in version 6.1. In previous versions, the <filter> command
could be used only for a container. The <filter>...[not] matching form
was added in version 6.1 to clarify the pattern handling. The
<filter>...into form was added in version 6.1. In previous versions, the
filter command always replaced the contents of the original container.
Filtering of array keys and elements was added in version 8.1. The where
form of the filter command was added in version 9.5.

References: replace (command), sort (command), matchChunk (function),
matchText (function), replaceText (function), it (keyword),
caseSensitive (property), container (glossary), expression (glossary),
regular expression (glossary), each (keyword)

Tags: text processing
