Hide navigation sidebar
Hide table of contents sidebar
Back to top
Edit this page
Toggle table of contents sidebar

# Basic String Operations

> This tutorial is based on the paper on stringi that was published in the Journal of Statistical Software; see [2]. To learn more about R, check out Marek’s open-access (free!) textbook Deep R Programming [3].

## Computing Length and Width

First, we shall review the functions related to determining the number of entities in each string.

Let’s consider the following character vector:

    x <- c("spam", "你好", "\u200b\u200b\u200b", NA_character_, "")

The `x` object consists of 5 character strings:

    length(x)
    ## [1] 5

`stri_length()` computes the length of each string. More precisely, the function gives the number of Unicode code points in each string, see Dealing with Unicode Code Points for more details.

    stri_length(x)
    ## [1]  4  2  3 NA  0

The first string carries 4 ASCII (English) letters, the second consists of 2 Chinese characters (U+4F60, U+597D; a greeting), and the third one is comprised of 3 zero-width spaces (U+200B). Note that the 5th element in `x` is an empty string, `""`. Hence, its length is 0. Moreover, there is a missing (`NA`) value at index 4. Therefore, the corresponding length is undefined as well.

When formatting strings for display (e.g., in a report dynamically generated with `Sweave()` or knitr [13]), a string’s width estimate may be more informative – an approximate number of text columns it will occupy when printed using a monospaced font. In particular, many Chinese, Japanese, Korean, and most emoji characters take up two text cells. On the other hand, some code points might be of width 0 (e.g., the said ZERO WIDTH SPACE, U+200B).

    stri_width(x)
    ## [1]  4  4  0 NA  0

## Joining

Below we describe the functions that are related to string concatenation.

### Operator `%s+%`

To join the corresponding strings in two character vectors, we may use the binary `%s+%` operator:

    x <- c("tasty", "delicious", "yummy", NA)
    x %s+% " " %s+% c("spam", "bacon")
    ## [1] "tasty spam"      "delicious bacon" "yummy spam"      NA

### Flattening

The elements in a character vector can be joined (“aggregated”) to form a single string via a call to `stri_flatten()`:

    stri_flatten(stri_omit_na(x), collapse=", ")
    ## [1] "tasty, delicious, yummy"

Note that the token separator, given by the `collapse` argument, defaults to the empty string.

### Generalisation

Both the `%s+%` operator and the `stri_flatten()` function are generalised by `stri_join()` (alias: `stri_paste()`, `stri_c()`):

    stri_join(c("X", "Y", "Z"), 1:6, "a")  # sep="", collapse=NULL
    ## [1] "X1a" "Y2a" "Z3a" "X4a" "Y5a" "Z6a"

By default, the `sep` argument, which controls how corresponding strings are delimited, is set to the empty string (like in the base `paste0()` but unlike in `paste()`). Moreover, `collapse` is `NULL`, which means that the resulting outputs will not be joined to form a single string. This can be changed if need be:

    stri_join(c("X", "Y", "Z"), 1:6, "a", sep="_", collapse=", ")
    ## [1] "X_1_a, Y_2_a, Z_3_a, X_4_a, Y_5_a, Z_6_a"

Note how the two (1st, 3rd) shorter vectors were recycled to match the longest (2nd) vector’s length. The latter was of numeric type, but it was implicitly coerced via a call to `as.character()`.

More examples:

    pythons <- c("Graham Chapman", "John Cleese", "Terry Gilliam",
      "Eric Idle", "Terry Jones", "Michael Palin")
    (pythons <- stri_split_fixed(pythons, " ", simplify=TRUE))
    ##      [,1]      [,2]
    ## [1,] "Graham"  "Chapman"
    ## [2,] "John"    "Cleese"
    ## [3,] "Terry"   "Gilliam"
    ## [4,] "Eric"    "Idle"
    ## [5,] "Terry"   "Jones"
    ## [6,] "Michael" "Palin"
    stri_join(pythons[, 2], pythons[, 1], sep=", ")
    ## [1] "Chapman, Graham" "Cleese, John"    "Gilliam, Terry"  "Idle, Eric"
    ## [5] "Jones, Terry"    "Palin, Michael"
    outer(LETTERS[1:3], 1:5, stri_join, sep=".")  # outer product
    ##      [,1]  [,2]  [,3]  [,4]  [,5]
    ## [1,] "A.1" "A.2" "A.3" "A.4" "A.5"
    ## [2,] "B.1" "B.2" "B.3" "B.4" "B.5"
    ## [3,] "C.1" "C.2" "C.3" "C.4" "C.5"

### Duplicating

To duplicate given strings, we call `stri_dup()` or the `%s*%` operator:

    stri_dup(letters[1:5], 1:5)
    ## [1] "a"     "bb"    "ccc"   "dddd"  "eeeee"

The above is synonymous with `letters[1:5] %s*% 1:5`.

### Within-List Joining

There is also a convenience function that applies `stri_flatten()` on each character vector in a given list:

    words <- list(c("spam", "bacon", "sausage", "spam"), c("eggs", "spam"))
    stri_join_list(words, sep=", ")  # collapse=NULL
    ## [1] "spam, bacon, sausage, spam" "eggs, spam"
    stri_join_list(words, sep=", ", collapse=";\n")
    ## [1] "spam, bacon, sausage, spam;\neggs, spam"

This way, a list of character vectors can be converted to a character vector. Such sequences of variable length sequences of strings are generated by, amongst others, `stri_sub_all()` and `stri_extract_all()`.

## Extracting and Replacing Substrings

The next group of functions deals with the extraction and replacement of particular sequences of code points in given strings.

### Indexing Vectors

To select a subsequence from any R vector, we use the square-bracket operator[1] with an index vector consisting of either non-negative integers, negative integers, or logical values[2].

For example, here is how to select specific elements in a vector:

    x <- c("spam", "buckwheat", "", NA, "bacon")
    x[1:3]                           # from 1st to 3rd string
    ## [1] "spam"      "buckwheat" ""
    x[c(1, length(x))]               # 1st and last
    ## [1] "spam"  "bacon"

Exclusion of elements at specific positions can be performed like:

    x[-1]                            # all but 1st
    ## [1] "buckwheat" ""          NA          "bacon"

Filtering based on a logical vector can be used to extract strings fulfilling desired criteria:

    x[!stri_isempty(x) & !is.na(x)]
    ## [1] "spam"      "buckwheat" "bacon"

### Extracting Substrings

A character vector is, in its very essence, a sequence of sequences of code points. To extract specific substrings from each string in a collection, we can use the `stri_sub()` function.

    y <- "spam, egg, spam, spam, bacon, and spam"
    stri_sub(y, 18)             # from 18th code point to end
    ## [1] "spam, bacon, and spam"
    stri_sub(y, 12, to=15)      # from 12th to 15th code point (inclusive)
    ## [1] "spam"

Negative indices count from the end of a string.

    stri_sub(y, -15, length=5)  # 5 code points from 15th last
    ## [1] "bacon"

### `stri_sub_all()` Function

If some deeper vectorisation level is necessary, `stri_sub_all()` comes in handy. It extracts multiple (possibly different) substrings from all the strings provided:

    (z <- stri_sub_all(
                  c("spam",     "bacon", "sorghum"),
      from   = list(c(1, 3, 4), -3,      c(2, 4)),
      length = list(1,           3,      c(4, 3))))
    ## [[1]]
    ## [1] "s" "a" "m"
    ##
    ## [[2]]
    ## [1] "con"
    ##
    ## [[3]]
    ## [1] "orgh" "ghu"

As the number of substrings to extract from each string might vary, the result is a list of character strings. We have obtained: substrings of length 1 starting at positions 1, 3, and 4 in `x[1]`, then a length-3 substring that starts at the 3rd code point from the end of `x[2]`, and length-4 and -3 substrings starting at, respectively, the 2nd and 4th code point of `x[3]` (where `x` denotes the subsetted vector).

Recall that the strings may all be concatenated by means of the aforementioned `stri_join_list()` function.

    stri_join_list(z, sep=", ")
    ## [1] "s, a, m"   "con"       "orgh, ghu"

There is also a more flexible version of the built-in `simplify2array()` function whose aim is to convert such lists to matrices.

    stri_list2matrix(z, by_row=TRUE, fill="", n_min=5)
    ##      [,1]   [,2]  [,3] [,4] [,5]
    ## [1,] "s"    "a"   "m"  ""   ""
    ## [2,] "con"  ""    ""   ""   ""
    ## [3,] "orgh" "ghu" ""   ""   ""

### “From–To” and “From–Length” Matrices

The second parameter of both `stri_sub()` and `stri_sub_list()` can also be fed with a two-column matrix of the form `cbind(from, to)`. Here, the first column gives the start indices, and the second column defines the end ones. Such matrices are generated, amongst others, by the `stri_locate_()` functions (see below for details).

    (from_to <- cbind(from=c(1, 12, 18), to=c(4, 15, 21))) # +optional labels
    ##      from to
    ## [1,]    1  4
    ## [2,]   12 15
    ## [3,]   18 21
    stri_sub(y, from_to)
    ## [1] "spam" "spam" "spam"

Another example (the recycling rule):

    (from_to <- matrix(1:8, ncol=2, byrow=TRUE))
    ##      [,1] [,2]
    ## [1,]    1    2
    ## [2,]    3    4
    ## [3,]    5    6
    ## [4,]    7    8
    stri_sub(c("abcdefgh", "ijklmnop"), from_to)
    ## [1] "ab" "kl" "ef" "op"

Due to recycling, this has extracted elements at positions 1:2 from the 1st string, at 3:4 from the 2nd one, 5:6 from the 1st, and 7:8 from the 2nd again.

Note the difference between the above output and the following one:

    stri_sub_all(c("abcdefgh", "ijklmnop"), from_to)
    ## [[1]]
    ## [1] "ab" "cd" "ef" "gh"
    ##
    ## [[2]]
    ## [1] "ij" "kl" "mn" "op"

This time, we extract four identical sections from the two inputs.

Moreover, if the second column of the index matrix is named `"length"` (and only if this is the case), i.e., the indexer is of the form `cbind(from, length=length)`, extraction will be based on the extracted chunk size.

### Permuting Code Points

Somewhat related to the above are different ways to construct various permutations (possibly with replacement) of code points in a string:

    stri_join_list(stri_sub_all("spam", c(4, 3, 2, 3, 1), length=1))
    ## [1] "mapas"
    stri_rand_shuffle("bacon")  # random order
    ## [1] "anobc"
    stri_reverse("spam")        # reverse order
    ## [1] "maps"

### Replacing Substrings

`stri_sub_replace()` returns a version of a character vector with some chunks replaced by other strings:

    stri_sub_replace(c("abcde", "ABCDE"),
      from=c(2, 4), length=c(1, 2), replacement=c("X", "uvw"))
    ## [1] "aXcde"  "ABCuvw"

The above replaced “`b`” (the length-1 substring starting at index 2 of the 1st string) with “`X`” and “`DE`” (the length-2 substring at index 4 of the 2nd string) with “`uvw`”.

Similarly, `stri_sub_replace_all()` replaces multiple substrings within each string in a character vector:

    stri_sub_replace_all(
                       c("abcde",  "ABCDE"),
      from        = list(c(2, 4),  c(0,    3,   6)),
      length      = list(  1,      c(0,    2,   0)),
      replacement = list(  "Z",    c("uu", "v", "wwww")))
    ## [1] "aZcZe"      "uuABvEwwww"

Note how we have obtained the insertion of new content at the start and the end of the 2nd input.

### Replacing Substrings In-Place

The corresponding replacement functions modify a character vector in-place:

    y <- "spam, egg, spam, spam, bacon, and spam"
    stri_sub(y, 7, length=3) <- "spam"  # in-place replacement, egg → spam
    print(y)                            # y has changed
    ## [1] "spam, spam, spam, spam, bacon, and spam"

Note that the state of `y` has changed so that the substring of length 3 starting at the 7th code point was replaced by a length-4 content.

Many replacements within a single string are also possible:

    y <- "aa bb cc"
    stri_sub_all(y, c(1, 4, 7), length=2) <- c("A", "BB", "CCC")
    print(y)                            # y has changed
    ## [1] "A BB CCC"

This has replaced 3 length-2 chunks within `y` with new content.

[1]

More precisely, `x[i]` is a syntactic sugar for a call to `‘[‘(x, i)`. Moreover, if `x` is a list, `x[[i]]` can be used to extract its `i`-th element (alias `‘[[‘(x, i)`). Knowing the “functional” form of the operators allows us to, for instance, extract all first elements from each vector in a list by simply calling `sapply(x, "[[", 1)`.

[2]

If an object’s `names` attribute is set, indexing with a character vector is also possible.