---
id: statistics
title: Statistics
sidebar_label: Statistics
slug: property-test-statistics.html
---

Sometimes you may like to know the types of values being generated by Kotest, to ensure that your generators are
configured in the way you expect. Property test _statistics_ are designed to fulfil this need.

The function `collect` is the entry point to statistcs and is used to count categories of values.
We use this by invoking it inside a property test with the category we want to increment.

For example, lets say we wanted to gather statistics on the values of `RoundingMode` used by a `BigDecimal`. We would
invoke `checkAll` as normal, passing the rounding mode to the collect function.

```kotlin
checkAll(Arb.enum<RoundingMode>(), Arb.bigDecimal()) { mode, decimal ->
  collect(mode)
  // test here
}
```

Now after the test is completed, Kotest will output the test name, and the counts/percentages for each category:

```
Statistics: [collecting stats] (1000 iterations, 1 args)

HALF_DOWN                                                     142 (14%)
HALF_UP                                                       141 (14%)
CEILING                                                       132 (13%)
FLOOR                                                         122 (12%)
UP                                                            119 (12%)
UNNECESSARY                                                   119 (12%)
HALF_EVEN                                                     118 (12%)
DOWN                                                          107 (11%)
```

The category we use does not have to be an enum. It can be any object, and you can wrap in conditionals if you want more
control. For example:

```kotlin
checkAll(Arb.int()) { k ->
  when {
    k % 2 == 0 -> collect("EVEN")
    else -> collect("ODD")
  }
  // test here
}
```

## Labels

Sometimes you may wish to have orthogonal sets of statistics. For example,
for a simple number test, we might want to confirm that a certain percentage are even numbers, and a certain percentage
are negative. One way would be to have EVEN_POS, EVEN_NEG, ODD_POS, ODD_NEG:

```kotlin
checkAll(Arb.int()) { k ->
  when {
    k > 0 && k % 2 == 0 -> collect("EVEN_POS")
    k % 2 == 0 -> collect("EVEN_NEG")
    k > 0 -> collect("ODD_POS")
    else -> collect("ODD_NEG")
  }
  // test here
}
```

This gives us one set of outputs:

```
EVEN_POS                                                       142 (27%)
EVEN_NEG                                                       141 (23%)
ODD_POS                                                        132 (24%)
ODD_NEG                                                        122 (26%)
```

However, as the combinations grow this will become unwieldy, so Kotest supports labelled statistics. You can think of
this as distinct sets of statistics. To use labels, just pass the label name as the first arg to the collect method.

```kotlin
checkAll(Arb.int()) { k ->
  when {
    k % 2 == 0 -> collect("even_odd", "EVEN")
    else -> collect("even_odd", "ODD")
  }
  when {
    k > 0 -> collect("pos_neg", "POS")
    else -> collect("pos_neg", "NEG")
  }
  // test here
}
```

Now, Kotest will output multiple sets of statistics, with the label name in the title:

```
Statistics: [collecting labelled stats] (1000 iterations, 1 args) [even_odd]

ODD                                                           520 (52%)
EVEN                                                          480 (48%)


Statistics: [collecting labelled stats] (1000 iterations, 1 args) [pos_neg]

NEG                                                           527 (53%)
POS                                                           473 (47%)
```


### Report Mode

By default, statistics are printed for every property test. There are four modes which can be configured using the global configuration object `PropertyTesting`.

The possible options are:

| Mode                                                                | Function                                   |
|---------------------------------------------------------------------|--------------------------------------------|
| PropertyTesting.statisticsReportMode = StatisticsReportMode.OFF     | disable all statistics reporting           |
| PropertyTesting.statisticsReportMode = StatisticsReportMode.OFF     | enables all statistics reporting           |
| PropertyTesting.statisticsReportMode = StatisticsReportMode.SUCCESS | output statistics only on successful tests |
| PropertyTesting.statisticsReportMode = StatisticsReportMode.FAILED  | output statistics only on failed tests     |


### Checking Coverage of Statistics

If you wish to programmatically assert that certain values are being generated, then you can use the specify constraints
that must be met.

For example, in our previous rounding example, we can check that at least 10% of inputs are covering HALF_DOWN, and 10%
are covering FLOOR using `withCoveragePercentages`:

```kotlin
withCoveragePercentages(mapOf(RoundingMode.HALF_DOWN to 10.0, RoundingMode.FLOOR to 10.0)) {
  checkAll(Arb.enum<RoundingMode>(), Arb.bigDecimal()) { mode, decimal ->
    collect(mode)
    // use the mode / decimal
  }
}
```

If we want to check by absolute numbers rather than percentages, we can use `withCoverageCounts`:

```kotlin
withCoverageCounts(mapOf(RoundingMode.HALF_DOWN to 75, RoundingMode.FLOOR to 75)) {
  checkAll(Arb.enum<RoundingMode>(), Arb.bigDecimal()) { mode, decimal ->
    collect(mode)
    // use the mode / decimal
  }
}
```

### Custom Reports

You can customize the report format, or generate reports from the raw data, by using your own instance of `StatisticsReporter`.
This is configured via the global configuration object `PropertyTesting`.

For example:

```
object MyStatisticsReporter : object : StatisticsReporter { ... }
PropertyTesting.statisticsReporter = MyStatisticsReporter
```
