This release includes many PRs from the Hacktoberfest.
We are very grateful to all the contributors who spent their time to help us make BenchmarkDotNet even better!

## Highlights

In this release, we have many improvements in different areas:

* **Diagnosers**
  * EtwProfiler (allows profiling benchmarks on Windows and exporting the data to a trace file)
* **Execution:**
  * Comparing NuGet packages (now it's possible to compare different versions of the same package)
  * .NET Core 3.0 support
  * Deferred Execution Validator
* **Command-line:**
  * `--list`: List of benchmarks
  * `--info`: Print environment info
  * `--runtimes`: Choosing execution runtimes (`--runtimes net472 netcoreapp2.1` will executed a benchmark on .NET 4.7.2 and .NET Core 2.1)
  * Options for number of invocations and iterations
  * Custom default settings for console argument parser
  * Case-insensitive filter
  * Benchmarking with different CoreRun instances
  * Hardware counters command-line support
* **Exporters:**
  * Markdown output for DisassemblyDiagnoser
  * Diff view for disassembler output
  * Improved LINQPad support (colored monospaced logs)
  * Better CPU brand strings
* **Attributes:**
  * Async `[GlobalSetup]` and `[GlobalCleanup]` support
  * Introduced `[ParamsAllValues]`
  * Selecting Baseline across Methods and Jobs
* **Statistics:**
  * Better statistical tests (Welch's t-test and Mann-Whitney U-test)
  * ZeroMeasurementAnalyser
  * RatioColumn
* **Other:**
  * Azure Pipelines support for internal builds
  * Many minor bug fixes
  * Improved documentation
  * New tests

---

## Diagnosers

### EtwProfiler

`EtwProfiler` allows to profile the benchmarked .NET code on Windows and exports the data to a trace file which can be opened with [PerfView](https://github.com/Microsoft/perfview) or [Windows Performance Analyzer](https://learn.microsoft.com/windows-hardware/test/wpt/windows-performance-analyzer).

`EtwProfiler` uses `TraceEvent` library which internally uses Event Tracing for Windows (ETW) to capture stack traces and important .NET Runtime events.
Before the process with benchmarked code is started, EtwProfiler starts User and Kernel ETW sessions. Every session writes data to it's own file and captures different data. User session listens for the .NET Runtime events (GC, JIT etc) while the Kernel session gets CPU stacks and Hardware Counter events. After this, the process with benchmarked code is started. During the benchmark execution all the data is captured and written to a trace file. Moreover, BenchmarkDotNet Engine emits it's own events to be able to differentiate jitting, warmup, pilot and actual workload when analyzing the trace file. When the benchmarking is over, both sessions are closed and the two trace files are merged into one.

![](https://adamsitnik.com/images/etwprofiler/flamegraph.png)

You can find more details
  in the [documentation](xref:docs.etwprofiler) and
  in the [blog post](https://adamsitnik.com/ETW-Profiler/) by Adam Sitnik.

* [#878](https://github.com/dotnet/BenchmarkDotNet/pull/878) EtwProfiler Diagnoser (by [@adamsitnik](https://github.com/adamsitnik))
* [04a715](https://github.com/dotnet/BenchmarkDotNet/commit/04a71586206a822bca56f0abdacefdc2e5fc1b01) EtwProfiler Diagnoser (#878) (by [@adamsitnik](https://github.com/adamsitnik))

---

## Execution

### Comparing NuGet packages

Now it's possible to compare performance of several versions of the same NuGet package.
An example:

```cs
[Config(typeof(Config))]
public class IntroNuGet
{
  // Specify jobs with different versions of the same NuGet package to benchmark.
  // The NuGet versions referenced on these jobs must be greater or equal to the
  // same NuGet version referenced in this benchmark project.
  // Example: This benchmark project references Newtonsoft.Json 9.0.1
  private class Config : ManualConfig
  {
    public Config()
    {
      var baseJob = Job.MediumRun.With(CsProjCoreToolchain.Current.Value);
      Add(baseJob.WithNuGet("Newtonsoft.Json", "11.0.2").WithId("11.0.2"));
      Add(baseJob.WithNuGet("Newtonsoft.Json", "11.0.1").WithId("11.0.1"));
      Add(baseJob.WithNuGet("Newtonsoft.Json", "10.0.3").WithId("10.0.3"));
      Add(baseJob.WithNuGet("Newtonsoft.Json", "10.0.2").WithId("10.0.2"));
      Add(baseJob.WithNuGet("Newtonsoft.Json", "10.0.1").WithId("10.0.1"));
      Add(baseJob.WithNuGet("Newtonsoft.Json", "9.0.1").WithId("9.0.1"));
    }
  }
  
  [Benchmark]
  public void SerializeAnonymousObject()
    => JsonConvert.SerializeObject(
      new { hello = "world", price = 1.99, now = DateTime.UtcNow });
}
```

See also: @BenchmarkDotNet.Samples.IntroNuGet

* [#290](https://github.com/dotnet/BenchmarkDotNet/issues/290) Question: Any official way to benchmark same method between different assembly versions?
* [#931](https://github.com/dotnet/BenchmarkDotNet/issues/931) Same NuGet version used when benchmarking different packages
* [#922](https://github.com/dotnet/BenchmarkDotNet/pull/922) Enables benchmarking betweeen different Nuget packages (by [@Shazwazza](https://github.com/Shazwazza))
* [#932](https://github.com/dotnet/BenchmarkDotNet/pull/932) Partition benchmark run info based on added nuget packages (by [@blairconrad](https://github.com/blairconrad))
* [92a786](https://github.com/dotnet/BenchmarkDotNet/commit/92a7869aaa30aeacaf1da2dcc45bc65c8333ae73) Enables benchmarking betweeen different Nuget packages (#922) fixes #290 (by [@Shazwazza](https://github.com/Shazwazza))
* [510685](https://github.com/dotnet/BenchmarkDotNet/commit/510685f48ce2baf57682aa82e18c6486989e9625) Partition benchmark run info based on added nuget packages (#932) (by [@blairconrad](https://github.com/blairconrad))
* [cf84a4](https://github.com/dotnet/BenchmarkDotNet/commit/cf84a44d108d5bf3860129e0a2a78cace9c95626) NuGet casing fix (by [@AndreyAkinshin](https://github.com/AndreyAkinshin))

### .NET Core 3.0 support

Now it's possible to run benchmarks on preview versions of .NET Core 3.0.

* [#874](https://github.com/dotnet/BenchmarkDotNet/issues/874) .NET Core 3.0 support (assignee: [@adamsitnik](https://github.com/adamsitnik))
* [2e398c](https://github.com/dotnet/BenchmarkDotNet/commit/2e398c89561b3b1c89ec64b94f656ae20236efd1) detect .NET Core 3.0 and use the appropriate target framework moniker, fixes ... (by [@adamsitnik](https://github.com/adamsitnik))

### Deferred Execution Validator

In LINQ, execution of a query is usually [deferred](https://learn.microsoft.com/dotnet/standard/linq/deferred-execution-example) until the moment when you actually request the data. If your benchmark just returns `IEnumerable` or `IQueryable` it's not measuring the execution of the query, just the creation.

This is why we decided to warn you about this issue whenever it happens:

```log
Benchmark IntroDeferredExecution.Wrong returns a deferred execution result (IEnumerable<Int32>). You need to either change the method declaration to return a materialized result or consume it on your own. You can use .Consume() extension method to do that.
```

Don't worry! We are also providing you with a `Consume` extension method which can execute given `IEnumerable` or `IQueryable` and consume its results. All you need to do is to create a [`Consumer`](xref:BenchmarkDotNet.Engines.Consumer) instance, preferably store it in a field (to exclude the cost of creating Consumer from the benchmark itself) and pass it to `Consume` extension method.

**Do not call `.ToArray()` because it's an expensive operation and it might dominate given benchmark!**

See also: @BenchmarkDotNet.Samples.IntroDeferredExecution

* [#858](https://github.com/dotnet/BenchmarkDotNet/issues/858) Should the Engine iterate over and consume IEnumerable and IQueryable results? (assignee: [@adamsitnik](https://github.com/adamsitnik))
* [cebe2a](https://github.com/dotnet/BenchmarkDotNet/commit/cebe2a0f84fa21acb6db9613fe3a4326d635f129) Deferred Execution Validator, fixes #858 (by [@adamsitnik](https://github.com/adamsitnik))

---

## Command-line

In this release, we have tons of improvements for command-line experience.

### `--list`: List of benchmarks

The `--list` allows you to print all of the available benchmark names. Available options are: 

* `flat` - prints list of the available benchmarks: `--list flat`
```ini
BenchmarkDotNet.Samples.Algo_Md5VsSha256.Md5
BenchmarkDotNet.Samples.Algo_Md5VsSha256.Sha256
BenchmarkDotNet.Samples.IntroArguments.Benchmark
BenchmarkDotNet.Samples.IntroArgumentsSource.SingleArgument
BenchmarkDotNet.Samples.IntroArgumentsSource.ManyArguments
BenchmarkDotNet.Samples.IntroArrayParam.ArrayIndexOf
BenchmarkDotNet.Samples.IntroArrayParam.ManualIndexOf
BenchmarkDotNet.Samples.IntroBasic.Sleep
[...]
```
* `tree` - prints tree of the available benchmarks: `--list tree`
```ini
BenchmarkDotNet
 └─Samples
    ├─Algo_Md5VsSha256
    │  ├─Md5
    │  └─Sha256
    ├─IntroArguments
    │  └─Benchmark
    ├─IntroArgumentsSource
    │  ├─SingleArgument
    │  └─ManyArguments
    ├─IntroArrayParam
    │  ├─ArrayIndexOf
    │  └─ManualIndexOf
    ├─IntroBasic
    │  ├─Sleep
[...]
```

The `--list` option works with the `--filter` option. Examples:

* `--list flat --filter *IntroSetupCleanup*` prints:
```ini
BenchmarkDotNet.Samples.IntroSetupCleanupGlobal.Logic
BenchmarkDotNet.Samples.IntroSetupCleanupIteration.Benchmark
BenchmarkDotNet.Samples.IntroSetupCleanupTarget.BenchmarkA
BenchmarkDotNet.Samples.IntroSetupCleanupTarget.BenchmarkB
BenchmarkDotNet.Samples.IntroSetupCleanupTarget.BenchmarkC
BenchmarkDotNet.Samples.IntroSetupCleanupTarget.BenchmarkD
```
* `--list tree --filter *IntroSetupCleanup*` prints:
```ini
BenchmarkDotNet
 └─Samples
    ├─IntroSetupCleanupGlobal
    │  └─Logic
    ├─IntroSetupCleanupIteration
    │  └─Benchmark
    └─IntroSetupCleanupTarget
       ├─BenchmarkA
       ├─BenchmarkB
       ├─BenchmarkC
       └─BenchmarkD
```

* [#905](https://github.com/dotnet/BenchmarkDotNet/issues/905) Implement `--list`
* [#914](https://github.com/dotnet/BenchmarkDotNet/pull/914) Implement `--list` - fixes #905 (by [@wojtpl2](https://github.com/wojtpl2))
* [#916](https://github.com/dotnet/BenchmarkDotNet/pull/916) Update console-args.md - add information about `--list` option (by [@wojtpl2](https://github.com/wojtpl2))
* [330f66](https://github.com/dotnet/BenchmarkDotNet/commit/330f66c3a3d94d1369d5c0b629bbb0085d5db8eb) Implement `--list` - fixes #905 (#914) (by [@wojtpl2](https://github.com/wojtpl2))
* [6c7521](https://github.com/dotnet/BenchmarkDotNet/commit/6c7521d4fd6776098667944321c8a65848382ae5) Update console-args.md - add information about `--list` option (#916) (by [@wojtpl2](https://github.com/wojtpl2))

### `--info`: Print environment info

Some of our users really like the info we print about hardware and OS.
Now we have the `--info` console line argument which does not run the benchmarks, but simply prints the info.

```ini
BenchmarkDotNet=v0.11.1.786-nightly, OS=Windows 10.0.17134.285 (1803/April2018Update/Redstone4)
Intel Xeon CPU E5-1650 v4 3.60GHz, 1 CPU, 12 logical and 6 physical cores
Frequency=3507500 Hz, Resolution=285.1033 ns, Timer=TSC
.NET Core SDK=3.0.100-alpha1-009642
  [Host] : .NET Core 3.0.0-preview1-27004-04 (CoreCLR 4.6.27003.04, CoreFX 4.6.27003.02), 64bit RyuJIT
```
 
* [#904](https://github.com/dotnet/BenchmarkDotNet/issues/904) Implement `--info`
* [#907](https://github.com/dotnet/BenchmarkDotNet/pull/907) fixes #904 Implement `--info` (by [@lahma](https://github.com/lahma))
* [4be28d](https://github.com/dotnet/BenchmarkDotNet/commit/4be28d25fa9ab79ca194c615783148042738bdad) fixes #904 Implement `--info` (#907) (by [@lahma](https://github.com/lahma))

### `--runtimes`: Choosing execution runtimes

The `--runtimes` or just `-r` allows you to run the benchmarks for selected Runtimes. Available options are: Mono, CoreRT, Core, Clr net46, net461, net462, net47, net471, net472, netcoreapp2.0, netcoreapp2.1, netcoreapp2.2, netcoreapp3.0.

Example: run the benchmarks for .NET 4.7.2 and .NET Core 2.1:

```log
dotnet run -c Release -- --runtimes net472 netcoreapp2.1
```

* [#913](https://github.com/dotnet/BenchmarkDotNet/pull/913) .NET Core Toolchains improvements (by [@adamsitnik](https://github.com/adamsitnik))
* [0f721c](https://github.com/dotnet/BenchmarkDotNet/commit/0f721c8e0e100fc951a54b6045eb7b58c55c2a1f) make it possible to specify runtimes using explicit tfms like net472 or netco... (by [@adamsitnik](https://github.com/adamsitnik))
* [1c581e](https://github.com/dotnet/BenchmarkDotNet/commit/1c581e5bf5b4ba9f40d113ae09e0731a60523a60) .NET Core Toolchains improvements (#913) (by [@adamsitnik](https://github.com/adamsitnik))

### Options for number of invocations and iterations

* `--launchCount` - how many times we should launch process with target benchmark. The default is 1.
* `--warmupCount` - how many warmup iterations should be performed. If you set it, the minWarmupCount and maxWarmupCount are ignored. By default calculated by the heuristic.
* `--minWarmupCount` - minimum count of warmup iterations that should be performed. The default is 6.
* `--maxWarmupCount` - maximum count of warmup iterations that should be performed. The default is 50.
* `--iterationTime` - desired time of execution of an iteration. Used by Pilot stage to estimate the number of invocations per iteration. 500ms by default.
* `--iterationCount` - how many target iterations should be performed. By default calculated by the heuristic.
* `--minIterationCount` - minimum number of iterations to run. The default is 15.
* `--maxIterationCount` - maximum number of iterations to run. The default is 100.
* `--invocationCount` - invocation count in a single iteration. By default calculated by the heuristic.
* `--unrollFactor` - how many times the benchmark method will be invoked per one iteration of a generated loop. 16 by default
* `--runOncePerIteration` - run the benchmark exactly once per iteration. False by default.

Example: run single warmup iteration, from 9 to 12 actual workload iterations.

```log
dotnet run -c Release -- --warmupCount 1 --minIterationCount 9 --maxIterationCount 12
```

* [#902](https://github.com/dotnet/BenchmarkDotNet/pull/902) More command line args (by [@adamsitnik](https://github.com/adamsitnik))
* [ba0d22](https://github.com/dotnet/BenchmarkDotNet/commit/ba0d22b41fd25022e3a945fe5ef1ae8aea697cf7) allow to configure the number of invocations and iterations from command line (by [@adamsitnik](https://github.com/adamsitnik))

### Custom default settings for console argument parser

If you want to have a possibility to specify custom default Job settings programmatically and optionally overwrite it with console line arguments, then you should create a global config with single job marked as `.AsDefault` and pass it to `BenchmarkSwitcher` together with the console line arguments.

Example: run single warmup iteration by default.

```cs
static void Main(string[] args)
    => BenchmarkSwitcher
        .FromAssembly(typeof(Program).Assembly)
        .Run(args, GetGlobalConfig());

static IConfig GetGlobalConfig()
    => DefaultConfig.Instance
        .With(Job.Default
            .WithWarmupCount(1)
            .AsDefault()); // the KEY to get it working
```

Now, the default settings are: `WarmupCount=1` but you might still overwrite it from console args like in the example below: 

```log
dotnet run -c Release -- --warmupCount 2
```

### Case-insensitive filter

The `--filter` or just `-f` allows you to filter the benchmarks by their full name (`namespace.typeName.methodName`) using glob patterns.

Examples:

1. Run all benchmarks from System.Memory namespace: `-f System.Memory*`
2. Run all benchmarks: `-f *`
3. Run all benchmarks from ClassA and ClassB `-f *ClassA* *ClassB*`

Now this filter expression is case-insensitive.

* [#864](https://github.com/dotnet/BenchmarkDotNet/issues/864) Make the filter case insensitive (assignee: [@adamsitnik](https://github.com/adamsitnik))
* [106777](https://github.com/dotnet/BenchmarkDotNet/commit/106777f7f575a8535f16292f1de80e8ffba2853a) make the filter case insensitive invariant culture, fixes #864 (by [@adamsitnik](https://github.com/adamsitnik))

### Benchmarking with different CoreRun instances

CoreRun is a simpler version of `dotnet run`, used for developing CoreCLR and CoreFX.

Typically when working on the performance of .NET Core a developer has more than 1 copy of CoreRun.
Example: CoreRun before my changes, and after my changes.
This change allows to simply run same benchmark for few different CoreRuns to compare the perf in easy way.

Sample usage:

```log
dotnet run -c Release -f netcoreapp2.1 -- -f *Empty.method --job dry --coreRun
C:\Projects\coreclr_upstream\bin\tests\Windows_NT.x64.Release\Tests\Core_Root\CoreRun.exe
C:\Projects\coreclr_upstream\bin\tests\Windows_NT.x64.Release\Tests\Core_Root_beforeMyChanges\CoreRun.exe
```

Sample output:

![image](https://user-images.githubusercontent.com/6011991/47417561-860f5800-d778-11e8-8dc6-b4e3586a9c90.png)

* [#925](https://github.com/dotnet/BenchmarkDotNet/issues/925) Make it possible to run the benchmark with multiple CoreRun.exe (assignee: [@adamsitnik](https://github.com/adamsitnik))
* [901616](https://github.com/dotnet/BenchmarkDotNet/commit/90161654725efd5639e0190638a3383d6a49e34c) when user provides CoreRun path and runtime in explicit way, we should use th... (by [@adamsitnik](https://github.com/adamsitnik))
* [46bebf](https://github.com/dotnet/BenchmarkDotNet/commit/46bebf1497d4e9314c6dfd2d4e10df81332aa4fa) allow the users to run the same benchmarks using few different CoreRun.exe, f... (by [@adamsitnik](https://github.com/adamsitnik))

### Hardware counters command-line support

```log
--counters CacheMisses+InstructionRetired
```

* [1e3df7](https://github.com/dotnet/BenchmarkDotNet/commit/1e3df74b2f927f541bed723f65c2d571fa850c53) make it possible to specify hardware counters from command line (by [@adamsitnik](https://github.com/adamsitnik))
* [a4f91a](https://github.com/dotnet/BenchmarkDotNet/commit/a4f91a392675e4851a785095af162b977d249ba3) better handling of edge cases for parsing hardware counters from the console ... (by [@adamsitnik](https://github.com/adamsitnik))

---

## Exporters

### Markdown output for DisassemblyDiagnoser

Now `DisassemblyDiagnoser` generates markdown version of the assembly listing.

* [#560](https://github.com/dotnet/BenchmarkDotNet/issues/560) Suggestion: markdown output for DisassemblyDiagnoser (assignee: [@adamsitnik](https://github.com/adamsitnik))
* [1e6235](https://github.com/dotnet/BenchmarkDotNet/commit/1e62355f209a25c7a33f9ab7e7e03b0afe7d851f) github markdown exporter for Disassembler, fixes #560 (by [@adamsitnik](https://github.com/adamsitnik))

### Diff view for disassembler output

Now we have `PrettyGithubMarkdownDiffDisassemblyExporter` which can generates
  nice diffs between assembly listings.
This mode can be activated via the `--disasmDiff` command line argument or
  the `printDiff: true` argument of `DisassemblyDiagnoserConfig`.
An output example (Diff between SumLocal and SumField on .NET Core 2.1.4 (CoreCLR 4.6.26814.03, CoreFX 4.6.26814.02), 64bit RyuJIT)

```diff
-; BenchmarkDotNet.Samples.IntroDisassemblyRyuJit.SumLocal()
-                   var local = field; // we use local variable that points to the field
-            ^^^^^^^^^^^^^^^^^^
-       mov     rax,qword ptr [rcx+8]
+; BenchmarkDotNet.Samples.IntroDisassemblyRyuJit.SumField()
                    int sum = 0;
             ^^^^^^^^^^^^
-       xor     edx,edx
-                   for (int i = 0; i < local.Length; i++)
+       xor     eax,eax
+                   for (int i = 0; i < field.Length; i++)
                  ^^^^^^^^^
-       xor     ecx,ecx
-                   for (int i = 0; i < local.Length; i++)
+       xor     edx,edx
+                   for (int i = 0; i < field.Length; i++)
                             ^^^^^^^^^^^^^^^^
-       mov     r8d,dword ptr [rax+8]
-       test    r8d,r8d
+       mov     rcx,qword ptr [rcx+8]
+       cmp     dword ptr [rcx+8],0
        jle     M00_L01
-                       sum += local[i];
+                       sum += field[i];
                 ^^^^^^^^^^^^^^^^
 M00_L00:
-       movsxd  r9,ecx
-       add     edx,dword ptr [rax+r9*4+10h]
-                   for (int i = 0; i < local.Length; i++)
+       mov     r8,rcx
+       cmp     edx,dword ptr [r8+8]
+       jae     00007ff9`0c412c1f
+       movsxd  r9,edx
+       add     eax,dword ptr [r8+r9*4+10h]
+                   for (int i = 0; i < field.Length; i++)
                                               ^^^
-       inc     ecx
-       cmp     r8d,ecx
+       inc     edx
+       cmp     dword ptr [rcx+8],edx
        jg      M00_L00
                    return sum;
             ^^^^^^^^^^^
 M00_L01:
-       mov     eax,edx
-; Total bytes of code 34
+       add     rsp,28h
+; Total bytes of code 42
```

* [#544](https://github.com/dotnet/BenchmarkDotNet/issues/544) Diff view for disassembler output (assignee: [@wojtpl2](https://github.com/wojtpl2))
* [#927](https://github.com/dotnet/BenchmarkDotNet/pull/927) Improve Disassembly exporters and add PrettyGithubMarkdownDiffDisassemblyExporter (by [@wojtpl2](https://github.com/wojtpl2))
* [#936](https://github.com/dotnet/BenchmarkDotNet/issues/936) Producing the asm diff reports on demand
* [#937](https://github.com/dotnet/BenchmarkDotNet/pull/937) Producing the asm diff reports on demand - fix for #936 (by [@wojtpl2](https://github.com/wojtpl2))
* [1903a1](https://github.com/dotnet/BenchmarkDotNet/commit/1903a1bd96d207ed51611d1dc546920f5bfb0d86) Improve Disassembly exporters and add PrettyGithubMarkdownDiffDisassemblyExpo... (by [@wojtpl2](https://github.com/wojtpl2))
* [dd103b](https://github.com/dotnet/BenchmarkDotNet/commit/dd103b60a4af0d3b9e7efb523c0923e7cbd8b62d) Producing the asm diff reports on demand - fixes #936 (#937) (by [@wojtpl2](https://github.com/wojtpl2))

### Improved LINQPad support

If you run BenchmarkDotNet v0.11.2+ in LINQPad, your logs will be colored and monospaced:

![linqpad](https://user-images.githubusercontent.com/2259237/47839043-4ec92880-ddc2-11e8-838d-960f6d5449e5.png)

* [#447](https://github.com/dotnet/BenchmarkDotNet/issues/447) Implement ColoredLogger for LinqPad
* [#903](https://github.com/dotnet/BenchmarkDotNet/pull/903) Add LINQPad logging (by [@bgrainger](https://github.com/bgrainger))
* [#915](https://github.com/dotnet/BenchmarkDotNet/pull/915) Use a monospaced font for LINQPad logging output (by [@bgrainger](https://github.com/bgrainger))
* [c3b609](https://github.com/dotnet/BenchmarkDotNet/commit/c3b6095b933b132c1773ced3af126f282465b980) Add LINQPad logging (#903) (by [@bgrainger](https://github.com/bgrainger))
* [10fdd0](https://github.com/dotnet/BenchmarkDotNet/commit/10fdd0998b46c4358f6fa38aacc21e57a7730724) Use a monospaced font for LINQPad logging output. (#915) (by [@bgrainger](https://github.com/bgrainger))

### Better CPU brand strings

We did a lot of changes which improve the presentation form of the CPU brand string.
Here is an example of such string in the previous version of BenchmarkDotNet:

```log
AMD Ryzen 7 2700X Eight-Core Processor (Max: 4.10GHz), 1 CPU, 16 logical and 8 physical cores
```

Now it becomes:

```log
AMD Ryzen 7 2700X 4.10GHz, 1 CPU, 16 logical and 8 physical cores
```

As you can see, "Eight-Core Processor" was removed (because we already have "8 physical cores");
  "(Max: 4.10GHz)" was replaced by 4.10GHz (because the original CPU brand string doesn't contain the nominal frequency).

* [#859](https://github.com/dotnet/BenchmarkDotNet/issues/859) Strange max frequency values on Windows (assignee: [@Rizzen](https://github.com/Rizzen))
* [#909](https://github.com/dotnet/BenchmarkDotNet/issues/909) Improve CPU Brand Strings without frequency
* [#860](https://github.com/dotnet/BenchmarkDotNet/pull/860) Fix strange CPU Frequency values (by [@Rizzen](https://github.com/Rizzen))
* [#910](https://github.com/dotnet/BenchmarkDotNet/pull/910) Simplify AMD Ryzen CPU brand info (by [@lahma](https://github.com/lahma))
* [a78b38](https://github.com/dotnet/BenchmarkDotNet/commit/a78b38b0e89d04ad3fe8934162c7adb42f81eabe) Fix strange CPU Frequency values (#860) (by [@Rizzen](https://github.com/Rizzen))
* [5df1e6](https://github.com/dotnet/BenchmarkDotNet/commit/5df1e6434b791eb5da6f6ef42505fc6a94ebd008) Simplify AMD Ryzen CPU brand info (#910) (by [@lahma](https://github.com/lahma))

---

## Attributes

### Async GlobalSetup and GlobalCleanup

Now GlobalSetup and GlobalCleanup methods can be async.

See also: docs.setup-and-cleanup

* [#521](https://github.com/dotnet/BenchmarkDotNet/issues/521) Support async Setup/Cleanup
* [#892](https://github.com/dotnet/BenchmarkDotNet/pull/892) Added support for async GlobalSetup. (by [@dlemstra](https://github.com/dlemstra))
* [#923](https://github.com/dotnet/BenchmarkDotNet/pull/923) async GlobalCleanup support (by [@dlemstra](https://github.com/dlemstra))
* [#926](https://github.com/dotnet/BenchmarkDotNet/pull/926) Added support for async GlobalCleanup. (by [@dlemstra](https://github.com/dlemstra))
* [e0f7a6](https://github.com/dotnet/BenchmarkDotNet/commit/e0f7a67681860ead87cef76fa0db349460b34eb0) Added support for async GlobalSetup. (#892) (by [@dlemstra](https://github.com/dlemstra))
* [a971a4](https://github.com/dotnet/BenchmarkDotNet/commit/a971a435ce6e6ca25d246e5e2cd56c5b2cf4739d) async GlobalCleanup support (#923) (by [@dlemstra](https://github.com/dlemstra))
* [e4c7b8](https://github.com/dotnet/BenchmarkDotNet/commit/e4c7b852e5593bb280881e28ece51d26687c5ba9) Added support for async GlobalCleanup. (#926), fixes #521 (by [@dlemstra](https://github.com/dlemstra))

### Introduced ParamsAllValues

If you want to use all possible values of an `enum` or another type with a small number of values, you can use the [`[ParamsAllValues]`](xref:BenchmarkDotNet.Attributes.ParamsAllValuesAttribute) attribute, instead of listing all the values by hand. The types supported by the attribute are:

* `bool`
* any `enum` that is not marked with `[Flags]`
* `Nullable<T>`, where `T` is an enum or boolean

An example:

```cs
public class IntroParamsAllValues
{
    public enum CustomEnum
    {
        A,
        BB,
        CCC
    }

    [ParamsAllValues]
    public CustomEnum E { get; set; }

    [ParamsAllValues]
    public bool? B { get; set; }

    [Benchmark]
    public void Benchmark()
    {
        Thread.Sleep(
            E.ToString().Length * 100 +
            (B == true ? 20 : B == false ? 10 : 0));
    }
}
```

Output:

```markdown
    Method |   E |     B |     Mean | Error |
---------- |---- |------ |---------:|------:|
 Benchmark |   A |     ? | 101.9 ms |    NA |
 Benchmark |   A | False | 111.9 ms |    NA |
 Benchmark |   A |  True | 122.3 ms |    NA |
 Benchmark |  BB |     ? | 201.5 ms |    NA |
 Benchmark |  BB | False | 211.8 ms |    NA |
 Benchmark |  BB |  True | 221.4 ms |    NA |
 Benchmark | CCC |     ? | 301.8 ms |    NA |
 Benchmark | CCC | False | 312.3 ms |    NA |
 Benchmark | CCC |  True | 322.2 ms |    NA |

// * Legends *
  E     : Value of the 'E' parameter
  B     : Value of the 'B' parameter
```

* [#658](https://github.com/dotnet/BenchmarkDotNet/issues/658) [Params] for enums should include all values by default
* [#908](https://github.com/dotnet/BenchmarkDotNet/pull/908) Added [ParamsAllValues] (by [@gsomix](https://github.com/gsomix))
* [922dff](https://github.com/dotnet/BenchmarkDotNet/commit/922dfff62d6cf6fd808865e705a09eee63690a2e) Added [ParamsAllValues] (#908), fixes #658 (by [@gsomix](https://github.com/gsomix))
* [846d08](https://github.com/dotnet/BenchmarkDotNet/commit/846d0863b6456d3e1e6ccab06d8e61c5cd064194) ParamsAllValuesValidator fixes (by [@AndreyAkinshin](https://github.com/AndreyAkinshin))

### Selecting Baseline across Methods and Jobs

Now it's possible to mark a method and a job as baselines at the same time:

```cs
public class TheBaselines
{
    [Benchmark(Baseline = true)]
    public void Sleep100ms() => Thread.Sleep(TimeSpan.FromMilliseconds(100));
    
    [Benchmark]
    public void Sleep50ms() => Thread.Sleep(TimeSpan.FromMilliseconds(50));
}

static void Main(string[] args)
    => BenchmarkSwitcher
        .FromTypes(new[] { typeof(TheBaselines) })
        .Run(args,
                DefaultConfig.Instance
                    .With(Job.Core.AsBaseline())
                    .With(Job.Clr.WithId("CLR 4.7.2")));
```

* [#880](https://github.com/dotnet/BenchmarkDotNet/issues/880) Select Baseline across Methods and Jobs (assignee: [@AndreyAkinshin](https://github.com/AndreyAkinshin))
* [21a007](https://github.com/dotnet/BenchmarkDotNet/commit/21a0073cc8b486f41b2e84deafacd00a1303013a) Support method-job baseline pairs, fixes #880 (by [@AndreyAkinshin](https://github.com/AndreyAkinshin))

---

## Statistics

### Statistical tests

In this release, statistical testing was significantly improved.
Now it's possible to compare all benchmarks against baseline with the help
  of Welch's t-test or Mann–Whitney U test.

An example:

```cs
[StatisticalTestColumn(
    StatisticalTestKind.Welch, ThresholdUnit.Microseconds, 1, true)]
[StatisticalTestColumn(
    StatisticalTestKind.MannWhitney, ThresholdUnit.Microseconds, 1, true)]
[StatisticalTestColumn(
    StatisticalTestKind.Welch, ThresholdUnit.Ratio, 0.03, true)]
[StatisticalTestColumn(
    StatisticalTestKind.MannWhitney, ThresholdUnit.Ratio, 0.03, true)]
[SimpleJob(warmupCount: 0, targetCount: 5)]
public class IntroStatisticalTesting
{
    [Benchmark] public void Sleep50() => Thread.Sleep(50);
    [Benchmark] public void Sleep97() => Thread.Sleep(97);
    [Benchmark] public void Sleep99() => Thread.Sleep(99);
    [Benchmark(Baseline = true)] public void Sleep100() => Thread.Sleep(100);
    [Benchmark] public void Sleep101() => Thread.Sleep(101);
    [Benchmark] public void Sleep103() => Thread.Sleep(103);
    [Benchmark] public void Sleep150() => Thread.Sleep(150);
}
```

Output:

|   Method |      Mean |     Error |    StdDev | Ratio |   Welch(1us)/p-values |    Welch(3%)/p-values | MannWhitney(1us)/p-values | MannWhitney(3%)/p-values |
|--------- |----------:|----------:|----------:|------:|---------------------- |---------------------- |-------------------------- |------------------------- |
|  Sleep50 |  53.13 ms | 0.5901 ms | 0.1532 ms |  0.51 | Faster: 1.0000/0.0000 | Faster: 1.0000/0.0000 |     Faster: 1.0000/0.0040 |    Faster: 1.0000/0.0040 |
|  Sleep97 | 100.07 ms | 0.9093 ms | 0.2361 ms |  0.97 | Faster: 1.0000/0.0000 |   Same: 1.0000/0.1290 |     Faster: 1.0000/0.0040 |      Same: 1.0000/0.1111 |
|  Sleep99 | 102.23 ms | 2.4462 ms | 0.6353 ms |  0.99 | Faster: 0.9928/0.0072 |   Same: 1.0000/0.9994 |     Faster: 0.9960/0.0079 |      Same: 1.0000/1.0000 |
| Sleep100 | 103.34 ms | 0.8180 ms | 0.2124 ms |  1.00 |   Base: 0.5029/0.5029 |   Base: 1.0000/1.0000 |       Base: 0.7262/0.7262 |      Base: 1.0000/1.0000 |
| Sleep101 | 103.73 ms | 2.1591 ms | 0.5607 ms |  1.00 |   Same: 0.1041/0.8969 |   Same: 0.9999/1.0000 |       Same: 0.1111/0.9246 |      Same: 1.0000/1.0000 |
| Sleep103 | 106.21 ms | 1.2511 ms | 0.3249 ms |  1.03 | Slower: 0.0000/1.0000 |   Same: 0.9447/1.0000 |     Slower: 0.0040/1.0000 |      Same: 0.9246/1.0000 |
| Sleep150 | 153.16 ms | 3.4929 ms | 0.9071 ms |  1.48 | Slower: 0.0000/1.0000 | Slower: 0.0000/1.0000 |     Slower: 0.0040/1.0000 |    Slower: 0.0040/1.0000 |

```log
// * Legends *
  Mean                      : Arithmetic mean of all measurements
  Error                     : Half of 99.9% confidence interval
  StdDev                    : Standard deviation of all measurements
  Ratio                     : Mean of the ratio distribution ([Current]/[Baseline])
  Welch(1us)/p-values       : Welch-based TOST equivalence test with 1 us threshold. Format: 'Result: p-value(Slower)|p-value(Faster)'
  Welch(3%)/p-values        : Welch-based TOST equivalence test with 3% threshold. Format: 'Result: p-value(Slower)|p-value(Faster)'
  MannWhitney(1us)/p-values : MannWhitney-based TOST equivalence test with 1 us threshold. Format: 'Result: p-value(Slower)|p-value(Faster)'
  MannWhitney(3%)/p-values  : MannWhitney-based TOST equivalence test with 3% threshold. Format: 'Result: p-value(Slower)|p-value(Faster)'
  1 ms                      : 1 Millisecond (0.001 sec)
```

The statistical testing is a work-in-progress feature.
In future versions of BenchmarkDotNet, we are planning to improve API, fill missed docs, and introduce more parameters for customization.

See also: @BenchmarkDotNet.Samples.IntroStatisticalTesting

* [60eca0](https://github.com/dotnet/BenchmarkDotNet/commit/60eca005326970202a33891e5aecd2ef6b7e4cd0) Threshold API for WelchTTest; Improve Student accuracy for small n (by [@AndreyAkinshin](https://github.com/AndreyAkinshin))
* [05cc8d](https://github.com/dotnet/BenchmarkDotNet/commit/05cc8d15ef88e382bbb1827d766d7275c3e42abd) Statistical testing improvements (by [@AndreyAkinshin](https://github.com/AndreyAkinshin))

### ZeroMeasurementAnalyser

When you have an empty benchmark like this

```cs
[Benchmark]
public void Empty() { }
```

The expected duration of this method is zero.
However, you can get the mean value like `0.0023ns` because of the natural noise.
It's a pretty confusing result for many developers.
Since v0.11.2, we have `ZeroMeasurementAnalyser` which warn you about such methods.
By default, BenchmarkDotNet automatically evaluate overhead.
In this case, `ZeroMeasurementAnalyser` runs Welch's t-test and compare actual and overhead measurements.
If the overhead evaluation is disabled, it runs one-sample Student's t-test against a half of CPU cycle.

* [#906](https://github.com/dotnet/BenchmarkDotNet/pull/906) Zero measurement analyser (by [@Rizzen](https://github.com/Rizzen))
* [48d193](https://github.com/dotnet/BenchmarkDotNet/commit/48d193e30c780eb43e65b21f892c48db5dab6f6b) Zero measurement analyser (#906) (by [@Rizzen](https://github.com/Rizzen))

### RatioColumn

The `Ratio` column was formerly known as `Scaled`.
The old title was a source of misunderstanding and confusion because
  many developers interpreted it as the ratio of means (e.g., `50.46`/`100.39` for `Time50`).
The ratio of distribution means and the mean of the ratio distribution are pretty close to each other in most cases,
  but they are not equal.

See also:
  @BenchmarkDotNet.Samples.IntroBenchmarkBaseline,
  @BenchmarkDotNet.Samples.IntroRatioSD,
  @docs.baselines.

* [4e64c9](https://github.com/dotnet/BenchmarkDotNet/commit/4e64c94cfe7b49bbdc06aabb6ee1f262bd370862) Ratio/RatioSD columns (by [@AndreyAkinshin](https://github.com/AndreyAkinshin))

