---
title: The gem5 Standard Library
author: Bobby Bruce
slides_code: "using-gem5-02-gem5-stdlib-tutorial.pdf"
livestream_code: vbruiMyIFsA
example_code: /materials/using-gem5/02-stdlib
---

In this section, we learn how to use the components and simulation modules of the standard library and how to extend the standard library with new components.

This page closely follows the slides.
Please consider these as notes for the presentation.
It may be helpful to reference the slides while following these notes.

This tutorial relies heavily on code provided in [materials/using-gem5/02-stdlib](https://github.com/gem5bootcamp/gem5-bootcamp-env/blob/main/materials/using-gem5/02-stdlib/) for exercises.
Completed exercises can be found in [materials/using-gem5/02-stdlib/complete](https://github.com/gem5bootcamp/gem5-bootcamp-env/blob/main/materials/using-gem5/02-stdlib/complete/).

## What is the standard library for?

Prior to the development of the gem5 stdlib users would need to create large configuration files, often 100s of lines of Python for even a basic simulation.

This was problematic.
Typically this resulted in communities of gem5 users passing around pre-built scripts and modifying/extending them endlessly.
We observed that typically users only needed to change a very small part in an otherwise "ordinary" simulation and, therefore, scripts containing "ordinary" simulation setups were frequently utilized.

In response to this the stdlib was created.
The stdlib aims to remove "boilerplate" gem5 Python configuration code.
It provides an abstraction where users can specify simulations at a high level and extend via established APIs.
It is analogous to standard libraries provided by modern programming languages:
A suite of helpful code to aid users, provided and supported by the project.

## The stdlib modular metaphor

The standard library aims to be modular.
That is, components should be interchangeable with components of the same type:
I.e., it should be possible to swap one type of memory system with another, or one type of Processor with another.
There are 4 high-level component types provided as part of the gem5 stdlib:

* **Processor**: A processor is a component that contains CPU Cores.
* **Memory System**: A memory system to be used.
* **Cache Hierarchy**: While mostly encompassing the cache hierarchy system, these components describe any component that may sit between the Processor and Memory System.
* **Board**: Analogous to a motherboard.
This typically accepts a Processor, Cache Hierarchy, and Memory System as part of it's constructor and connects them together.
The board is responsible for ensuring all the components interact with on another in a sensible manner.

The stdlib enforces a Object Orientated design philosophy.
For example, to create a processor you must inherit from the `AbstractProcessor` class and implement all the abstract methods.
All boards inherit from `AbstractBoard` and accepts `AbstractProcessor`, `AbstractMemorySystem` and `AbstractCacheHierarchy` in its constructor.

## Where to find stuff

All the source code for the stdlib is found in the "src/python/gem5" directory in the gem5 repository.

As this is python, the directory structure maps to the import string.
For example, to import the `SimpleBoard` declared within "src/python/gem5/components/boards/simple_board.py", a user would include the following in their configuration file:

```py
from gem5.components.boards.simple_board import SimpleBoard
```

## Getting started

To get started we create a very basic SE Mode simulation which will run a "Hello World" binary.
This will use the stdlib.

The [materials/using-gem5/02-stdlib/hello-world.py](https://github.com/gem5bootcamp/gem5-bootcamp-env/blob/main/materials/using-gem5/02-stdlib/hello-world.py) file has been provided as a starting point.
It provides all the necessary imports.
This file should be extended for this exercise.
The completed example is found in [materials/using-gem5/02-stdlib/complete/hello-world-complete.py](https://github.com/gem5bootcamp/gem5-bootcamp-env/blob/main/materials/using-gem5/02-stdlib/complete/hello-world-complete.py).

Those following this tutorial should add the following lines:

```py
# Obtain the components
cache_hierarchy = NoCache()
memory = SingleChannelDDR3_1600("1GiB")
processor = SimpleProcessor(cpu_type=CPUTypes.TIMING, num_cores=1)
```

This declares the components we'll be using for this simulation:

* **NoCache**: A dummy cache hierarchy setup which simply connects processor cores directly to main memory.
* **SingleChannelDDR3_1600**: A Single Channel DDR3 1600 memory system set to a size of "1GiB".
**Note:** "GiB" not "GB".
gem5 assumes base 2 in most cases.
* **SimpleProcessor**: A Simple processor with a single TIMING cores (A `SimpleProcessor` is a processor of N cores, all of the same type).

```py
board = SimpleBoard(
    clk_freq="3GHz",
    processor=processor,
    memory=memory,
    cache_hierarchy=cache_hierarchy,
)
```

This adds all the components to the board and sets the frequency to "3GHz".
In gem5 the frequency is set on the board.

The `SimpleBoard` is used to run basic SE-Mode simulations.

### gem5 Resources

The gem5 resources is a repository providing resources that are known to be compatible with gem5.

These resources are not necessary for the compilation or running of gem5 but may aid users in running simulations: E.g.: disk images, kernels, applications, cross-compilers, etc.

Resources are held on gem5's Google Cloud Bucket and the sources for these resources are found at:https://gem5.googlesource.com/public/gem5-resources/

The stdlib can automatically obtain and use these resources.

It is our hope to, at some point, provide a better web interface to navigate the resources.
However, at present, the best approach is to navigate the https://resources.gem5.org/resources.json file.
**Warning:** This is a machine readable format.

In the "resources.json" file you can see each resource.
Each as a "name" which can be used to reference (and obtain) a given resources.
E.g., "riscv-disk-img" is the ID/name for a simple RISCV disk image based on busybox".

#### Obtaining Resources in the stdlib

The following will obtain the "riscv-disk-img" resource then print where it was obtained:

```
gem5-x86 materials/using-gem5/02-stdlib/obtaining-resources.py
```

Please consult the [materials/using-gem5/02-stdlib/obtaining-resources.py](https://github.com/gem5bootcamp/gem5-bootcamp-env/blob/main/materials/using-gem5/02-stdlib/obtaining-resources.py) file to see how this is done.
It is the `Resource` class which does all the work.

```py
from gem5.resources.resource import Resource

resource = Resource("riscv-disk-img")

print(f"the resource is available at {resource.get_local_path()}")
```

If the same command is run again, the resource will not be re-retrieved. gem5-resources caches what is required.

#### Custom Resource

If a resource is required which is not provided by gem5-resources but instead provided by the gem5 user, `CustomResource` class can be used.

```py
from gem5.resources.resource import CustomResource

CustomResource("tests/test-progs/hello/bin/x86/linux/hello")
```

#### Adding Resources to "hello-world.py"

Going back to "materials/using-gem5/02-stdlib/hello-world.py" we add:

```
# Set the workload.
binary = Resource("x86-hello64-static")
board.set_se_binary_workload(binary)
```

The `set_se_binary_workload` function is used to set a Syscall Emulation mode workload with a single binary.

### Adding the Simulator

The following should be appended:

```
# Setup the simulator and run the simulation.
simulator = Simulator(board=board)
simulator.run()
```

This declares the board that is to be simulated.
The `run()` function begins the simulation.

### Running the simulation

Save the file then execute:

```
gem5-x86 materials/using-gem5/02-stdlib/hello-world.py
```

This will run the simulation and produce an output to the terminal giving information on the simulation.

## More detailed output

To get stats users can look into "m5out/stats.txt".

The "m5out/config.*" file detail the system configuration.
Various formats exist.
"m5out/config.ini*" is the most human-readable.

*In full-system simulation the terminal output can be found in the "m5out" directory also.

## Extending the "hello-world.py" simulation

gem5 is modular.
In general, you can replace components with other components of the same type.

In this example we will add a real cache implementation to the design.

The [materials/using-gem5/02-stdlib/hello-world.py](https://github.com/gem5bootcamp/gem5-bootcamp-env/blob/main/materials/using-gem5/02-stdlib/hello-world.py) file from the previous examples will be extended for this.
A completed example can be found in [materials/using-gem5/02-stdlib/complete/hello-world-with-cache-complete.py](https://github.com/gem5bootcamp/gem5-bootcamp-env/blob/main/materials/using-gem5/02-stdlbi/complete/hello-world-with-cache-complete.py).

```py
#from gem5.components.cachehierarchies.classic.no_cache import NoCache
from gem5.components.cachehierarchies.classic.private_l1_private_l2_cache_hierarchy import PrivateL1PrivateL2CacheHierarchy
```

This imports the `PrivateL1PrivateL2CacheHierarchy`.

```py
# cache_hierarchy = NoCache()
cache_hierarchy = PrivateL1PrivateL2CacheHierarchy(
    l1d_size="32KiB",
    l1i_size="32KiB",
    l2_size="64KiB",
)
```

This sets up our new cache hierarchy.
The simulation "Hello World" simulation can then be rerun with:

```sh
gem5-x86 materials/using-gem5/02-stdlib/hello-world.py
```

**Short exercise**: Compete the "m5out/stats.txt" for running the simulation with and without a cache hierarchy.

## An X86 full-system simulation

In this exercise a X86 full system simulation will be created.
While including a more advanced full system setup, it will also establish more advanced ideas of how to use the stdlib.

The [materials/02-stdlib/x86-full-system.py](https://github.com/gem5bootcamp/gem5-bootcamp-env/blob/main/materials/using-gem5/02-stdlib/x86-full-system.py) provides a starting point for this exercise.
It is assumed those following along with this exercise will append this file.
The completed example for this exercise is found in [materials/using-gem5/02-stdlib/complete/x86-full-system-complete.py](https://github.com/gem5bootcamp/gem5-bootcamp-env/blob/main/materials/using-gem5/02-stdlib/complete/x86-full-system-complete.py).

To begin:

```py
requires(
    isa_required=ISA.X86,
    coherence_protocol_required=CoherenceProtocol.MESI_TWO_LEVEL,
)
```

This adds a check for the gem5 binary parsing the script.
It is a safety to ensure the gem5 binary you are running is correct.
While using this `requires` function is optional, it can stop the simulation failing in a catastrophic manner.

In this case the `requires` function is checking:

1. The binary supports the X86 ISA.
2. The binary supports the MESI Two Level coherence protocol.

Moving on we add the following:

```py
cache_hierarchy = MESITwoLevelCacheHierarchy(
    l1d_size="32KiB",
    l1d_assoc=8,
    l1i_size="32KiB",
    l1i_assoc=8,
    l2_assoc=16,
    num_l2_banks=1,
)

memory = SingleChannelDDR3_1600("2GiB")
```

This adds a Ruby Cache: The `MESITwoLevelCacheHierarchy`.
The interface for Ruby Caches and Classic Caches are the same at this level and therefore are interchangeable.
This is nothing particularly special about the memory system.
It is a 2GiB Single Channel DDR3 1600 setup.

```py
processor = SimpleSwitchableProcessor(
    starting_core_type=CPUTypes.TIMING,
    switch_core_type=CPUTypes.O3,
    num_cores=2,
)
```

The `SimpleSwitchingProcessor` allows for different types of cores to be swapped during a simulation with `processor.switch()`.

This can be a useful tactic when wanting to switch to and from a faster (but lower accuracy) CPU cores, to a slower (but more accurate) CPU cores.
The faster CPU cores can be used to execute regions of code of lesser interest while the more detailed cores can be used on regions of code of greater interest (e.g., you can boot the OS on a faster CPU type execute the program of interest on the slower, more accurate CPU type).

We then add all these components to the board:

```py
board = X86Board(
    clk_freq="3GHz",
    processor=processor,
    memory=memory,
    cache_hierarchy=cache_hierarchy,
)
```

In this case we use the `X86Board`;
a board used to execute Full-System simulations using the X86 ISA.

Next we add:

```py
command = "m5 exit;" \
        + "echo 'This is running on O3 CPU cores.';" \
        + "sleep 1;" \
        + "m5 exit;"

board.set_kernel_disk_workload(
    kernel=Resource("x86-linux-kernel-5.4.49"),
    disk_image=Resource("x86-ubunutu-18.04-img"),
    readfile_conents=command,
)
```

The `set_kernel_disk_workload` function is used to run a full system workload.
The `kernel` and `disk_image` resources must be specified as a resource.
In this case we are using the x86 Linux v5.4.49 resource and the Ubuntu 18.04 resource.

The `readfile_contents` parameter utilizes gem5's `readfile` command (part of `m5 ops`).
It is a way of passing a file's contents to the gem5 simulation.
It can then be used by the simulation.
The "x86-ubuntu-18.04-img" disk image resource will copy the contents to a file then execute it as a strict, if set, once the boot is complete (if not set, the simulation simply exits after a boot).
Therefore, in this example the value of `readfile_contents` is a shell script to be executed:
An `m5 exit` instruction, an echo statement, a sleep statement, then another `m5 exit` instruction.

### The Simulator Module

During a simulation you can have "Exit Events".
In this example there are two, both triggered by the `m5 exit` instructions.

The simulation loop (that in which the simulation executes) exists upon reaching an exit event and returns to the Python configuration script.

Once back in the configuration script other actions can be taken then the simulation loop re-entered via a `simulator.run()` command.
When the simulation loop is re-entered it re-enters into the same stats in which it left.

Go to the [materials/using-gem5/02-stdlib/simulator-use.py](https://github.com/gem5bootcamp/gem5-bootcamp-env/blob/main/materials/using-gem5/02-stdlib/simulator-use.py) file.

This is a pretty normal looking script, very similar to the "Hello World" program except we are executing the "m5-exit-example" binary.

Navigating to [materials/using-gem5/02-stdlib/m5-exit-example/m5-exit/example.c](https://github.com/gem5bootcamp/gem5-bootcamp-env/blob/main/materials/using-gem5/02-stdlib/m5-exit-example/m5-exit-example.c) we can see the source.
In it a `m5_exit(0)` command is called indefinitely while also printing how many times the the program has exited thus far.

Going back to [materials/using-gem5/02-stdlib/simulator-use.py](https://github.com/gem5bootcamp/gem5-bootcamp-env/blob/main/materials/using-gem5/02-stdlib/simulator-use.py) we can append the following to the file:

```py
simulator = Simulator(board=board)
simulator.run()

print("We can do stuff after an m5 exit event. Prior to continuing the simulation")

simulator.run()

print("And again...")

simulator.run()
```

This can then be ran with:

```sh
gem5-x86 materials/using-gem5/02-stdlib/simulator-use.py
```

The purpose of this exercise is to show that it's possible to exit the simulation loop and run things in the configuration script before re-entering the simulation loop.

We can, however, be more clever.
Remove the code appended to the file and instead, add the following:

```py
simulator = Simulator(
    board=board,
    on_exit_event={
        ExitEvent.Exit: (print(statement) for statement in ["Just exited the first exit event",
                                                            "Just exited the second exit event",
                                                            "Just exited the third exit event"])
    }
)
```

Again we execute with:

```sh
gem5-x86 materials/using-gem5/02-stdlib/simulator-use.py
```

Here we are making use of the `Simulator` class's `on_exit_event` parameter.
This parameter is a Python Dict which maps exit event types to a generator which yields on each execution of that exit event.

In this example we are simply printing a statement for the first three times the exit event is reached.

Here we're only covering the "Exit" type exit event but there are other types.

You can override different types for different things.
The Simulator modular has default behavior for each (see [gem5/src/python/gem5/simulate/exit_event_generators.py](https://gem5.googlesource.com/public/gem5/+/refs/tags/v22.0.0.2/src/python/gem5/simulate/exit_event_generators.py)).


Going back to "materials/using-gem5/02-stdlib/x86-full-system.py" and add the following

```py
simulator = Simulator(
    board=board,
    on_exit_event={
        ExitEvent.Exit : (func() for func in [processor.switch]),
    },
)
simulator.run()
```

At this point the simulation setup is complete.
It can be executed using:

```sh
gem5-x86 materials/using-gem5/02-stdlib/x86-full-system.py
```

**Warning:** This will take a a long time to complete execution. Use Cntrl+C to exit.

## Extending the library

In this exercise we'll extend the gem5 stdlib to add a new component.
In this we'll be creating a new classic cache hierarchy called `UniqueCacheHierarchy`.

In this exercise we'll be modifying files in [materials/using-gem5/02-stdlib/unique_cache_hierarchy/](https://github.com/gem5bootcamp/gem5-bootcamp-env/blob/main/materials/using-gem5/02-stdlib/unique_cache_hierarchy/).
A completed exercise example can be found in [materials/using-gem5/02-stdlib/complete/unique_cache_hierarchy](https://github.com/gem5bootcamp/gem5-bootcamp-env/blob/main/materials/using-gem5/02-stdlib/complete/unique_cache_hierarchy).

Starting with "materials/using-gem5/02-stdlib/unique_cache_hierarchy/unique_cache_hierarchy.py" we have extended the `AbstractClassicCacheHierarchy` class and have exposed the abstract methods that must be implemented.
Going through these one by one:

```py
def __init__(self) -> None:
    AbstractClassicCacheHierarchy.__init__(self=self)
    self.membus = SystemXBar(width=64)
```

Here we setup a memory bus for our simulations

```py
def get_mem_side_port(self) -> Port:
    return self.membus.mem_side_ports
```

Here we set the cache hierarchy memory-side ports to that of the membus.

```py
def get_cpu_side_port(self) -> Port:
    return self.membus.cpu_side_ports
```

Here we set the cache hierarchy CPU side ports to that of the membus.

Next we navigate to [materials/using-gem5/02-stdlib/unique_cache_hierarchy/l1cache.py](https://github.com/gem5bootcamp/gem5-bootcamp-env/blob/main/materials/using-gem5/02-stdlib/unique_cache_hierarchy/l1cache.py).
We require an implementation of the `Cache` SimObject for the cache hierarchy component.
In this file the `Cache` SimObject has been extended but the SimObject's member variables must be set, which we do as so:

```py
class L1Cache(Cache):
    """
    A simple cache with default values.
    """

    def __init__(self):
        super().__init__()
        self.size="32KiB"
        self.assoc=8
        self.tag_latency=1
        self.data_latency=1
        self.response_latency=1
        self.mshrs=16
        self.tgts_per_mshr=20
        self.writeback_clean=True
        self.prefetcher = StridePrefetcher()
```

SimObject member variables are special.
You can only set SimObject variables.
Code like `self.custom_variable = 7` will cause an error.

If you want to create a non-SimObject variable, the variable name must have a preceding underscore: e.g., `self._custom_variable = 7`.

Going back to "materials/using-gem5/02-stdlib/unique_cache_hierarchy/unique_cache_hierarchy.py", we will implement the `incrporate_cache` function:

```
def incorporate_cache(self, board: AbstractBoard) -> None:
    # Set up the system port for functional access from the simulator.
    board.connect_system_port(self.membus.cpu_side_ports)

    for cntr in board.get_memory().get_memory_controllers():
        cntr.port = self.membus.mem_side_ports

    self.l1icaches = [
        L1Cache()
        for i in range(board.get_processor().get_num_cores())
    ]

    self.l1dcaches = [
        L1Cache()
        for i in range(board.get_processor().get_num_cores())
    ]

    for i, cpu in enumerate(board.get_processor().get_cores()):
        cpu.connect_icache(self.l1icaches[i].cpu_side)
        cpu.connect_dcache(self.l1dcaches[i].cpu_side)

        self.l1icaches[i].mem_side = self.membus.cpu_side_ports
        self.l1dcaches[i].mem_side = self.membus.cpu_side_ports

        int_req_port = self.membus.mem_side_ports
        int resp_port = self.membus.cpu_side_ports
        cpu.connect_interrupt(int_req_port, int_resp_port)
```

from here the custom cache hierarchy can be incorporated into a stdlib design by simply importing the class correctly.

## Create your own cache

As a challenge, create your own gem5 Private L1 Shared L2 Cache hierarchy.

Once done, modify the "hello-world.py" program to see the performance difference between using the Private L1 Cache/Private L2 Cache vs your custom Private L1/Shared L2 Cache hierarchy.
