Last year, we presented our initial investigations and stress testing as we prepared to deploy a large-scale Jenkins installation at Google. Now, with a year of real-world use under our belts, we'll discuss how our expectations held up, what new issues we encountered and how we have addressed them.

- Kokoro
- 1 Master, 500 Build Agents
- Save the artifacts externally
- Save the logs externally
- Virtually no configuration is done via the UI
- There is a hot failover system waiting for master to crash ...

## Ways to store configuration I version control

- Literate
- YAML
- Pipelines
- DSL

Neither of them met the requirements...