# Source data

This is the folder for benchmark dataset files. The following benchmarks are included:

## GSM8K

Downloaded from https://github.com/openai/grade-school-math.

## Massive Multitask Language Understanding (MMLU)

Downloaded from https://github.com/hendrycks/test. We did not present prompt optimization and evaluation results in our paper because the headroom of prompt optimization was observed to be much smaller (usually <10% for the four categories) than the other benchmarks.

Because of file size limits, we are only including the test split in this
repository.

## Big-Bench Hard (BBH)

Downloaded from https://github.com/suzgunmirac/BIG-Bench-Hard.

## MultiArith

## AQuA

Downloaded from https://github.com/google-deepmind/AQuA.

---

All copyrights belong to the original benchmark authors.

*Disclaimer: this is not an officially supported Google product.*
