{% load static %}

<!DOCTYPE html>
<html lang="en">
  <head>
    <meta charset="UTF-8">
    <meta name="viewport" content="width=device-width, initial-scale=1" />
    <title>Tensor Trust: Interpretable Prompt Injection Attacks from an Online Game</title>
    <link rel="icon" type="image/png" href="{% static 'images/favicon.png' %}" sizes="128x128">
    <link href="https://fonts.googleapis.com/css2?family=Noto+Sans:wght@300;500&display=swap" rel="stylesheet">
    <link href="https://maxcdn.bootstrapcdn.com/font-awesome/4.7.0/css/font-awesome.min.css" rel="stylesheet">
    {% load static tailwind_tags gameui %}
    {% canonical_link 'paper' %}
    {% tailwind_css %}
  </head>
  <style>
    body {
      font-family: 'Noto Sans';
      font-weight: 300;
      color: #363636;
    }

    i {
      padding-right: 8px;
    }

    .buttons {
      font-size: 1rem;
    }

    .button {
      padding: 0.5rem 1rem;
      background-color: hsl(240, 5%, 25%);
      border-radius: 0.5rem;
      color: white;
      font-weight: 500;
      text-decoration: none;
      cursor: pointer;
    }

    .button:hover {
      background-color: hsl(204, 86%, 53%);
      color: white;
      text-decoration: none;
    }

    a {
      color: hsl(204, 86%, 53%);
    }

    .button.disabled {
      opacity: 0.4;
      cursor: not-allowed;
      border-color: hsl(0, 0%, 51%);
      color: hsl(0, 0%, 51%);
    }

    .button.disabled:hover {
      opacity: 0.4;
      background-color: hsl(240, 5%, 25%);
      text-decoration: none;
    }

    .button.play {
      color: hsl(40, 100%, 56%);
    }

    .button.play:hover {
      background-color: hsl(40, 86%, 53%);
      color: white;
      text-decoration: none;
    }

    a:hover {
      text-decoration: underline;
    }

    h1 {
      font-size: 2rem;
    }

    h2 {
      font-size: 1.2rem;
    }

    p, pre {
      font-size: 1rem;
    }
  </style>
  <body>
    <div class="w-full flex justify-center">
      <div class="w-full max-w-[1024px] min-h-screen p-6 flex flex-col items-center justify-center space-y-4 pb-20">
        <header class="flex flex-col items-center text-center space-y-4 pb-4">
          <h1 class="py-2 pt-6 font-bold">Tensor Trust: Interpretable Prompt Injection Attacks from an Online Game</h1>
          <h2>
            <a target="_blank" href="https://twitter.com/sdtoyer">Sam Toyer</a><sup>1</sup>, <a target="_blank" href="https://twitter.com/OliviaGWatkins2">Olivia Watkins</a><sup>1</sup>, <a target="_blank" href="https://twitter.com/EthanMendes3">Ethan Adrian Mendes</a><sup>1, 2</sup>, <a target="_blank" href="https://twitter.com/justinsvegliato">Justin Svegliato</a><sup>1</sup>, <a target="_blank" href="https://twitter.com/LukeBailey181">Luke Bailey</a><sup>1, 3</sup>, <a target="_blank" href="https://twitter.com/cnnmonsugar">Tiffany Wang</a><sup>1</sup>, <a target="_blank" href="https://twitter.com/isaacongjw">Isaac Ong</a><sup>1</sup>, <a target="_blank" href="https://kael.tech.blog/">Karim Elmaaroufi</a><sup>1</sup>, <a target="_blank" href="http://people.eecs.berkeley.edu/~pabbeel/">Pieter Abbeel</a><sup>1</sup>, <a target="_blank" href="https://people.eecs.berkeley.edu/~trevor/">Trevor Darrell</a><sup>1</sup>, <a target="_blank" href="https://aritter.github.io/">Alan Ritter</a><sup>2</sup>, <a target="_blank" href="http://people.eecs.berkeley.edu/~russell/">Stuart Russell</a><sup>1</sup>
          </h2>
          <h2 class="space-x-8">
            <sup>1</sup>UC Berkeley
            <sup>2</sup>Georgia Tech
            <sup>3</sup>Harvard University
          </h2>
          <div class="buttons flex flex-row space-x-2 py-2 wrap-break flex-wrap justify-center gap-2">
            <a target="_blank" class="button" href="https://arxiv.org/pdf/2311.01011.pdf"><i class="fa fa-file-pdf-o"></i>Paper</a>
            <a target="_blank" class="button" href="https://github.com/HumanCompatibleAI/tensor-trust-data"><i class="fa fa-table"></i>Data</a>
            <a target="_blank" class="button" href="https://colab.research.google.com/github/HumanCompatibleAI/tensor-trust-data/blob/main/Using%20the%20Tensor%20Trust%20dataset.ipynb"><i class="fa fa-laptop"></i>Colab</a>
            <a target="_blank" class="button" href="https://github.com/HumanCompatibleAI/tensor-trust"><i class="fa fa-github"></i>Code</a>
            <a target="_blank" class="button play" href="https://tensortrust.ai/"><i class="fa fa-play"></i>Play the game!</a>
          </div>
        </header>
        <img src="{% static 'images/lead_figure.png' %}" alt="Lead figure of the paper. Shows: Defender sets a defense prompt and an access code. Next, attacker attempts to break in." height={500} class="py-4" />
        <h2 class="text-3xl">Abstract</h2>
        <p>
          While Large Language Models (LLMs) are increasingly being used in
          real-world applications, they remain vulnerable to prompt injection
          attacks: malicious third party prompts that subvert the intent of the
          system designer. To help researchers study this problem, we present a
          dataset of over 126,000 prompt injection attacks and 46,000
          prompt-based "defenses" against prompt injection, all created by
          players of an online game called Tensor Trust. To the best of our
          knowledge, this is currently the largest dataset of human-generated
          adversarial examples for instruction-following LLMs. The attacks in
          our dataset have a lot of easily interpretable stucture, and shed
          light on the weaknesses of LLMs. We also use the dataset to create a
          benchmark for resistance to two types of prompt injection, which we
          refer to as prompt extraction and prompt hijacking. Our benchmark
          results show that many models are vulnerable to the attack strategies
          in the Tensor Trust dataset. Furthermore, we show that some attack
          strategies from the dataset generalize to deployed LLM-based
          applications, even though they have a very different set of
          constraints to the game. We release all data and source code at
          <a href="https://tensortrust.ai/paper">tensortrust.ai/paper</a>.
        </p>
        <details class="text-left mt-4">
          <summary class="cursor-pointer font-semibold text-center">Cite this work</summary>
          <pre class="whitespace-pre-wrap my-2"><code>@misc{toyer2023tensor,
    title={{Tensor Trust}: Interpretable Prompt Injection Attacks from an Online Game},
    author={Toyer, Sam and Watkins, Olivia and Mendes, Ethan Adrian and Svegliato, Justin and Bailey, Luke and Wang, Tiffany and Ong, Isaac and Elmaaroufi, Karim and Abbeel, Pieter and Darrell, Trevor and Ritter, Alan and Russell, Stuart},
    year={2023},
    journal={arXiv preprint arXiv:2311.01011},
    url={https://arxiv.org/pdf/2311.01011.pdf}
}</code></pre>
        </details>
      </div>
    </div>
  </body>
</html>
