---
title: "Webhook Evaluators"
---

Sometimes, custom evaluators on Agenta may not be sufficient for your specific use case, or you may wish to host your own evaluators. In such cases, you can create a webhook evaluator to suit your specific needs. Webhook evaluators only need to comply with a predefined API.

:::info
For the moment, there are limitation on the security measures related to webhook evaluators. There is no authentication and no authorization, and the endpoint must be publicly available.
:::

## Webhook API

The input parameters are as follows:

1. `inputs`: A dictionary containing the inputs to the webhook evaluators.
2. `output`: The output generated by the LLM app.
3. `correct_answer`: The expected correct answer for the LLM app.

The webhook request body has the following signature:

```json
{
    "inputs": { "{input_1}": <String> [,"{input_2}": <String>, ...] }
    "output": <String>,
    "correct_answer": <String>
}
```

The webhook should return the following:

`score`: A float value corresponding to the score of the evaluation. The
score should be between 0 and 1, with 0 meaning the evaluation failed, with its
lowest score, and 1 meaning the evaluation succeeded, with its highest score.

The webhook response body should have this signature:

```json
{
    "score": <Float>
}
```

The webhook response `status` must be `200` for successful evaluations, the webhook response `Content-Type` header should be set to `application/json`, and the webhook response body must be a properly-formatted JSON object.

## Example Implementation

Here's a simple webhook evaluator implementation that returns `1.0` when the lowercase `output` matches the lowercase `correct answer`. You can use this webhook for testing by copying [the endpoint](https://mmabrouk-agenta_webhook_eval.web.val.run). Then fork and modify it for your specific use case:

<iframe
  width="100%"
  height="500px"
  src="https://www.val.town/embed/mmabrouk/agenta_webhook_eval"
  title="Example webhook evaluator"
  frameborder="0"
  allow="web-share"
  allowfullscreen
></iframe>
