Toggle Menu
Prev Up Next

# `langchain.evaluation.schema`.AgentTrajectoryEvaluator

class langchain.evaluation.schema.AgentTrajectoryEvaluator

  Interface for evaluating agent trajectories.

  Attributes

  `requires_input` | Whether this evaluator requires an input string.
  `requires_reference` | Whether this evaluator requires a reference label.

  Methods

  `__init__`() |
  `aevaluate_agent_trajectory`(*, prediction, ...) | Asynchronously evaluate a trajectory.
  `evaluate_agent_trajectory`(*, prediction, ...) | Evaluate a trajectory.

  __init__()
  async aevaluate_agent_trajectory(*, prediction: str, agent_trajectory: Sequence[Tuple[AgentAction, str]], input: str, reference: Optional[str] = None, **kwargs: Any) → dict

    Asynchronously evaluate a trajectory.

    Parameters

      - prediction (str) – The final predicted response.

      - agent_trajectory (List[Tuple[AgentAction, str]]) – The intermediate steps forming the agent trajectory.

      - input (str) – The input to the agent.

      - reference (Optional[str]) – The reference answer.

      - kwargs (Any) –

    Returns

      The evaluation result.

    Return type

      dict

  evaluate_agent_trajectory(*, prediction: str, agent_trajectory: Sequence[Tuple[AgentAction, str]], input: str, reference: Optional[str] = None, **kwargs: Any) → dict

    Evaluate a trajectory.

    Parameters

      - prediction (str) – The final predicted response.

      - agent_trajectory (List[Tuple[AgentAction, str]]) – The intermediate steps forming the agent trajectory.

      - input (str) – The input to the agent.

      - reference (Optional[str]) – The reference answer.

      - kwargs (Any) –

    Returns

      The evaluation result.

    Return type

      dict

# Examples using AgentTrajectoryEvaluator

- custom.md