Toggle Menu
Prev Up Next

# `langchain.evaluation.regex_match.base`.RegexMatchStringEvaluator

class langchain.evaluation.regex_match.base.RegexMatchStringEvaluator(*, flags: int = 0, **kwargs: Any)

  Compute a regex match between the prediction and the reference.

  Examples

      >>> evaluator = RegexMatchStringEvaluator(flags=re.IGNORECASE)
      >>> evaluator.evaluate_strings(
              prediction="Mindy is the CTO",
              reference="^mindy.*cto$",
          )  # This will return {'score': 1.0} due to the IGNORECASE flag

      >>> evaluator = RegexMatchStringEvaluator()
      >>> evaluator.evaluate_strings(
              prediction="Mindy is the CTO",
              reference="^Mike.*CEO$",
          )  # This will return {'score': 0.0}

      >>> evaluator.evaluate_strings(
              prediction="Mindy is the CTO",
              reference="^Mike.*CEO$|^Mindy.*CTO$",
          )  # This will return {'score': 1.0} as the prediction matches the second pattern in the union

  Attributes

  `evaluation_name` | Get the evaluation name.
  `input_keys` | Get the input keys.
  `requires_input` | This evaluator does not require input.
  `requires_reference` | This evaluator requires a reference.

  Methods

  `__init__`(*[, flags]) |
  `aevaluate_strings`(*, prediction[, ...]) | Asynchronously evaluate Chain or LLM output, based on optional input and label.
  `evaluate_strings`(*, prediction[, reference, ...]) | Evaluate Chain or LLM output, based on optional input and label.

  Parameters

    - flags (int) –

    - kwargs (Any) –

  __init__(*, flags: int = 0, **kwargs: Any)
    Parameters

      - flags (int) –

      - kwargs (Any) –

  async aevaluate_strings(*, prediction: str, reference: Optional[str] = None, input: Optional[str] = None, **kwargs: Any) → dict

    Asynchronously evaluate Chain or LLM output, based on optional input and label.

    Parameters

      - prediction (str) – The LLM or chain prediction to evaluate.

      - reference (Optional[str], optional) – The reference label to evaluate against.

      - input (Optional[str], optional) – The input to consider during evaluation.

      - **kwargs – Additional keyword arguments, including callbacks, tags, etc.

    Returns

      The evaluation results containing the score or value.

    Return type

      dict

  evaluate_strings(*, prediction: str, reference: Optional[str] = None, input: Optional[str] = None, **kwargs: Any) → dict

    Evaluate Chain or LLM output, based on optional input and label.

    Parameters

      - prediction (str) – The LLM or chain prediction to evaluate.

      - reference (Optional[str], optional) – The reference label to evaluate against.

      - input (Optional[str], optional) – The input to consider during evaluation.

      - **kwargs – Additional keyword arguments, including callbacks, tags, etc.

    Returns

      The evaluation results containing the score or value.

    Return type

      dict

# Examples using RegexMatchStringEvaluator

- Check for the presence of a YYYY-MM-DD string.