Skip to main content
`Ctrl`+`K`

# LLMs

Wrappers on top of large language models APIs.

pydantic model langchain.llms.AI21

  Wrapper around AI21 large language models.

  To use, you should have the environment variable `AI21_API_KEY` set with your API key.

  Example

      from langchain.llms import AI21
      ai21 = AI21(model="j2-jumbo-instruct")

  Validators

    - `set_callback_manager` » `callback_manager`

    - `set_verbose` » `verbose`

    - `validate_environment` » `all fields`

  field base_url: Optional[str] = None

    Base url to use, if None decides based on model name.

  field countPenalty: langchain.llms.ai21.AI21PenaltyData = AI21PenaltyData(scale=0, applyToWhitespaces=True, applyToPunctuations=True, applyToNumbers=True, applyToStopwords=True, applyToEmojis=True)

    Penalizes repeated tokens according to count.

  field frequencyPenalty: langchain.llms.ai21.AI21PenaltyData = AI21PenaltyData(scale=0, applyToWhitespaces=True, applyToPunctuations=True, applyToNumbers=True, applyToStopwords=True, applyToEmojis=True)

    Penalizes repeated tokens according to frequency.

  field logitBias: Optional[Dict[str, float]] = None

    Adjust the probability of specific tokens being generated.

  field maxTokens: int = 256

    The maximum number of tokens to generate in the completion.

  field minTokens: int = 0

    The minimum number of tokens to generate in the completion.

  field model: str = 'j2-jumbo-instruct'

    Model name to use.

  field numResults: int = 1

    How many completions to generate for each prompt.

  field presencePenalty: langchain.llms.ai21.AI21PenaltyData = AI21PenaltyData(scale=0, applyToWhitespaces=True, applyToPunctuations=True, applyToNumbers=True, applyToStopwords=True, applyToEmojis=True)

    Penalizes repeated tokens.

  field temperature: float = 0.7

    What sampling temperature to use.

  field topP: float = 1.0

    Total probability mass of tokens to consider at each step.

  __call__(prompt: str, stop: Optional[List[str]] = None) → str

    Check Cache and run the LLM on the given prompt and input.

  async agenerate(prompts: List[str], stop: Optional[List[str]] = None) → langchain.schema.LLMResult

    Run the LLM on the given prompt and input.

  async agenerate_prompt(prompts: List[langchain.schema.PromptValue], stop: Optional[List[str]] = None) → langchain.schema.LLMResult

    Take in a list of prompt values and return an LLMResult.

  classmethod construct(_fields_set: Optional[SetStr] = None, **values: Any) → Model

    Creates a new model setting __dict__ and __fields_set__ from trusted or pre-validated data. Default values are respected, but no other validation is performed. Behaves as if Config.extra = ‘allow’ was set since it adds all passed values

  copy(*, include: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, exclude: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, update: Optional[DictStrAny] = None, deep: bool = False) → Model

    Duplicate a model, optionally choose which fields to include, exclude and change.

    Parameters

      - include – fields to include in new model

      - exclude – fields to exclude from new model, as with values this takes precedence over include

      - update – values to change/add in the new model. Note: the data is not validated before creating the new model: you should trust this data

      - deep – set to True to make a deep copy of the model

    Returns

      new model instance

  dict(**kwargs: Any) → Dict

    Return a dictionary of the LLM.

  generate(prompts: List[str], stop: Optional[List[str]] = None) → langchain.schema.LLMResult

    Run the LLM on the given prompt and input.

  generate_prompt(prompts: List[langchain.schema.PromptValue], stop: Optional[List[str]] = None) → langchain.schema.LLMResult

    Take in a list of prompt values and return an LLMResult.

  get_num_tokens(text: str) → int

    Get the number of tokens present in the text.

  get_num_tokens_from_messages(messages: List[langchain.schema.BaseMessage]) → int

    Get the number of tokens in the message.

  json(*, include: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, exclude: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, by_alias: bool = False, skip_defaults: Optional[bool] = None, exclude_unset: bool = False, exclude_defaults: bool = False, exclude_none: bool = False, encoder: Optional[Callable[[Any], Any]] = None, models_as_dict: bool = True, **dumps_kwargs: Any) → unicode

    Generate a JSON representation of the model, include and exclude arguments as per dict().

    encoder is an optional function to supply as default to json.dumps(), other arguments as per json.dumps().

  save(file_path: Union[pathlib.Path, str]) → None

    Save the LLM.

    Parameters

      file_path – Path to file to save the LLM to.

    Example: .. code-block:: python

    > llm.save(file_path=”path/llm.yaml”)

  classmethod update_forward_refs(**localns: Any) → None

    Try to update ForwardRefs on fields based on this Model, globalns and localns.

pydantic model langchain.llms.AlephAlpha

  Wrapper around Aleph Alpha large language models.

  To use, you should have the `aleph_alpha_client` python package installed, and the environment variable `ALEPH_ALPHA_API_KEY` set with your API key, or pass it as a named parameter to the constructor.

  Parameters are explained more in depth here: Aleph-Alpha/aleph-alpha-client

  Example

      from langchain.llms import AlephAlpha
      alpeh_alpha = AlephAlpha(aleph_alpha_api_key="my-api-key")

  Validators

    - `set_callback_manager` » `callback_manager`

    - `set_verbose` » `verbose`

    - `validate_environment` » `all fields`

  field aleph_alpha_api_key: Optional[str] = None

    API key for Aleph Alpha API.

  field best_of: Optional[int] = None

    returns the one with the “best of” results (highest log probability per token)

  field completion_bias_exclusion_first_token_only: bool = False

    Only consider the first token for the completion_bias_exclusion.

  field contextual_control_threshold: Optional[float] = None

    If set to None, attention control parameters only apply to those tokens that have explicitly been set in the request. If set to a non-None value, control parameters are also applied to similar tokens.

  field control_log_additive: Optional[bool] = True

    True: apply control by adding the log(control_factor) to attention scores. False: (attention_scores - - attention_scores.min(-1)) * control_factor

  field echo: bool = False

    Echo the prompt in the completion.

  field frequency_penalty: float = 0.0

    Penalizes repeated tokens according to frequency.

  field log_probs: Optional[int] = None

    Number of top log probabilities to be returned for each generated token.

  field logit_bias: Optional[Dict[int, float]] = None

    The logit bias allows to influence the likelihood of generating tokens.

  field maximum_tokens: int = 64

    The maximum number of tokens to be generated.

  field minimum_tokens: Optional[int] = 0

    Generate at least this number of tokens.

  field model: Optional[str] = 'luminous-base'

    Model name to use.

  field n: int = 1

    How many completions to generate for each prompt.

  field penalty_bias: Optional[str] = None

    Penalty bias for the completion.

  field penalty_exceptions: Optional[List[str]] = None

    List of strings that may be generated without penalty, regardless of other penalty settings

  field penalty_exceptions_include_stop_sequences: Optional[bool] = None

    Should stop_sequences be included in penalty_exceptions.

  field presence_penalty: float = 0.0

    Penalizes repeated tokens.

  field raw_completion: bool = False

    Force the raw completion of the model to be returned.

  field repetition_penalties_include_completion: bool = True

    Flag deciding whether presence penalty or frequency penalty are updated from the completion.

  field repetition_penalties_include_prompt: Optional[bool] = False

    Flag deciding whether presence penalty or frequency penalty are updated from the prompt.

  field stop_sequences: Optional[List[str]] = None

    Stop sequences to use.

  field temperature: float = 0.0

    A non-negative float that tunes the degree of randomness in generation.

  field tokens: Optional[bool] = False

    return tokens of completion.

  field top_k: int = 0

    Number of most likely tokens to consider at each step.

  field top_p: float = 0.0

    Total probability mass of tokens to consider at each step.

  field use_multiplicative_presence_penalty: Optional[bool] = False

    Flag deciding whether presence penalty is applied multiplicatively (True) or additively (False).

  __call__(prompt: str, stop: Optional[List[str]] = None) → str

    Check Cache and run the LLM on the given prompt and input.

  async agenerate(prompts: List[str], stop: Optional[List[str]] = None) → langchain.schema.LLMResult

    Run the LLM on the given prompt and input.

  async agenerate_prompt(prompts: List[langchain.schema.PromptValue], stop: Optional[List[str]] = None) → langchain.schema.LLMResult

    Take in a list of prompt values and return an LLMResult.

  classmethod construct(_fields_set: Optional[SetStr] = None, **values: Any) → Model

    Creates a new model setting __dict__ and __fields_set__ from trusted or pre-validated data. Default values are respected, but no other validation is performed. Behaves as if Config.extra = ‘allow’ was set since it adds all passed values

  copy(*, include: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, exclude: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, update: Optional[DictStrAny] = None, deep: bool = False) → Model

    Duplicate a model, optionally choose which fields to include, exclude and change.

    Parameters

      - include – fields to include in new model

      - exclude – fields to exclude from new model, as with values this takes precedence over include

      - update – values to change/add in the new model. Note: the data is not validated before creating the new model: you should trust this data

      - deep – set to True to make a deep copy of the model

    Returns

      new model instance

  dict(**kwargs: Any) → Dict

    Return a dictionary of the LLM.

  generate(prompts: List[str], stop: Optional[List[str]] = None) → langchain.schema.LLMResult

    Run the LLM on the given prompt and input.

  generate_prompt(prompts: List[langchain.schema.PromptValue], stop: Optional[List[str]] = None) → langchain.schema.LLMResult

    Take in a list of prompt values and return an LLMResult.

  get_num_tokens(text: str) → int

    Get the number of tokens present in the text.

  get_num_tokens_from_messages(messages: List[langchain.schema.BaseMessage]) → int

    Get the number of tokens in the message.

  json(*, include: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, exclude: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, by_alias: bool = False, skip_defaults: Optional[bool] = None, exclude_unset: bool = False, exclude_defaults: bool = False, exclude_none: bool = False, encoder: Optional[Callable[[Any], Any]] = None, models_as_dict: bool = True, **dumps_kwargs: Any) → unicode

    Generate a JSON representation of the model, include and exclude arguments as per dict().

    encoder is an optional function to supply as default to json.dumps(), other arguments as per json.dumps().

  save(file_path: Union[pathlib.Path, str]) → None

    Save the LLM.

    Parameters

      file_path – Path to file to save the LLM to.

    Example: .. code-block:: python

    > llm.save(file_path=”path/llm.yaml”)

  classmethod update_forward_refs(**localns: Any) → None

    Try to update ForwardRefs on fields based on this Model, globalns and localns.

pydantic model langchain.llms.Anthropic

  Wrapper around Anthropic’s large language models.

  To use, you should have the `anthropic` python package installed, and the environment variable `ANTHROPIC_API_KEY` set with your API key, or pass it as a named parameter to the constructor.

  Example

  Validators

    - `set_callback_manager` » `callback_manager`

    - `set_verbose` » `verbose`

    - `validate_environment` » `all fields`

  field max_tokens_to_sample: int = 256

    Denotes the number of tokens to predict per generation.

  field model: str = 'claude-v1'

    Model name to use.

  field streaming: bool = False

    Whether to stream the results.

  field temperature: Optional[float] = None

    A non-negative float that tunes the degree of randomness in generation.

  field top_k: Optional[int] = None

    Number of most likely tokens to consider at each step.

  field top_p: Optional[float] = None

    Total probability mass of tokens to consider at each step.

  __call__(prompt: str, stop: Optional[List[str]] = None) → str

    Check Cache and run the LLM on the given prompt and input.

  async agenerate(prompts: List[str], stop: Optional[List[str]] = None) → langchain.schema.LLMResult

    Run the LLM on the given prompt and input.

  async agenerate_prompt(prompts: List[langchain.schema.PromptValue], stop: Optional[List[str]] = None) → langchain.schema.LLMResult

    Take in a list of prompt values and return an LLMResult.

  classmethod construct(_fields_set: Optional[SetStr] = None, **values: Any) → Model

    Creates a new model setting __dict__ and __fields_set__ from trusted or pre-validated data. Default values are respected, but no other validation is performed. Behaves as if Config.extra = ‘allow’ was set since it adds all passed values

  copy(*, include: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, exclude: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, update: Optional[DictStrAny] = None, deep: bool = False) → Model

    Duplicate a model, optionally choose which fields to include, exclude and change.

    Parameters

      - include – fields to include in new model

      - exclude – fields to exclude from new model, as with values this takes precedence over include

      - update – values to change/add in the new model. Note: the data is not validated before creating the new model: you should trust this data

      - deep – set to True to make a deep copy of the model

    Returns

      new model instance

  dict(**kwargs: Any) → Dict

    Return a dictionary of the LLM.

  generate(prompts: List[str], stop: Optional[List[str]] = None) → langchain.schema.LLMResult

    Run the LLM on the given prompt and input.

  generate_prompt(prompts: List[langchain.schema.PromptValue], stop: Optional[List[str]] = None) → langchain.schema.LLMResult

    Take in a list of prompt values and return an LLMResult.

  get_num_tokens(text: str) → int

    Get the number of tokens present in the text.

  get_num_tokens_from_messages(messages: List[langchain.schema.BaseMessage]) → int

    Get the number of tokens in the message.

  json(*, include: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, exclude: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, by_alias: bool = False, skip_defaults: Optional[bool] = None, exclude_unset: bool = False, exclude_defaults: bool = False, exclude_none: bool = False, encoder: Optional[Callable[[Any], Any]] = None, models_as_dict: bool = True, **dumps_kwargs: Any) → unicode

    Generate a JSON representation of the model, include and exclude arguments as per dict().

    encoder is an optional function to supply as default to json.dumps(), other arguments as per json.dumps().

  save(file_path: Union[pathlib.Path, str]) → None

    Save the LLM.

    Parameters

      file_path – Path to file to save the LLM to.

    Example: .. code-block:: python

    > llm.save(file_path=”path/llm.yaml”)

  stream(prompt: str, stop: Optional[List[str]] = None) → Generator

    Call Anthropic completion_stream and return the resulting generator.

    BETA: this is a beta feature while we figure out the right abstraction. Once that happens, this interface could change.

    Parameters

      - prompt – The prompt to pass into the model.

      - stop – Optional list of stop words to use when generating.

    Returns

      A generator representing the stream of tokens from Anthropic.

    Example

        prompt = "Write a poem about a stream."
        prompt = f"\n\nHuman: {prompt}\n\nAssistant:"
        generator = anthropic.stream(prompt)
        for token in generator:
            yield token

  classmethod update_forward_refs(**localns: Any) → None

    Try to update ForwardRefs on fields based on this Model, globalns and localns.

pydantic model langchain.llms.AzureOpenAI

  Wrapper around Azure-specific OpenAI large language models.

  To use, you should have the `openai` python package installed, and the environment variable `OPENAI_API_KEY` set with your API key.

  Any parameters that are valid to be passed to the openai.create call can be passed in, even if not explicitly saved on this class.

  Example

      from langchain.llms import AzureOpenAI
      openai = AzureOpenAI(model_name="text-davinci-003")

  Validators

    - `build_extra` » `all fields`

    - `set_callback_manager` » `callback_manager`

    - `set_verbose` » `verbose`

    - `validate_environment` » `all fields`

  field allowed_special: Union[Literal['all'], AbstractSet[str]] = {}

    Set of special tokens that are allowed。

  field batch_size: int = 20

    Batch size to use when passing multiple documents to generate.

  field best_of: int = 1

    Generates best_of completions server-side and returns the “best”.

  field deployment_name: str = ''

    Deployment name to use.

  field disallowed_special: Union[Literal['all'], Collection[str]] = 'all'

    Set of special tokens that are not allowed。

  field frequency_penalty: float = 0

    Penalizes repeated tokens according to frequency.

  field logit_bias: Optional[Dict[str, float]] [Optional]

    Adjust the probability of specific tokens being generated.

  field max_retries: int = 6

    Maximum number of retries to make when generating.

  field max_tokens: int = 256

    The maximum number of tokens to generate in the completion. -1 returns as many tokens as possible given the prompt and the models maximal context size.

  field model_kwargs: Dict[str, Any] [Optional]

    Holds any model parameters valid for create call not explicitly specified.

  field model_name: str = 'text-davinci-003'

    Model name to use.

  field n: int = 1

    How many completions to generate for each prompt.

  field presence_penalty: float = 0

    Penalizes repeated tokens.

  field request_timeout: Optional[Union[float, Tuple[float, float]]] = None

    Timeout for requests to OpenAI completion API. Default is 600 seconds.

  field streaming: bool = False

    Whether to stream the results or not.

  field temperature: float = 0.7

    What sampling temperature to use.

  field top_p: float = 1

    Total probability mass of tokens to consider at each step.

  field verbose: bool [Optional]

    Whether to print out response text.

  __call__(prompt: str, stop: Optional[List[str]] = None) → str

    Check Cache and run the LLM on the given prompt and input.

  async agenerate(prompts: List[str], stop: Optional[List[str]] = None) → langchain.schema.LLMResult

    Run the LLM on the given prompt and input.

  async agenerate_prompt(prompts: List[langchain.schema.PromptValue], stop: Optional[List[str]] = None) → langchain.schema.LLMResult

    Take in a list of prompt values and return an LLMResult.

  classmethod construct(_fields_set: Optional[SetStr] = None, **values: Any) → Model

    Creates a new model setting __dict__ and __fields_set__ from trusted or pre-validated data. Default values are respected, but no other validation is performed. Behaves as if Config.extra = ‘allow’ was set since it adds all passed values

  copy(*, include: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, exclude: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, update: Optional[DictStrAny] = None, deep: bool = False) → Model

    Duplicate a model, optionally choose which fields to include, exclude and change.

    Parameters

      - include – fields to include in new model

      - exclude – fields to exclude from new model, as with values this takes precedence over include

      - update – values to change/add in the new model. Note: the data is not validated before creating the new model: you should trust this data

      - deep – set to True to make a deep copy of the model

    Returns

      new model instance

  create_llm_result(choices: Any, prompts: List[str], token_usage: Dict[str, int]) → langchain.schema.LLMResult

    Create the LLMResult from the choices and prompts.

  dict(**kwargs: Any) → Dict

    Return a dictionary of the LLM.

  generate(prompts: List[str], stop: Optional[List[str]] = None) → langchain.schema.LLMResult

    Run the LLM on the given prompt and input.

  generate_prompt(prompts: List[langchain.schema.PromptValue], stop: Optional[List[str]] = None) → langchain.schema.LLMResult

    Take in a list of prompt values and return an LLMResult.

  get_num_tokens(text: str) → int

    Calculate num tokens with tiktoken package.

  get_num_tokens_from_messages(messages: List[langchain.schema.BaseMessage]) → int

    Get the number of tokens in the message.

  get_sub_prompts(params: Dict[str, Any], prompts: List[str], stop: Optional[List[str]] = None) → List[List[str]]

    Get the sub prompts for llm call.

  json(*, include: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, exclude: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, by_alias: bool = False, skip_defaults: Optional[bool] = None, exclude_unset: bool = False, exclude_defaults: bool = False, exclude_none: bool = False, encoder: Optional[Callable[[Any], Any]] = None, models_as_dict: bool = True, **dumps_kwargs: Any) → unicode

    Generate a JSON representation of the model, include and exclude arguments as per dict().

    encoder is an optional function to supply as default to json.dumps(), other arguments as per json.dumps().

  max_tokens_for_prompt(prompt: str) → int

    Calculate the maximum number of tokens possible to generate for a prompt.

    Parameters

      prompt – The prompt to pass into the model.

    Returns

      The maximum number of tokens to generate for a prompt.

    Example

        max_tokens = openai.max_token_for_prompt("Tell me a joke.")

  modelname_to_contextsize(modelname: str) → int

    Calculate the maximum number of tokens possible to generate for a model.

    Parameters

      modelname – The modelname we want to know the context size for.

    Returns

      The maximum context size

    Example

        max_tokens = openai.modelname_to_contextsize("text-davinci-003")

  prep_streaming_params(stop: Optional[List[str]] = None) → Dict[str, Any]

    Prepare the params for streaming.

  save(file_path: Union[pathlib.Path, str]) → None

    Save the LLM.

    Parameters

      file_path – Path to file to save the LLM to.

    Example: .. code-block:: python

    > llm.save(file_path=”path/llm.yaml”)

  stream(prompt: str, stop: Optional[List[str]] = None) → Generator

    Call OpenAI with streaming flag and return the resulting generator.

    BETA: this is a beta feature while we figure out the right abstraction. Once that happens, this interface could change.

    Parameters

      - prompt – The prompts to pass into the model.

      - stop – Optional list of stop words to use when generating.

    Returns

      A generator representing the stream of tokens from OpenAI.

    Example

        generator = openai.stream("Tell me a joke.")
        for token in generator:
            yield token

  classmethod update_forward_refs(**localns: Any) → None

    Try to update ForwardRefs on fields based on this Model, globalns and localns.

pydantic model langchain.llms.Banana

  Wrapper around Banana large language models.

  To use, you should have the `banana-dev` python package installed, and the environment variable `BANANA_API_KEY` set with your API key.

  Any parameters that are valid to be passed to the call can be passed in, even if not explicitly saved on this class.

  Example

  Validators

    - `build_extra` » `all fields`

    - `set_callback_manager` » `callback_manager`

    - `set_verbose` » `verbose`

    - `validate_environment` » `all fields`

  field model_key: str = ''

    model endpoint to use

  field model_kwargs: Dict[str, Any] [Optional]

    Holds any model parameters valid for create call not explicitly specified.

  __call__(prompt: str, stop: Optional[List[str]] = None) → str

    Check Cache and run the LLM on the given prompt and input.

  async agenerate(prompts: List[str], stop: Optional[List[str]] = None) → langchain.schema.LLMResult

    Run the LLM on the given prompt and input.

  async agenerate_prompt(prompts: List[langchain.schema.PromptValue], stop: Optional[List[str]] = None) → langchain.schema.LLMResult

    Take in a list of prompt values and return an LLMResult.

  classmethod construct(_fields_set: Optional[SetStr] = None, **values: Any) → Model

    Creates a new model setting __dict__ and __fields_set__ from trusted or pre-validated data. Default values are respected, but no other validation is performed. Behaves as if Config.extra = ‘allow’ was set since it adds all passed values

  copy(*, include: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, exclude: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, update: Optional[DictStrAny] = None, deep: bool = False) → Model

    Duplicate a model, optionally choose which fields to include, exclude and change.

    Parameters

      - include – fields to include in new model

      - exclude – fields to exclude from new model, as with values this takes precedence over include

      - update – values to change/add in the new model. Note: the data is not validated before creating the new model: you should trust this data

      - deep – set to True to make a deep copy of the model

    Returns

      new model instance

  dict(**kwargs: Any) → Dict

    Return a dictionary of the LLM.

  generate(prompts: List[str], stop: Optional[List[str]] = None) → langchain.schema.LLMResult

    Run the LLM on the given prompt and input.

  generate_prompt(prompts: List[langchain.schema.PromptValue], stop: Optional[List[str]] = None) → langchain.schema.LLMResult

    Take in a list of prompt values and return an LLMResult.

  get_num_tokens(text: str) → int

    Get the number of tokens present in the text.

  get_num_tokens_from_messages(messages: List[langchain.schema.BaseMessage]) → int

    Get the number of tokens in the message.

  json(*, include: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, exclude: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, by_alias: bool = False, skip_defaults: Optional[bool] = None, exclude_unset: bool = False, exclude_defaults: bool = False, exclude_none: bool = False, encoder: Optional[Callable[[Any], Any]] = None, models_as_dict: bool = True, **dumps_kwargs: Any) → unicode

    Generate a JSON representation of the model, include and exclude arguments as per dict().

    encoder is an optional function to supply as default to json.dumps(), other arguments as per json.dumps().

  save(file_path: Union[pathlib.Path, str]) → None

    Save the LLM.

    Parameters

      file_path – Path to file to save the LLM to.

    Example: .. code-block:: python

    > llm.save(file_path=”path/llm.yaml”)

  classmethod update_forward_refs(**localns: Any) → None

    Try to update ForwardRefs on fields based on this Model, globalns and localns.

pydantic model langchain.llms.CerebriumAI

  Wrapper around CerebriumAI large language models.

  To use, you should have the `cerebrium` python package installed, and the environment variable `CEREBRIUMAI_API_KEY` set with your API key.

  Any parameters that are valid to be passed to the call can be passed in, even if not explicitly saved on this class.

  Example

  Validators

    - `build_extra` » `all fields`

    - `set_callback_manager` » `callback_manager`

    - `set_verbose` » `verbose`

    - `validate_environment` » `all fields`

  field endpoint_url: str = ''

    model endpoint to use

  field model_kwargs: Dict[str, Any] [Optional]

    Holds any model parameters valid for create call not explicitly specified.

  __call__(prompt: str, stop: Optional[List[str]] = None) → str

    Check Cache and run the LLM on the given prompt and input.

  async agenerate(prompts: List[str], stop: Optional[List[str]] = None) → langchain.schema.LLMResult

    Run the LLM on the given prompt and input.

  async agenerate_prompt(prompts: List[langchain.schema.PromptValue], stop: Optional[List[str]] = None) → langchain.schema.LLMResult

    Take in a list of prompt values and return an LLMResult.

  classmethod construct(_fields_set: Optional[SetStr] = None, **values: Any) → Model

    Creates a new model setting __dict__ and __fields_set__ from trusted or pre-validated data. Default values are respected, but no other validation is performed. Behaves as if Config.extra = ‘allow’ was set since it adds all passed values

  copy(*, include: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, exclude: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, update: Optional[DictStrAny] = None, deep: bool = False) → Model

    Duplicate a model, optionally choose which fields to include, exclude and change.

    Parameters

      - include – fields to include in new model

      - exclude – fields to exclude from new model, as with values this takes precedence over include

      - update – values to change/add in the new model. Note: the data is not validated before creating the new model: you should trust this data

      - deep – set to True to make a deep copy of the model

    Returns

      new model instance

  dict(**kwargs: Any) → Dict

    Return a dictionary of the LLM.

  generate(prompts: List[str], stop: Optional[List[str]] = None) → langchain.schema.LLMResult

    Run the LLM on the given prompt and input.

  generate_prompt(prompts: List[langchain.schema.PromptValue], stop: Optional[List[str]] = None) → langchain.schema.LLMResult

    Take in a list of prompt values and return an LLMResult.

  get_num_tokens(text: str) → int

    Get the number of tokens present in the text.

  get_num_tokens_from_messages(messages: List[langchain.schema.BaseMessage]) → int

    Get the number of tokens in the message.

  json(*, include: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, exclude: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, by_alias: bool = False, skip_defaults: Optional[bool] = None, exclude_unset: bool = False, exclude_defaults: bool = False, exclude_none: bool = False, encoder: Optional[Callable[[Any], Any]] = None, models_as_dict: bool = True, **dumps_kwargs: Any) → unicode

    Generate a JSON representation of the model, include and exclude arguments as per dict().

    encoder is an optional function to supply as default to json.dumps(), other arguments as per json.dumps().

  save(file_path: Union[pathlib.Path, str]) → None

    Save the LLM.

    Parameters

      file_path – Path to file to save the LLM to.

    Example: .. code-block:: python

    > llm.save(file_path=”path/llm.yaml”)

  classmethod update_forward_refs(**localns: Any) → None

    Try to update ForwardRefs on fields based on this Model, globalns and localns.

pydantic model langchain.llms.Cohere

  Wrapper around Cohere large language models.

  To use, you should have the `cohere` python package installed, and the environment variable `COHERE_API_KEY` set with your API key, or pass it as a named parameter to the constructor.

  Example

      from langchain.llms import Cohere
      cohere = Cohere(model="gptd-instruct-tft", cohere_api_key="my-api-key")

  Validators

    - `set_callback_manager` » `callback_manager`

    - `set_verbose` » `verbose`

    - `validate_environment` » `all fields`

  field frequency_penalty: float = 0.0

    Penalizes repeated tokens according to frequency. Between 0 and 1.

  field k: int = 0

    Number of most likely tokens to consider at each step.

  field max_tokens: int = 256

    Denotes the number of tokens to predict per generation.

  field model: Optional[str] = None

    Model name to use.

  field p: int = 1

    Total probability mass of tokens to consider at each step.

  field presence_penalty: float = 0.0

    Penalizes repeated tokens. Between 0 and 1.

  field temperature: float = 0.75

    A non-negative float that tunes the degree of randomness in generation.

  field truncate: Optional[str] = None

    Specify how the client handles inputs longer than the maximum token length: Truncate from START, END or NONE

  __call__(prompt: str, stop: Optional[List[str]] = None) → str

    Check Cache and run the LLM on the given prompt and input.

  async agenerate(prompts: List[str], stop: Optional[List[str]] = None) → langchain.schema.LLMResult

    Run the LLM on the given prompt and input.

  async agenerate_prompt(prompts: List[langchain.schema.PromptValue], stop: Optional[List[str]] = None) → langchain.schema.LLMResult

    Take in a list of prompt values and return an LLMResult.

  classmethod construct(_fields_set: Optional[SetStr] = None, **values: Any) → Model

    Creates a new model setting __dict__ and __fields_set__ from trusted or pre-validated data. Default values are respected, but no other validation is performed. Behaves as if Config.extra = ‘allow’ was set since it adds all passed values

  copy(*, include: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, exclude: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, update: Optional[DictStrAny] = None, deep: bool = False) → Model

    Duplicate a model, optionally choose which fields to include, exclude and change.

    Parameters

      - include – fields to include in new model

      - exclude – fields to exclude from new model, as with values this takes precedence over include

      - update – values to change/add in the new model. Note: the data is not validated before creating the new model: you should trust this data

      - deep – set to True to make a deep copy of the model

    Returns

      new model instance

  dict(**kwargs: Any) → Dict

    Return a dictionary of the LLM.

  generate(prompts: List[str], stop: Optional[List[str]] = None) → langchain.schema.LLMResult

    Run the LLM on the given prompt and input.

  generate_prompt(prompts: List[langchain.schema.PromptValue], stop: Optional[List[str]] = None) → langchain.schema.LLMResult

    Take in a list of prompt values and return an LLMResult.

  get_num_tokens(text: str) → int

    Get the number of tokens present in the text.

  get_num_tokens_from_messages(messages: List[langchain.schema.BaseMessage]) → int

    Get the number of tokens in the message.

  json(*, include: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, exclude: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, by_alias: bool = False, skip_defaults: Optional[bool] = None, exclude_unset: bool = False, exclude_defaults: bool = False, exclude_none: bool = False, encoder: Optional[Callable[[Any], Any]] = None, models_as_dict: bool = True, **dumps_kwargs: Any) → unicode

    Generate a JSON representation of the model, include and exclude arguments as per dict().

    encoder is an optional function to supply as default to json.dumps(), other arguments as per json.dumps().

  save(file_path: Union[pathlib.Path, str]) → None

    Save the LLM.

    Parameters

      file_path – Path to file to save the LLM to.

    Example: .. code-block:: python

    > llm.save(file_path=”path/llm.yaml”)

  classmethod update_forward_refs(**localns: Any) → None

    Try to update ForwardRefs on fields based on this Model, globalns and localns.

pydantic model langchain.llms.DeepInfra

  Wrapper around DeepInfra deployed models.

  To use, you should have the `requests` python package installed, and the environment variable `DEEPINFRA_API_TOKEN` set with your API token, or pass it as a named parameter to the constructor.

  Only supports text-generation and text2text-generation for now.

  Example

      from langchain.llms import DeepInfra
      di = DeepInfra(model_id="google/flan-t5-xl",
                          deepinfra_api_token="my-api-key")

  Validators

    - `set_callback_manager` » `callback_manager`

    - `set_verbose` » `verbose`

    - `validate_environment` » `all fields`

  __call__(prompt: str, stop: Optional[List[str]] = None) → str

    Check Cache and run the LLM on the given prompt and input.

  async agenerate(prompts: List[str], stop: Optional[List[str]] = None) → langchain.schema.LLMResult

    Run the LLM on the given prompt and input.

  async agenerate_prompt(prompts: List[langchain.schema.PromptValue], stop: Optional[List[str]] = None) → langchain.schema.LLMResult

    Take in a list of prompt values and return an LLMResult.

  classmethod construct(_fields_set: Optional[SetStr] = None, **values: Any) → Model

    Creates a new model setting __dict__ and __fields_set__ from trusted or pre-validated data. Default values are respected, but no other validation is performed. Behaves as if Config.extra = ‘allow’ was set since it adds all passed values

  copy(*, include: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, exclude: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, update: Optional[DictStrAny] = None, deep: bool = False) → Model

    Duplicate a model, optionally choose which fields to include, exclude and change.

    Parameters

      - include – fields to include in new model

      - exclude – fields to exclude from new model, as with values this takes precedence over include

      - update – values to change/add in the new model. Note: the data is not validated before creating the new model: you should trust this data

      - deep – set to True to make a deep copy of the model

    Returns

      new model instance

  dict(**kwargs: Any) → Dict

    Return a dictionary of the LLM.

  generate(prompts: List[str], stop: Optional[List[str]] = None) → langchain.schema.LLMResult

    Run the LLM on the given prompt and input.

  generate_prompt(prompts: List[langchain.schema.PromptValue], stop: Optional[List[str]] = None) → langchain.schema.LLMResult

    Take in a list of prompt values and return an LLMResult.

  get_num_tokens(text: str) → int

    Get the number of tokens present in the text.

  get_num_tokens_from_messages(messages: List[langchain.schema.BaseMessage]) → int

    Get the number of tokens in the message.

  json(*, include: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, exclude: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, by_alias: bool = False, skip_defaults: Optional[bool] = None, exclude_unset: bool = False, exclude_defaults: bool = False, exclude_none: bool = False, encoder: Optional[Callable[[Any], Any]] = None, models_as_dict: bool = True, **dumps_kwargs: Any) → unicode

    Generate a JSON representation of the model, include and exclude arguments as per dict().

    encoder is an optional function to supply as default to json.dumps(), other arguments as per json.dumps().

  save(file_path: Union[pathlib.Path, str]) → None

    Save the LLM.

    Parameters

      file_path – Path to file to save the LLM to.

    Example: .. code-block:: python

    > llm.save(file_path=”path/llm.yaml”)

  classmethod update_forward_refs(**localns: Any) → None

    Try to update ForwardRefs on fields based on this Model, globalns and localns.

pydantic model langchain.llms.ForefrontAI

  Wrapper around ForefrontAI large language models.

  To use, you should have the environment variable `FOREFRONTAI_API_KEY` set with your API key.

  Example

      from langchain.llms import ForefrontAI
      forefrontai = ForefrontAI(endpoint_url="")

  Validators

    - `set_callback_manager` » `callback_manager`

    - `set_verbose` » `verbose`

    - `validate_environment` » `all fields`

  field base_url: Optional[str] = None

    Base url to use, if None decides based on model name.

  field endpoint_url: str = ''

    Model name to use.

  field length: int = 256

    The maximum number of tokens to generate in the completion.

  field repetition_penalty: int = 1

    Penalizes repeated tokens according to frequency.

  field temperature: float = 0.7

    What sampling temperature to use.

  field top_k: int = 40

    The number of highest probability vocabulary tokens to keep for top-k-filtering.

  field top_p: float = 1.0

    Total probability mass of tokens to consider at each step.

  __call__(prompt: str, stop: Optional[List[str]] = None) → str

    Check Cache and run the LLM on the given prompt and input.

  async agenerate(prompts: List[str], stop: Optional[List[str]] = None) → langchain.schema.LLMResult

    Run the LLM on the given prompt and input.

  async agenerate_prompt(prompts: List[langchain.schema.PromptValue], stop: Optional[List[str]] = None) → langchain.schema.LLMResult

    Take in a list of prompt values and return an LLMResult.

  classmethod construct(_fields_set: Optional[SetStr] = None, **values: Any) → Model

    Creates a new model setting __dict__ and __fields_set__ from trusted or pre-validated data. Default values are respected, but no other validation is performed. Behaves as if Config.extra = ‘allow’ was set since it adds all passed values

  copy(*, include: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, exclude: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, update: Optional[DictStrAny] = None, deep: bool = False) → Model

    Duplicate a model, optionally choose which fields to include, exclude and change.

    Parameters

      - include – fields to include in new model

      - exclude – fields to exclude from new model, as with values this takes precedence over include

      - update – values to change/add in the new model. Note: the data is not validated before creating the new model: you should trust this data

      - deep – set to True to make a deep copy of the model

    Returns

      new model instance

  dict(**kwargs: Any) → Dict

    Return a dictionary of the LLM.

  generate(prompts: List[str], stop: Optional[List[str]] = None) → langchain.schema.LLMResult

    Run the LLM on the given prompt and input.

  generate_prompt(prompts: List[langchain.schema.PromptValue], stop: Optional[List[str]] = None) → langchain.schema.LLMResult

    Take in a list of prompt values and return an LLMResult.

  get_num_tokens(text: str) → int

    Get the number of tokens present in the text.

  get_num_tokens_from_messages(messages: List[langchain.schema.BaseMessage]) → int

    Get the number of tokens in the message.

  json(*, include: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, exclude: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, by_alias: bool = False, skip_defaults: Optional[bool] = None, exclude_unset: bool = False, exclude_defaults: bool = False, exclude_none: bool = False, encoder: Optional[Callable[[Any], Any]] = None, models_as_dict: bool = True, **dumps_kwargs: Any) → unicode

    Generate a JSON representation of the model, include and exclude arguments as per dict().

    encoder is an optional function to supply as default to json.dumps(), other arguments as per json.dumps().

  save(file_path: Union[pathlib.Path, str]) → None

    Save the LLM.

    Parameters

      file_path – Path to file to save the LLM to.

    Example: .. code-block:: python

    > llm.save(file_path=”path/llm.yaml”)

  classmethod update_forward_refs(**localns: Any) → None

    Try to update ForwardRefs on fields based on this Model, globalns and localns.

pydantic model langchain.llms.GPT4All

  Wrapper around GPT4All language models.

  To use, you should have the `pyllamacpp` python package installed, the pre-trained model file, and the model’s config information.

  Example

      from langchain.llms import GPT4All
      model = GPT4All(model="./models/gpt4all-model.bin", n_ctx=512, n_threads=8)

      # Simplest invocation
      response = model("Once upon a time, ")

  Validators

    - `set_callback_manager` » `callback_manager`

    - `set_verbose` » `verbose`

    - `validate_environment` » `all fields`

  field echo: Optional[bool] = False

    Whether to echo the prompt.

  field embedding: bool = False

    Use embedding mode only.

  field f16_kv: bool = False

    Use half-precision for key/value cache.

  field logits_all: bool = False

    Return logits for all tokens, not just the last token.

  field model: str [Required]

    Path to the pre-trained GPT4All model file.

  field n_batch: int = 1

    Batch size for prompt processing.

  field n_ctx: int = 512

    Token context window.

  field n_parts: int = -1

    Number of parts to split the model into. If -1, the number of parts is automatically determined.

  field n_predict: Optional[int] = 256

    The maximum number of tokens to generate.

  field n_threads: Optional[int] = 4

    Number of threads to use.

  field repeat_last_n: Optional[int] = 64

    Last n tokens to penalize

  field repeat_penalty: Optional[float] = 1.3

    The penalty to apply to repeated tokens.

  field seed: int = 0

    Seed. If -1, a random seed is used.

  field stop: Optional[List[str]] = []

    A list of strings to stop generation when encountered.

  field streaming: bool = False

    Whether to stream the results or not.

  field temp: Optional[float] = 0.8

    The temperature to use for sampling.

  field top_k: Optional[int] = 40

    The top-k value to use for sampling.

  field top_p: Optional[float] = 0.95

    The top-p value to use for sampling.

  field use_mlock: bool = False

    Force system to keep model in RAM.

  field vocab_only: bool = False

    Only load the vocabulary, no weights.

  __call__(prompt: str, stop: Optional[List[str]] = None) → str

    Check Cache and run the LLM on the given prompt and input.

  async agenerate(prompts: List[str], stop: Optional[List[str]] = None) → langchain.schema.LLMResult

    Run the LLM on the given prompt and input.

  async agenerate_prompt(prompts: List[langchain.schema.PromptValue], stop: Optional[List[str]] = None) → langchain.schema.LLMResult

    Take in a list of prompt values and return an LLMResult.

  classmethod construct(_fields_set: Optional[SetStr] = None, **values: Any) → Model

    Creates a new model setting __dict__ and __fields_set__ from trusted or pre-validated data. Default values are respected, but no other validation is performed. Behaves as if Config.extra = ‘allow’ was set since it adds all passed values

  copy(*, include: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, exclude: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, update: Optional[DictStrAny] = None, deep: bool = False) → Model

    Duplicate a model, optionally choose which fields to include, exclude and change.

    Parameters

      - include – fields to include in new model

      - exclude – fields to exclude from new model, as with values this takes precedence over include

      - update – values to change/add in the new model. Note: the data is not validated before creating the new model: you should trust this data

      - deep – set to True to make a deep copy of the model

    Returns

      new model instance

  dict(**kwargs: Any) → Dict

    Return a dictionary of the LLM.

  generate(prompts: List[str], stop: Optional[List[str]] = None) → langchain.schema.LLMResult

    Run the LLM on the given prompt and input.

  generate_prompt(prompts: List[langchain.schema.PromptValue], stop: Optional[List[str]] = None) → langchain.schema.LLMResult

    Take in a list of prompt values and return an LLMResult.

  get_num_tokens(text: str) → int

    Get the number of tokens present in the text.

  get_num_tokens_from_messages(messages: List[langchain.schema.BaseMessage]) → int

    Get the number of tokens in the message.

  json(*, include: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, exclude: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, by_alias: bool = False, skip_defaults: Optional[bool] = None, exclude_unset: bool = False, exclude_defaults: bool = False, exclude_none: bool = False, encoder: Optional[Callable[[Any], Any]] = None, models_as_dict: bool = True, **dumps_kwargs: Any) → unicode

    Generate a JSON representation of the model, include and exclude arguments as per dict().

    encoder is an optional function to supply as default to json.dumps(), other arguments as per json.dumps().

  save(file_path: Union[pathlib.Path, str]) → None

    Save the LLM.

    Parameters

      file_path – Path to file to save the LLM to.

    Example: .. code-block:: python

    > llm.save(file_path=”path/llm.yaml”)

  classmethod update_forward_refs(**localns: Any) → None

    Try to update ForwardRefs on fields based on this Model, globalns and localns.

pydantic model langchain.llms.GooseAI

  Wrapper around OpenAI large language models.

  To use, you should have the `openai` python package installed, and the environment variable `GOOSEAI_API_KEY` set with your API key.

  Any parameters that are valid to be passed to the openai.create call can be passed in, even if not explicitly saved on this class.

  Example

  Validators

    - `build_extra` » `all fields`

    - `set_callback_manager` » `callback_manager`

    - `set_verbose` » `verbose`

    - `validate_environment` » `all fields`

  field frequency_penalty: float = 0

    Penalizes repeated tokens according to frequency.

  field logit_bias: Optional[Dict[str, float]] [Optional]

    Adjust the probability of specific tokens being generated.

  field max_tokens: int = 256

    The maximum number of tokens to generate in the completion. -1 returns as many tokens as possible given the prompt and the models maximal context size.

  field min_tokens: int = 1

    The minimum number of tokens to generate in the completion.

  field model_kwargs: Dict[str, Any] [Optional]

    Holds any model parameters valid for create call not explicitly specified.

  field model_name: str = 'gpt-neo-20b'

    Model name to use

  field n: int = 1

    How many completions to generate for each prompt.

  field presence_penalty: float = 0

    Penalizes repeated tokens.

  field temperature: float = 0.7

    What sampling temperature to use

  field top_p: float = 1

    Total probability mass of tokens to consider at each step.

  __call__(prompt: str, stop: Optional[List[str]] = None) → str

    Check Cache and run the LLM on the given prompt and input.

  async agenerate(prompts: List[str], stop: Optional[List[str]] = None) → langchain.schema.LLMResult

    Run the LLM on the given prompt and input.

  async agenerate_prompt(prompts: List[langchain.schema.PromptValue], stop: Optional[List[str]] = None) → langchain.schema.LLMResult

    Take in a list of prompt values and return an LLMResult.

  classmethod construct(_fields_set: Optional[SetStr] = None, **values: Any) → Model

    Creates a new model setting __dict__ and __fields_set__ from trusted or pre-validated data. Default values are respected, but no other validation is performed. Behaves as if Config.extra = ‘allow’ was set since it adds all passed values

  copy(*, include: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, exclude: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, update: Optional[DictStrAny] = None, deep: bool = False) → Model

    Duplicate a model, optionally choose which fields to include, exclude and change.

    Parameters

      - include – fields to include in new model

      - exclude – fields to exclude from new model, as with values this takes precedence over include

      - update – values to change/add in the new model. Note: the data is not validated before creating the new model: you should trust this data

      - deep – set to True to make a deep copy of the model

    Returns

      new model instance

  dict(**kwargs: Any) → Dict

    Return a dictionary of the LLM.

  generate(prompts: List[str], stop: Optional[List[str]] = None) → langchain.schema.LLMResult

    Run the LLM on the given prompt and input.

  generate_prompt(prompts: List[langchain.schema.PromptValue], stop: Optional[List[str]] = None) → langchain.schema.LLMResult

    Take in a list of prompt values and return an LLMResult.

  get_num_tokens(text: str) → int

    Get the number of tokens present in the text.

  get_num_tokens_from_messages(messages: List[langchain.schema.BaseMessage]) → int

    Get the number of tokens in the message.

  json(*, include: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, exclude: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, by_alias: bool = False, skip_defaults: Optional[bool] = None, exclude_unset: bool = False, exclude_defaults: bool = False, exclude_none: bool = False, encoder: Optional[Callable[[Any], Any]] = None, models_as_dict: bool = True, **dumps_kwargs: Any) → unicode

    Generate a JSON representation of the model, include and exclude arguments as per dict().

    encoder is an optional function to supply as default to json.dumps(), other arguments as per json.dumps().

  save(file_path: Union[pathlib.Path, str]) → None

    Save the LLM.

    Parameters

      file_path – Path to file to save the LLM to.

    Example: .. code-block:: python

    > llm.save(file_path=”path/llm.yaml”)

  classmethod update_forward_refs(**localns: Any) → None

    Try to update ForwardRefs on fields based on this Model, globalns and localns.

pydantic model langchain.llms.HuggingFaceEndpoint

  Wrapper around HuggingFaceHub Inference Endpoints.

  To use, you should have the `huggingface_hub` python package installed, and the environment variable `HUGGINGFACEHUB_API_TOKEN` set with your API token, or pass it as a named parameter to the constructor.

  Only supports text-generation and text2text-generation for now.

  Example

      from langchain.llms import HuggingFaceEndpoint
      endpoint_url = (
          "https://abcdefghijklmnop.us-east-1.aws.endpoints.huggingface.cloud"
      )
      hf = HuggingFaceEndpoint(
          endpoint_url=endpoint_url,
          huggingfacehub_api_token="my-api-key"
      )

  Validators

    - `set_callback_manager` » `callback_manager`

    - `set_verbose` » `verbose`

    - `validate_environment` » `all fields`

  field endpoint_url: str = ''

    Endpoint URL to use.

  field model_kwargs: Optional[dict] = None

    Key word arguments to pass to the model.

  field task: Optional[str] = None

    Task to call the model with. Should be a task that returns generated_text.

  __call__(prompt: str, stop: Optional[List[str]] = None) → str

    Check Cache and run the LLM on the given prompt and input.

  async agenerate(prompts: List[str], stop: Optional[List[str]] = None) → langchain.schema.LLMResult

    Run the LLM on the given prompt and input.

  async agenerate_prompt(prompts: List[langchain.schema.PromptValue], stop: Optional[List[str]] = None) → langchain.schema.LLMResult

    Take in a list of prompt values and return an LLMResult.

  classmethod construct(_fields_set: Optional[SetStr] = None, **values: Any) → Model

    Creates a new model setting __dict__ and __fields_set__ from trusted or pre-validated data. Default values are respected, but no other validation is performed. Behaves as if Config.extra = ‘allow’ was set since it adds all passed values

  copy(*, include: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, exclude: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, update: Optional[DictStrAny] = None, deep: bool = False) → Model

    Duplicate a model, optionally choose which fields to include, exclude and change.

    Parameters

      - include – fields to include in new model

      - exclude – fields to exclude from new model, as with values this takes precedence over include

      - update – values to change/add in the new model. Note: the data is not validated before creating the new model: you should trust this data

      - deep – set to True to make a deep copy of the model

    Returns

      new model instance

  dict(**kwargs: Any) → Dict

    Return a dictionary of the LLM.

  generate(prompts: List[str], stop: Optional[List[str]] = None) → langchain.schema.LLMResult

    Run the LLM on the given prompt and input.

  generate_prompt(prompts: List[langchain.schema.PromptValue], stop: Optional[List[str]] = None) → langchain.schema.LLMResult

    Take in a list of prompt values and return an LLMResult.

  get_num_tokens(text: str) → int

    Get the number of tokens present in the text.

  get_num_tokens_from_messages(messages: List[langchain.schema.BaseMessage]) → int

    Get the number of tokens in the message.

  json(*, include: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, exclude: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, by_alias: bool = False, skip_defaults: Optional[bool] = None, exclude_unset: bool = False, exclude_defaults: bool = False, exclude_none: bool = False, encoder: Optional[Callable[[Any], Any]] = None, models_as_dict: bool = True, **dumps_kwargs: Any) → unicode

    Generate a JSON representation of the model, include and exclude arguments as per dict().

    encoder is an optional function to supply as default to json.dumps(), other arguments as per json.dumps().

  save(file_path: Union[pathlib.Path, str]) → None

    Save the LLM.

    Parameters

      file_path – Path to file to save the LLM to.

    Example: .. code-block:: python

    > llm.save(file_path=”path/llm.yaml”)

  classmethod update_forward_refs(**localns: Any) → None

    Try to update ForwardRefs on fields based on this Model, globalns and localns.

pydantic model langchain.llms.HuggingFaceHub

  Wrapper around HuggingFaceHub models.

  To use, you should have the `huggingface_hub` python package installed, and the environment variable `HUGGINGFACEHUB_API_TOKEN` set with your API token, or pass it as a named parameter to the constructor.

  Only supports text-generation and text2text-generation for now.

  Example

      from langchain.llms import HuggingFaceHub
      hf = HuggingFaceHub(repo_id="gpt2", huggingfacehub_api_token="my-api-key")

  Validators

    - `set_callback_manager` » `callback_manager`

    - `set_verbose` » `verbose`

    - `validate_environment` » `all fields`

  field model_kwargs: Optional[dict] = None

    Key word arguments to pass to the model.

  field repo_id: str = 'gpt2'

    Model name to use.

  field task: Optional[str] = None

    Task to call the model with. Should be a task that returns generated_text.

  __call__(prompt: str, stop: Optional[List[str]] = None) → str

    Check Cache and run the LLM on the given prompt and input.

  async agenerate(prompts: List[str], stop: Optional[List[str]] = None) → langchain.schema.LLMResult

    Run the LLM on the given prompt and input.

  async agenerate_prompt(prompts: List[langchain.schema.PromptValue], stop: Optional[List[str]] = None) → langchain.schema.LLMResult

    Take in a list of prompt values and return an LLMResult.

  classmethod construct(_fields_set: Optional[SetStr] = None, **values: Any) → Model

    Creates a new model setting __dict__ and __fields_set__ from trusted or pre-validated data. Default values are respected, but no other validation is performed. Behaves as if Config.extra = ‘allow’ was set since it adds all passed values

  copy(*, include: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, exclude: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, update: Optional[DictStrAny] = None, deep: bool = False) → Model

    Duplicate a model, optionally choose which fields to include, exclude and change.

    Parameters

      - include – fields to include in new model

      - exclude – fields to exclude from new model, as with values this takes precedence over include

      - update – values to change/add in the new model. Note: the data is not validated before creating the new model: you should trust this data

      - deep – set to True to make a deep copy of the model

    Returns

      new model instance

  dict(**kwargs: Any) → Dict

    Return a dictionary of the LLM.

  generate(prompts: List[str], stop: Optional[List[str]] = None) → langchain.schema.LLMResult

    Run the LLM on the given prompt and input.

  generate_prompt(prompts: List[langchain.schema.PromptValue], stop: Optional[List[str]] = None) → langchain.schema.LLMResult

    Take in a list of prompt values and return an LLMResult.

  get_num_tokens(text: str) → int

    Get the number of tokens present in the text.

  get_num_tokens_from_messages(messages: List[langchain.schema.BaseMessage]) → int

    Get the number of tokens in the message.

  json(*, include: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, exclude: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, by_alias: bool = False, skip_defaults: Optional[bool] = None, exclude_unset: bool = False, exclude_defaults: bool = False, exclude_none: bool = False, encoder: Optional[Callable[[Any], Any]] = None, models_as_dict: bool = True, **dumps_kwargs: Any) → unicode

    Generate a JSON representation of the model, include and exclude arguments as per dict().

    encoder is an optional function to supply as default to json.dumps(), other arguments as per json.dumps().

  save(file_path: Union[pathlib.Path, str]) → None

    Save the LLM.

    Parameters

      file_path – Path to file to save the LLM to.

    Example: .. code-block:: python

    > llm.save(file_path=”path/llm.yaml”)

  classmethod update_forward_refs(**localns: Any) → None

    Try to update ForwardRefs on fields based on this Model, globalns and localns.

pydantic model langchain.llms.HuggingFacePipeline

  Wrapper around HuggingFace Pipeline API.

  To use, you should have the `transformers` python package installed.

  Only supports text-generation and text2text-generation for now.

  Example using from_model_id:

        from langchain.llms import HuggingFacePipeline
        hf = HuggingFacePipeline.from_model_id(
            model_id="gpt2", task="text-generation"
        )

  Example passing pipeline in directly:

        from langchain.llms import HuggingFacePipeline
        from transformers import AutoModelForCausalLM, AutoTokenizer, pipeline

        model_id = "gpt2"
        tokenizer = AutoTokenizer.from_pretrained(model_id)
        model = AutoModelForCausalLM.from_pretrained(model_id)
        pipe = pipeline(
            "text-generation", model=model, tokenizer=tokenizer, max_new_tokens=10
        )
        hf = HuggingFacePipeline(pipeline=pipe)

  Validators

    - `set_callback_manager` » `callback_manager`

    - `set_verbose` » `verbose`

  field model_id: str = 'gpt2'

    Model name to use.

  field model_kwargs: Optional[dict] = None

    Key word arguments to pass to the model.

  __call__(prompt: str, stop: Optional[List[str]] = None) → str

    Check Cache and run the LLM on the given prompt and input.

  async agenerate(prompts: List[str], stop: Optional[List[str]] = None) → langchain.schema.LLMResult

    Run the LLM on the given prompt and input.

  async agenerate_prompt(prompts: List[langchain.schema.PromptValue], stop: Optional[List[str]] = None) → langchain.schema.LLMResult

    Take in a list of prompt values and return an LLMResult.

  classmethod construct(_fields_set: Optional[SetStr] = None, **values: Any) → Model

    Creates a new model setting __dict__ and __fields_set__ from trusted or pre-validated data. Default values are respected, but no other validation is performed. Behaves as if Config.extra = ‘allow’ was set since it adds all passed values

  copy(*, include: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, exclude: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, update: Optional[DictStrAny] = None, deep: bool = False) → Model

    Duplicate a model, optionally choose which fields to include, exclude and change.

    Parameters

      - include – fields to include in new model

      - exclude – fields to exclude from new model, as with values this takes precedence over include

      - update – values to change/add in the new model. Note: the data is not validated before creating the new model: you should trust this data

      - deep – set to True to make a deep copy of the model

    Returns

      new model instance

  dict(**kwargs: Any) → Dict

    Return a dictionary of the LLM.

  classmethod from_model_id(model_id: str, task: str, device: int = - 1, model_kwargs: Optional[dict] = None, **kwargs: Any) → langchain.llms.base.LLM

    Construct the pipeline object from model_id and task.

  generate(prompts: List[str], stop: Optional[List[str]] = None) → langchain.schema.LLMResult

    Run the LLM on the given prompt and input.

  generate_prompt(prompts: List[langchain.schema.PromptValue], stop: Optional[List[str]] = None) → langchain.schema.LLMResult

    Take in a list of prompt values and return an LLMResult.

  get_num_tokens(text: str) → int

    Get the number of tokens present in the text.

  get_num_tokens_from_messages(messages: List[langchain.schema.BaseMessage]) → int

    Get the number of tokens in the message.

  json(*, include: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, exclude: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, by_alias: bool = False, skip_defaults: Optional[bool] = None, exclude_unset: bool = False, exclude_defaults: bool = False, exclude_none: bool = False, encoder: Optional[Callable[[Any], Any]] = None, models_as_dict: bool = True, **dumps_kwargs: Any) → unicode

    Generate a JSON representation of the model, include and exclude arguments as per dict().

    encoder is an optional function to supply as default to json.dumps(), other arguments as per json.dumps().

  save(file_path: Union[pathlib.Path, str]) → None

    Save the LLM.

    Parameters

      file_path – Path to file to save the LLM to.

    Example: .. code-block:: python

    > llm.save(file_path=”path/llm.yaml”)

  classmethod update_forward_refs(**localns: Any) → None

    Try to update ForwardRefs on fields based on this Model, globalns and localns.

pydantic model langchain.llms.LlamaCpp

  Wrapper around the llama.cpp model.

  To use, you should have the llama-cpp-python library installed, and provide the path to the Llama model as a named parameter to the constructor. Check out: abetlen/llama-cpp-python

  Example

      from langchain.llms import LlamaCppEmbeddings
      llm = LlamaCppEmbeddings(model_path="/path/to/llama/model")

  Validators

    - `set_callback_manager` » `callback_manager`

    - `set_verbose` » `verbose`

    - `validate_environment` » `all fields`

  field echo: Optional[bool] = False

    Whether to echo the prompt.

  field f16_kv: bool = True

    Use half-precision for key/value cache.

  field last_n_tokens_size: Optional[int] = 64

    The number of tokens to look back when applying the repeat_penalty.

  field logits_all: bool = False

    Return logits for all tokens, not just the last token.

  field logprobs: Optional[int] = None

    The number of logprobs to return. If None, no logprobs are returned.

  field lora_base: Optional[str] = None

    The path to the Llama LoRA base model.

  field lora_path: Optional[str] = None

    The path to the Llama LoRA. If None, no LoRa is loaded.

  field max_tokens: Optional[int] = 256

    The maximum number of tokens to generate.

  field model_path: str [Required]

    The path to the Llama model file.

  field n_batch: Optional[int] = 8

    Number of tokens to process in parallel. Should be a number between 1 and n_ctx.

  field n_ctx: int = 512

    Token context window.

  field n_parts: int = -1

    Number of parts to split the model into. If -1, the number of parts is automatically determined.

  field n_threads: Optional[int] = None

    Number of threads to use. If None, the number of threads is automatically determined.

  field repeat_penalty: Optional[float] = 1.1

    The penalty to apply to repeated tokens.

  field seed: int = -1

    Seed. If -1, a random seed is used.

  field stop: Optional[List[str]] = []

    A list of strings to stop generation when encountered.

  field streaming: bool = True

    Whether to stream the results, token by token.

  field suffix: Optional[str] = None

    A suffix to append to the generated text. If None, no suffix is appended.

  field temperature: Optional[float] = 0.8

    The temperature to use for sampling.

  field top_k: Optional[int] = 40

    The top-k value to use for sampling.

  field top_p: Optional[float] = 0.95

    The top-p value to use for sampling.

  field use_mlock: bool = False

    Force system to keep model in RAM.

  field use_mmap: Optional[bool] = True

    Whether to keep the model loaded in RAM

  field vocab_only: bool = False

    Only load the vocabulary, no weights.

  __call__(prompt: str, stop: Optional[List[str]] = None) → str

    Check Cache and run the LLM on the given prompt and input.

  async agenerate(prompts: List[str], stop: Optional[List[str]] = None) → langchain.schema.LLMResult

    Run the LLM on the given prompt and input.

  async agenerate_prompt(prompts: List[langchain.schema.PromptValue], stop: Optional[List[str]] = None) → langchain.schema.LLMResult

    Take in a list of prompt values and return an LLMResult.

  classmethod construct(_fields_set: Optional[SetStr] = None, **values: Any) → Model

    Creates a new model setting __dict__ and __fields_set__ from trusted or pre-validated data. Default values are respected, but no other validation is performed. Behaves as if Config.extra = ‘allow’ was set since it adds all passed values

  copy(*, include: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, exclude: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, update: Optional[DictStrAny] = None, deep: bool = False) → Model

    Duplicate a model, optionally choose which fields to include, exclude and change.

    Parameters

      - include – fields to include in new model

      - exclude – fields to exclude from new model, as with values this takes precedence over include

      - update – values to change/add in the new model. Note: the data is not validated before creating the new model: you should trust this data

      - deep – set to True to make a deep copy of the model

    Returns

      new model instance

  dict(**kwargs: Any) → Dict

    Return a dictionary of the LLM.

  generate(prompts: List[str], stop: Optional[List[str]] = None) → langchain.schema.LLMResult

    Run the LLM on the given prompt and input.

  generate_prompt(prompts: List[langchain.schema.PromptValue], stop: Optional[List[str]] = None) → langchain.schema.LLMResult

    Take in a list of prompt values and return an LLMResult.

  get_num_tokens(text: str) → int

    Get the number of tokens present in the text.

  get_num_tokens_from_messages(messages: List[langchain.schema.BaseMessage]) → int

    Get the number of tokens in the message.

  json(*, include: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, exclude: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, by_alias: bool = False, skip_defaults: Optional[bool] = None, exclude_unset: bool = False, exclude_defaults: bool = False, exclude_none: bool = False, encoder: Optional[Callable[[Any], Any]] = None, models_as_dict: bool = True, **dumps_kwargs: Any) → unicode

    Generate a JSON representation of the model, include and exclude arguments as per dict().

    encoder is an optional function to supply as default to json.dumps(), other arguments as per json.dumps().

  save(file_path: Union[pathlib.Path, str]) → None

    Save the LLM.

    Parameters

      file_path – Path to file to save the LLM to.

    Example: .. code-block:: python

    > llm.save(file_path=”path/llm.yaml”)

  stream(prompt: str, stop: Optional[List[str]] = None) → Generator[Dict, None, None]

    Yields results objects as they are generated in real time.

    > BETA: this is a beta feature while we figure out the right abstraction: Once that happens, this interface could change.

    > It also calls the callback manager’s on_llm_new_token event with similar parameters to the OpenAI LLM class method of the same name.

    > Args:

    >   prompt: The prompts to pass into the model. stop: Optional list of stop words to use when generating.

    > Returns:

    >   A generator representing the stream of tokens being generated.

    > Yields:

    >   A dictionary like objects containing a string token and metadata. See llama-cpp-python docs and below for more.

    > Example:

    >       from langchain.llms import LlamaCpp
    >       llm = LlamaCpp(
    >           model_path="/path/to/local/model.bin",
    >           temperature = 0.5
    >       )
    >       for chunk in llm.stream("Ask 'Hi, how are you?' like a pirate:'",
    >               stop=["'","

    “]):

      result = chunk[“choices”][0] print(result[“text”], end=’’, flush=True)

  classmethod update_forward_refs(**localns: Any) → None

    Try to update ForwardRefs on fields based on this Model, globalns and localns.

pydantic model langchain.llms.Modal

  Wrapper around Modal large language models.

  To use, you should have the `modal-client` python package installed.

  Any parameters that are valid to be passed to the call can be passed in, even if not explicitly saved on this class.

  Example

  Validators

    - `build_extra` » `all fields`

    - `set_callback_manager` » `callback_manager`

    - `set_verbose` » `verbose`

  field endpoint_url: str = ''

    model endpoint to use

  field model_kwargs: Dict[str, Any] [Optional]

    Holds any model parameters valid for create call not explicitly specified.

  __call__(prompt: str, stop: Optional[List[str]] = None) → str

    Check Cache and run the LLM on the given prompt and input.

  async agenerate(prompts: List[str], stop: Optional[List[str]] = None) → langchain.schema.LLMResult

    Run the LLM on the given prompt and input.

  async agenerate_prompt(prompts: List[langchain.schema.PromptValue], stop: Optional[List[str]] = None) → langchain.schema.LLMResult

    Take in a list of prompt values and return an LLMResult.

  classmethod construct(_fields_set: Optional[SetStr] = None, **values: Any) → Model

    Creates a new model setting __dict__ and __fields_set__ from trusted or pre-validated data. Default values are respected, but no other validation is performed. Behaves as if Config.extra = ‘allow’ was set since it adds all passed values

  copy(*, include: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, exclude: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, update: Optional[DictStrAny] = None, deep: bool = False) → Model

    Duplicate a model, optionally choose which fields to include, exclude and change.

    Parameters

      - include – fields to include in new model

      - exclude – fields to exclude from new model, as with values this takes precedence over include

      - update – values to change/add in the new model. Note: the data is not validated before creating the new model: you should trust this data

      - deep – set to True to make a deep copy of the model

    Returns

      new model instance

  dict(**kwargs: Any) → Dict

    Return a dictionary of the LLM.

  generate(prompts: List[str], stop: Optional[List[str]] = None) → langchain.schema.LLMResult

    Run the LLM on the given prompt and input.

  generate_prompt(prompts: List[langchain.schema.PromptValue], stop: Optional[List[str]] = None) → langchain.schema.LLMResult

    Take in a list of prompt values and return an LLMResult.

  get_num_tokens(text: str) → int

    Get the number of tokens present in the text.

  get_num_tokens_from_messages(messages: List[langchain.schema.BaseMessage]) → int

    Get the number of tokens in the message.

  json(*, include: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, exclude: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, by_alias: bool = False, skip_defaults: Optional[bool] = None, exclude_unset: bool = False, exclude_defaults: bool = False, exclude_none: bool = False, encoder: Optional[Callable[[Any], Any]] = None, models_as_dict: bool = True, **dumps_kwargs: Any) → unicode

    Generate a JSON representation of the model, include and exclude arguments as per dict().

    encoder is an optional function to supply as default to json.dumps(), other arguments as per json.dumps().

  save(file_path: Union[pathlib.Path, str]) → None

    Save the LLM.

    Parameters

      file_path – Path to file to save the LLM to.

    Example: .. code-block:: python

    > llm.save(file_path=”path/llm.yaml”)

  classmethod update_forward_refs(**localns: Any) → None

    Try to update ForwardRefs on fields based on this Model, globalns and localns.

pydantic model langchain.llms.NLPCloud

  Wrapper around NLPCloud large language models.

  To use, you should have the `nlpcloud` python package installed, and the environment variable `NLPCLOUD_API_KEY` set with your API key.

  Example

      from langchain.llms import NLPCloud
      nlpcloud = NLPCloud(model="gpt-neox-20b")

  Validators

    - `set_callback_manager` » `callback_manager`

    - `set_verbose` » `verbose`

    - `validate_environment` » `all fields`

  field bad_words: List[str] = []

    List of tokens not allowed to be generated.

  field do_sample: bool = True

    Whether to use sampling (True) or greedy decoding.

  field early_stopping: bool = False

    Whether to stop beam search at num_beams sentences.

  field length_no_input: bool = True

    Whether min_length and max_length should include the length of the input.

  field length_penalty: float = 1.0

    Exponential penalty to the length.

  field max_length: int = 256

    The maximum number of tokens to generate in the completion.

  field min_length: int = 1

    The minimum number of tokens to generate in the completion.

  field model_name: str = 'finetuned-gpt-neox-20b'

    Model name to use.

  field num_beams: int = 1

    Number of beams for beam search.

  field num_return_sequences: int = 1

    How many completions to generate for each prompt.

  field remove_end_sequence: bool = True

    Whether or not to remove the end sequence token.

  field remove_input: bool = True

    Remove input text from API response

  field repetition_penalty: float = 1.0

    Penalizes repeated tokens. 1.0 means no penalty.

  field temperature: float = 0.7

    What sampling temperature to use.

  field top_k: int = 50

    The number of highest probability tokens to keep for top-k filtering.

  field top_p: int = 1

    Total probability mass of tokens to consider at each step.

  __call__(prompt: str, stop: Optional[List[str]] = None) → str

    Check Cache and run the LLM on the given prompt and input.

  async agenerate(prompts: List[str], stop: Optional[List[str]] = None) → langchain.schema.LLMResult

    Run the LLM on the given prompt and input.

  async agenerate_prompt(prompts: List[langchain.schema.PromptValue], stop: Optional[List[str]] = None) → langchain.schema.LLMResult

    Take in a list of prompt values and return an LLMResult.

  classmethod construct(_fields_set: Optional[SetStr] = None, **values: Any) → Model

    Creates a new model setting __dict__ and __fields_set__ from trusted or pre-validated data. Default values are respected, but no other validation is performed. Behaves as if Config.extra = ‘allow’ was set since it adds all passed values

  copy(*, include: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, exclude: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, update: Optional[DictStrAny] = None, deep: bool = False) → Model

    Duplicate a model, optionally choose which fields to include, exclude and change.

    Parameters

      - include – fields to include in new model

      - exclude – fields to exclude from new model, as with values this takes precedence over include

      - update – values to change/add in the new model. Note: the data is not validated before creating the new model: you should trust this data

      - deep – set to True to make a deep copy of the model

    Returns

      new model instance

  dict(**kwargs: Any) → Dict

    Return a dictionary of the LLM.

  generate(prompts: List[str], stop: Optional[List[str]] = None) → langchain.schema.LLMResult

    Run the LLM on the given prompt and input.

  generate_prompt(prompts: List[langchain.schema.PromptValue], stop: Optional[List[str]] = None) → langchain.schema.LLMResult

    Take in a list of prompt values and return an LLMResult.

  get_num_tokens(text: str) → int

    Get the number of tokens present in the text.

  get_num_tokens_from_messages(messages: List[langchain.schema.BaseMessage]) → int

    Get the number of tokens in the message.

  json(*, include: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, exclude: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, by_alias: bool = False, skip_defaults: Optional[bool] = None, exclude_unset: bool = False, exclude_defaults: bool = False, exclude_none: bool = False, encoder: Optional[Callable[[Any], Any]] = None, models_as_dict: bool = True, **dumps_kwargs: Any) → unicode

    Generate a JSON representation of the model, include and exclude arguments as per dict().

    encoder is an optional function to supply as default to json.dumps(), other arguments as per json.dumps().

  save(file_path: Union[pathlib.Path, str]) → None

    Save the LLM.

    Parameters

      file_path – Path to file to save the LLM to.

    Example: .. code-block:: python

    > llm.save(file_path=”path/llm.yaml”)

  classmethod update_forward_refs(**localns: Any) → None

    Try to update ForwardRefs on fields based on this Model, globalns and localns.

pydantic model langchain.llms.OpenAI

  Wrapper around OpenAI large language models.

  To use, you should have the `openai` python package installed, and the environment variable `OPENAI_API_KEY` set with your API key.

  Any parameters that are valid to be passed to the openai.create call can be passed in, even if not explicitly saved on this class.

  Example

      from langchain.llms import OpenAI
      openai = OpenAI(model_name="text-davinci-003")

  Validators

    - `build_extra` » `all fields`

    - `set_callback_manager` » `callback_manager`

    - `set_verbose` » `verbose`

    - `validate_environment` » `all fields`

  field verbose: bool [Optional]

    Whether to print out response text.

  __call__(prompt: str, stop: Optional[List[str]] = None) → str

    Check Cache and run the LLM on the given prompt and input.

  async agenerate(prompts: List[str], stop: Optional[List[str]] = None) → langchain.schema.LLMResult

    Run the LLM on the given prompt and input.

  async agenerate_prompt(prompts: List[langchain.schema.PromptValue], stop: Optional[List[str]] = None) → langchain.schema.LLMResult

    Take in a list of prompt values and return an LLMResult.

  classmethod construct(_fields_set: Optional[SetStr] = None, **values: Any) → Model

    Creates a new model setting __dict__ and __fields_set__ from trusted or pre-validated data. Default values are respected, but no other validation is performed. Behaves as if Config.extra = ‘allow’ was set since it adds all passed values

  copy(*, include: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, exclude: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, update: Optional[DictStrAny] = None, deep: bool = False) → Model

    Duplicate a model, optionally choose which fields to include, exclude and change.

    Parameters

      - include – fields to include in new model

      - exclude – fields to exclude from new model, as with values this takes precedence over include

      - update – values to change/add in the new model. Note: the data is not validated before creating the new model: you should trust this data

      - deep – set to True to make a deep copy of the model

    Returns

      new model instance

  create_llm_result(choices: Any, prompts: List[str], token_usage: Dict[str, int]) → langchain.schema.LLMResult

    Create the LLMResult from the choices and prompts.

  dict(**kwargs: Any) → Dict

    Return a dictionary of the LLM.

  generate(prompts: List[str], stop: Optional[List[str]] = None) → langchain.schema.LLMResult

    Run the LLM on the given prompt and input.

  generate_prompt(prompts: List[langchain.schema.PromptValue], stop: Optional[List[str]] = None) → langchain.schema.LLMResult

    Take in a list of prompt values and return an LLMResult.

  get_num_tokens(text: str) → int

    Calculate num tokens with tiktoken package.

  get_num_tokens_from_messages(messages: List[langchain.schema.BaseMessage]) → int

    Get the number of tokens in the message.

  get_sub_prompts(params: Dict[str, Any], prompts: List[str], stop: Optional[List[str]] = None) → List[List[str]]

    Get the sub prompts for llm call.

  json(*, include: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, exclude: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, by_alias: bool = False, skip_defaults: Optional[bool] = None, exclude_unset: bool = False, exclude_defaults: bool = False, exclude_none: bool = False, encoder: Optional[Callable[[Any], Any]] = None, models_as_dict: bool = True, **dumps_kwargs: Any) → unicode

    Generate a JSON representation of the model, include and exclude arguments as per dict().

    encoder is an optional function to supply as default to json.dumps(), other arguments as per json.dumps().

  max_tokens_for_prompt(prompt: str) → int

    Calculate the maximum number of tokens possible to generate for a prompt.

    Parameters

      prompt – The prompt to pass into the model.

    Returns

      The maximum number of tokens to generate for a prompt.

    Example

        max_tokens = openai.max_token_for_prompt("Tell me a joke.")

  modelname_to_contextsize(modelname: str) → int

    Calculate the maximum number of tokens possible to generate for a model.

    Parameters

      modelname – The modelname we want to know the context size for.

    Returns

      The maximum context size

    Example

        max_tokens = openai.modelname_to_contextsize("text-davinci-003")

  prep_streaming_params(stop: Optional[List[str]] = None) → Dict[str, Any]

    Prepare the params for streaming.

  save(file_path: Union[pathlib.Path, str]) → None

    Save the LLM.

    Parameters

      file_path – Path to file to save the LLM to.

    Example: .. code-block:: python

    > llm.save(file_path=”path/llm.yaml”)

  stream(prompt: str, stop: Optional[List[str]] = None) → Generator

    Call OpenAI with streaming flag and return the resulting generator.

    BETA: this is a beta feature while we figure out the right abstraction. Once that happens, this interface could change.

    Parameters

      - prompt – The prompts to pass into the model.

      - stop – Optional list of stop words to use when generating.

    Returns

      A generator representing the stream of tokens from OpenAI.

    Example

        generator = openai.stream("Tell me a joke.")
        for token in generator:
            yield token

  classmethod update_forward_refs(**localns: Any) → None

    Try to update ForwardRefs on fields based on this Model, globalns and localns.

pydantic model langchain.llms.OpenAIChat

  Wrapper around OpenAI Chat large language models.

  To use, you should have the `openai` python package installed, and the environment variable `OPENAI_API_KEY` set with your API key.

  Any parameters that are valid to be passed to the openai.create call can be passed in, even if not explicitly saved on this class.

  Example

      from langchain.llms import OpenAIChat
      openaichat = OpenAIChat(model_name="gpt-3.5-turbo")

  Validators

    - `build_extra` » `all fields`

    - `set_callback_manager` » `callback_manager`

    - `set_verbose` » `verbose`

    - `validate_environment` » `all fields`

  field allowed_special: Union[Literal['all'], AbstractSet[str]] = {}

    Set of special tokens that are allowed。

  field disallowed_special: Union[Literal['all'], Collection[str]] = 'all'

    Set of special tokens that are not allowed。

  field max_retries: int = 6

    Maximum number of retries to make when generating.

  field model_kwargs: Dict[str, Any] [Optional]

    Holds any model parameters valid for create call not explicitly specified.

  field model_name: str = 'gpt-3.5-turbo'

    Model name to use.

  field prefix_messages: List [Optional]

    Series of messages for Chat input.

  field streaming: bool = False

    Whether to stream the results or not.

  field verbose: bool [Optional]

    Whether to print out response text.

  __call__(prompt: str, stop: Optional[List[str]] = None) → str

    Check Cache and run the LLM on the given prompt and input.

  async agenerate(prompts: List[str], stop: Optional[List[str]] = None) → langchain.schema.LLMResult

    Run the LLM on the given prompt and input.

  async agenerate_prompt(prompts: List[langchain.schema.PromptValue], stop: Optional[List[str]] = None) → langchain.schema.LLMResult

    Take in a list of prompt values and return an LLMResult.

  classmethod construct(_fields_set: Optional[SetStr] = None, **values: Any) → Model

    Creates a new model setting __dict__ and __fields_set__ from trusted or pre-validated data. Default values are respected, but no other validation is performed. Behaves as if Config.extra = ‘allow’ was set since it adds all passed values

  copy(*, include: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, exclude: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, update: Optional[DictStrAny] = None, deep: bool = False) → Model

    Duplicate a model, optionally choose which fields to include, exclude and change.

    Parameters

      - include – fields to include in new model

      - exclude – fields to exclude from new model, as with values this takes precedence over include

      - update – values to change/add in the new model. Note: the data is not validated before creating the new model: you should trust this data

      - deep – set to True to make a deep copy of the model

    Returns

      new model instance

  dict(**kwargs: Any) → Dict

    Return a dictionary of the LLM.

  generate(prompts: List[str], stop: Optional[List[str]] = None) → langchain.schema.LLMResult

    Run the LLM on the given prompt and input.

  generate_prompt(prompts: List[langchain.schema.PromptValue], stop: Optional[List[str]] = None) → langchain.schema.LLMResult

    Take in a list of prompt values and return an LLMResult.

  get_num_tokens(text: str) → int

    Calculate num tokens with tiktoken package.

  get_num_tokens_from_messages(messages: List[langchain.schema.BaseMessage]) → int

    Get the number of tokens in the message.

  json(*, include: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, exclude: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, by_alias: bool = False, skip_defaults: Optional[bool] = None, exclude_unset: bool = False, exclude_defaults: bool = False, exclude_none: bool = False, encoder: Optional[Callable[[Any], Any]] = None, models_as_dict: bool = True, **dumps_kwargs: Any) → unicode

    Generate a JSON representation of the model, include and exclude arguments as per dict().

    encoder is an optional function to supply as default to json.dumps(), other arguments as per json.dumps().

  save(file_path: Union[pathlib.Path, str]) → None

    Save the LLM.

    Parameters

      file_path – Path to file to save the LLM to.

    Example: .. code-block:: python

    > llm.save(file_path=”path/llm.yaml”)

  classmethod update_forward_refs(**localns: Any) → None

    Try to update ForwardRefs on fields based on this Model, globalns and localns.

pydantic model langchain.llms.Petals

  Wrapper around Petals Bloom models.

  To use, you should have the `petals` python package installed, and the environment variable `HUGGINGFACE_API_KEY` set with your API key.

  Any parameters that are valid to be passed to the call can be passed in, even if not explicitly saved on this class.

  Example

  Validators

    - `build_extra` » `all fields`

    - `set_callback_manager` » `callback_manager`

    - `set_verbose` » `verbose`

    - `validate_environment` » `all fields`

  field client: Any = None

    The client to use for the API calls.

  field do_sample: bool = True

    Whether or not to use sampling; use greedy decoding otherwise.

  field max_length: Optional[int] = None

    The maximum length of the sequence to be generated.

  field max_new_tokens: int = 256

    The maximum number of new tokens to generate in the completion.

  field model_kwargs: Dict[str, Any] [Optional]

    Holds any model parameters valid for create call not explicitly specified.

  field model_name: str = 'bigscience/bloom-petals'

    The model to use.

  field temperature: float = 0.7

    What sampling temperature to use

  field tokenizer: Any = None

    The tokenizer to use for the API calls.

  field top_k: Optional[int] = None

    The number of highest probability vocabulary tokens to keep for top-k-filtering.

  field top_p: float = 0.9

    The cumulative probability for top-p sampling.

  __call__(prompt: str, stop: Optional[List[str]] = None) → str

    Check Cache and run the LLM on the given prompt and input.

  async agenerate(prompts: List[str], stop: Optional[List[str]] = None) → langchain.schema.LLMResult

    Run the LLM on the given prompt and input.

  async agenerate_prompt(prompts: List[langchain.schema.PromptValue], stop: Optional[List[str]] = None) → langchain.schema.LLMResult

    Take in a list of prompt values and return an LLMResult.

  classmethod construct(_fields_set: Optional[SetStr] = None, **values: Any) → Model

    Creates a new model setting __dict__ and __fields_set__ from trusted or pre-validated data. Default values are respected, but no other validation is performed. Behaves as if Config.extra = ‘allow’ was set since it adds all passed values

  copy(*, include: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, exclude: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, update: Optional[DictStrAny] = None, deep: bool = False) → Model

    Duplicate a model, optionally choose which fields to include, exclude and change.

    Parameters

      - include – fields to include in new model

      - exclude – fields to exclude from new model, as with values this takes precedence over include

      - update – values to change/add in the new model. Note: the data is not validated before creating the new model: you should trust this data

      - deep – set to True to make a deep copy of the model

    Returns

      new model instance

  dict(**kwargs: Any) → Dict

    Return a dictionary of the LLM.

  generate(prompts: List[str], stop: Optional[List[str]] = None) → langchain.schema.LLMResult

    Run the LLM on the given prompt and input.

  generate_prompt(prompts: List[langchain.schema.PromptValue], stop: Optional[List[str]] = None) → langchain.schema.LLMResult

    Take in a list of prompt values and return an LLMResult.

  get_num_tokens(text: str) → int

    Get the number of tokens present in the text.

  get_num_tokens_from_messages(messages: List[langchain.schema.BaseMessage]) → int

    Get the number of tokens in the message.

  json(*, include: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, exclude: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, by_alias: bool = False, skip_defaults: Optional[bool] = None, exclude_unset: bool = False, exclude_defaults: bool = False, exclude_none: bool = False, encoder: Optional[Callable[[Any], Any]] = None, models_as_dict: bool = True, **dumps_kwargs: Any) → unicode

    Generate a JSON representation of the model, include and exclude arguments as per dict().

    encoder is an optional function to supply as default to json.dumps(), other arguments as per json.dumps().

  save(file_path: Union[pathlib.Path, str]) → None

    Save the LLM.

    Parameters

      file_path – Path to file to save the LLM to.

    Example: .. code-block:: python

    > llm.save(file_path=”path/llm.yaml”)

  classmethod update_forward_refs(**localns: Any) → None

    Try to update ForwardRefs on fields based on this Model, globalns and localns.

pydantic model langchain.llms.PredictionGuard

  Wrapper around Prediction Guard large language models. To use, you should have the `predictionguard` python package installed, and the environment variable `PREDICTIONGUARD_TOKEN` set with your access token, or pass it as a named parameter to the constructor. .. rubric:: Example

  Validators

    - `set_callback_manager` » `callback_manager`

    - `set_verbose` » `verbose`

    - `validate_environment` » `all fields`

  field max_tokens: int = 256

    Denotes the number of tokens to predict per generation.

  field name: Optional[str] = 'default-text-gen'

    Proxy name to use.

  field temperature: float = 0.75

    A non-negative float that tunes the degree of randomness in generation.

  __call__(prompt: str, stop: Optional[List[str]] = None) → str

    Check Cache and run the LLM on the given prompt and input.

  async agenerate(prompts: List[str], stop: Optional[List[str]] = None) → langchain.schema.LLMResult

    Run the LLM on the given prompt and input.

  async agenerate_prompt(prompts: List[langchain.schema.PromptValue], stop: Optional[List[str]] = None) → langchain.schema.LLMResult

    Take in a list of prompt values and return an LLMResult.

  classmethod construct(_fields_set: Optional[SetStr] = None, **values: Any) → Model

    Creates a new model setting __dict__ and __fields_set__ from trusted or pre-validated data. Default values are respected, but no other validation is performed. Behaves as if Config.extra = ‘allow’ was set since it adds all passed values

  copy(*, include: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, exclude: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, update: Optional[DictStrAny] = None, deep: bool = False) → Model

    Duplicate a model, optionally choose which fields to include, exclude and change.

    Parameters

      - include – fields to include in new model

      - exclude – fields to exclude from new model, as with values this takes precedence over include

      - update – values to change/add in the new model. Note: the data is not validated before creating the new model: you should trust this data

      - deep – set to True to make a deep copy of the model

    Returns

      new model instance

  dict(**kwargs: Any) → Dict

    Return a dictionary of the LLM.

  generate(prompts: List[str], stop: Optional[List[str]] = None) → langchain.schema.LLMResult

    Run the LLM on the given prompt and input.

  generate_prompt(prompts: List[langchain.schema.PromptValue], stop: Optional[List[str]] = None) → langchain.schema.LLMResult

    Take in a list of prompt values and return an LLMResult.

  get_num_tokens(text: str) → int

    Get the number of tokens present in the text.

  get_num_tokens_from_messages(messages: List[langchain.schema.BaseMessage]) → int

    Get the number of tokens in the message.

  json(*, include: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, exclude: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, by_alias: bool = False, skip_defaults: Optional[bool] = None, exclude_unset: bool = False, exclude_defaults: bool = False, exclude_none: bool = False, encoder: Optional[Callable[[Any], Any]] = None, models_as_dict: bool = True, **dumps_kwargs: Any) → unicode

    Generate a JSON representation of the model, include and exclude arguments as per dict().

    encoder is an optional function to supply as default to json.dumps(), other arguments as per json.dumps().

  save(file_path: Union[pathlib.Path, str]) → None

    Save the LLM.

    Parameters

      file_path – Path to file to save the LLM to.

    Example: .. code-block:: python

    > llm.save(file_path=”path/llm.yaml”)

  classmethod update_forward_refs(**localns: Any) → None

    Try to update ForwardRefs on fields based on this Model, globalns and localns.

pydantic model langchain.llms.PromptLayerOpenAI

  Wrapper around OpenAI large language models.

  To use, you should have the `openai` and `promptlayer` python package installed, and the environment variable `OPENAI_API_KEY` and `PROMPTLAYER_API_KEY` set with your openAI API key and promptlayer key respectively.

  All parameters that can be passed to the OpenAI LLM can also be passed here. The PromptLayerOpenAI LLM adds two optional :param `pl_tags`: List of strings to tag the request with. :param `return_pl_id`: If True, the PromptLayer request ID will be

  > returned in the `generation_info` field of the `Generation` object.

  Example

      from langchain.llms import PromptLayerOpenAI
      openai = PromptLayerOpenAI(model_name="text-davinci-003")

  Validators

    - `build_extra` » `all fields`

    - `set_callback_manager` » `callback_manager`

    - `set_verbose` » `verbose`

    - `validate_environment` » `all fields`

  __call__(prompt: str, stop: Optional[List[str]] = None) → str

    Check Cache and run the LLM on the given prompt and input.

  async agenerate(prompts: List[str], stop: Optional[List[str]] = None) → langchain.schema.LLMResult

    Run the LLM on the given prompt and input.

  async agenerate_prompt(prompts: List[langchain.schema.PromptValue], stop: Optional[List[str]] = None) → langchain.schema.LLMResult

    Take in a list of prompt values and return an LLMResult.

  classmethod construct(_fields_set: Optional[SetStr] = None, **values: Any) → Model

    Creates a new model setting __dict__ and __fields_set__ from trusted or pre-validated data. Default values are respected, but no other validation is performed. Behaves as if Config.extra = ‘allow’ was set since it adds all passed values

  copy(*, include: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, exclude: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, update: Optional[DictStrAny] = None, deep: bool = False) → Model

    Duplicate a model, optionally choose which fields to include, exclude and change.

    Parameters

      - include – fields to include in new model

      - exclude – fields to exclude from new model, as with values this takes precedence over include

      - update – values to change/add in the new model. Note: the data is not validated before creating the new model: you should trust this data

      - deep – set to True to make a deep copy of the model

    Returns

      new model instance

  create_llm_result(choices: Any, prompts: List[str], token_usage: Dict[str, int]) → langchain.schema.LLMResult

    Create the LLMResult from the choices and prompts.

  dict(**kwargs: Any) → Dict

    Return a dictionary of the LLM.

  generate(prompts: List[str], stop: Optional[List[str]] = None) → langchain.schema.LLMResult

    Run the LLM on the given prompt and input.

  generate_prompt(prompts: List[langchain.schema.PromptValue], stop: Optional[List[str]] = None) → langchain.schema.LLMResult

    Take in a list of prompt values and return an LLMResult.

  get_num_tokens(text: str) → int

    Calculate num tokens with tiktoken package.

  get_num_tokens_from_messages(messages: List[langchain.schema.BaseMessage]) → int

    Get the number of tokens in the message.

  get_sub_prompts(params: Dict[str, Any], prompts: List[str], stop: Optional[List[str]] = None) → List[List[str]]

    Get the sub prompts for llm call.

  json(*, include: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, exclude: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, by_alias: bool = False, skip_defaults: Optional[bool] = None, exclude_unset: bool = False, exclude_defaults: bool = False, exclude_none: bool = False, encoder: Optional[Callable[[Any], Any]] = None, models_as_dict: bool = True, **dumps_kwargs: Any) → unicode

    Generate a JSON representation of the model, include and exclude arguments as per dict().

    encoder is an optional function to supply as default to json.dumps(), other arguments as per json.dumps().

  max_tokens_for_prompt(prompt: str) → int

    Calculate the maximum number of tokens possible to generate for a prompt.

    Parameters

      prompt – The prompt to pass into the model.

    Returns

      The maximum number of tokens to generate for a prompt.

    Example

        max_tokens = openai.max_token_for_prompt("Tell me a joke.")

  modelname_to_contextsize(modelname: str) → int

    Calculate the maximum number of tokens possible to generate for a model.

    Parameters

      modelname – The modelname we want to know the context size for.

    Returns

      The maximum context size

    Example

        max_tokens = openai.modelname_to_contextsize("text-davinci-003")

  prep_streaming_params(stop: Optional[List[str]] = None) → Dict[str, Any]

    Prepare the params for streaming.

  save(file_path: Union[pathlib.Path, str]) → None

    Save the LLM.

    Parameters

      file_path – Path to file to save the LLM to.

    Example: .. code-block:: python

    > llm.save(file_path=”path/llm.yaml”)

  stream(prompt: str, stop: Optional[List[str]] = None) → Generator

    Call OpenAI with streaming flag and return the resulting generator.

    BETA: this is a beta feature while we figure out the right abstraction. Once that happens, this interface could change.

    Parameters

      - prompt – The prompts to pass into the model.

      - stop – Optional list of stop words to use when generating.

    Returns

      A generator representing the stream of tokens from OpenAI.

    Example

        generator = openai.stream("Tell me a joke.")
        for token in generator:
            yield token

  classmethod update_forward_refs(**localns: Any) → None

    Try to update ForwardRefs on fields based on this Model, globalns and localns.

pydantic model langchain.llms.PromptLayerOpenAIChat

  Wrapper around OpenAI large language models.

  To use, you should have the `openai` and `promptlayer` python package installed, and the environment variable `OPENAI_API_KEY` and `PROMPTLAYER_API_KEY` set with your openAI API key and promptlayer key respectively.

  All parameters that can be passed to the OpenAIChat LLM can also be passed here. The PromptLayerOpenAIChat adds two optional :param `pl_tags`: List of strings to tag the request with. :param `return_pl_id`: If True, the PromptLayer request ID will be

  > returned in the `generation_info` field of the `Generation` object.

  Example

      from langchain.llms import PromptLayerOpenAIChat
      openaichat = PromptLayerOpenAIChat(model_name="gpt-3.5-turbo")

  Validators

    - `build_extra` » `all fields`

    - `set_callback_manager` » `callback_manager`

    - `set_verbose` » `verbose`

    - `validate_environment` » `all fields`

  field allowed_special: Union[Literal['all'], AbstractSet[str]] = {}

    Set of special tokens that are allowed。

  field disallowed_special: Union[Literal['all'], Collection[str]] = 'all'

    Set of special tokens that are not allowed。

  field max_retries: int = 6

    Maximum number of retries to make when generating.

  field model_kwargs: Dict[str, Any] [Optional]

    Holds any model parameters valid for create call not explicitly specified.

  field model_name: str = 'gpt-3.5-turbo'

    Model name to use.

  field prefix_messages: List [Optional]

    Series of messages for Chat input.

  field streaming: bool = False

    Whether to stream the results or not.

  __call__(prompt: str, stop: Optional[List[str]] = None) → str

    Check Cache and run the LLM on the given prompt and input.

  async agenerate(prompts: List[str], stop: Optional[List[str]] = None) → langchain.schema.LLMResult

    Run the LLM on the given prompt and input.

  async agenerate_prompt(prompts: List[langchain.schema.PromptValue], stop: Optional[List[str]] = None) → langchain.schema.LLMResult

    Take in a list of prompt values and return an LLMResult.

  classmethod construct(_fields_set: Optional[SetStr] = None, **values: Any) → Model

    Creates a new model setting __dict__ and __fields_set__ from trusted or pre-validated data. Default values are respected, but no other validation is performed. Behaves as if Config.extra = ‘allow’ was set since it adds all passed values

  copy(*, include: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, exclude: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, update: Optional[DictStrAny] = None, deep: bool = False) → Model

    Duplicate a model, optionally choose which fields to include, exclude and change.

    Parameters

      - include – fields to include in new model

      - exclude – fields to exclude from new model, as with values this takes precedence over include

      - update – values to change/add in the new model. Note: the data is not validated before creating the new model: you should trust this data

      - deep – set to True to make a deep copy of the model

    Returns

      new model instance

  dict(**kwargs: Any) → Dict

    Return a dictionary of the LLM.

  generate(prompts: List[str], stop: Optional[List[str]] = None) → langchain.schema.LLMResult

    Run the LLM on the given prompt and input.

  generate_prompt(prompts: List[langchain.schema.PromptValue], stop: Optional[List[str]] = None) → langchain.schema.LLMResult

    Take in a list of prompt values and return an LLMResult.

  get_num_tokens(text: str) → int

    Calculate num tokens with tiktoken package.

  get_num_tokens_from_messages(messages: List[langchain.schema.BaseMessage]) → int

    Get the number of tokens in the message.

  json(*, include: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, exclude: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, by_alias: bool = False, skip_defaults: Optional[bool] = None, exclude_unset: bool = False, exclude_defaults: bool = False, exclude_none: bool = False, encoder: Optional[Callable[[Any], Any]] = None, models_as_dict: bool = True, **dumps_kwargs: Any) → unicode

    Generate a JSON representation of the model, include and exclude arguments as per dict().

    encoder is an optional function to supply as default to json.dumps(), other arguments as per json.dumps().

  save(file_path: Union[pathlib.Path, str]) → None

    Save the LLM.

    Parameters

      file_path – Path to file to save the LLM to.

    Example: .. code-block:: python

    > llm.save(file_path=”path/llm.yaml”)

  classmethod update_forward_refs(**localns: Any) → None

    Try to update ForwardRefs on fields based on this Model, globalns and localns.

pydantic model langchain.llms.RWKV

  Wrapper around RWKV language models.

  To use, you should have the `rwkv` python package installed, the pre-trained model file, and the model’s config information.

  Example

      from langchain.llms import RWKV
      model = RWKV(model="./models/rwkv-3b-fp16.bin", strategy="cpu fp32")

      # Simplest invocation
      response = model("Once upon a time, ")

  Validators

    - `set_callback_manager` » `callback_manager`

    - `set_verbose` » `verbose`

    - `validate_environment` » `all fields`

  field CHUNK_LEN: int = 256

    Batch size for prompt processing.

  field max_tokens_per_generation: int = 256

    Maximum number of tokens to generate.

  field model: str [Required]

    Path to the pre-trained RWKV model file.

  field penalty_alpha_frequency: float = 0.4

    Positive values penalize new tokens based on their existing frequency in the text so far, decreasing the model’s likelihood to repeat the same line verbatim..

  field penalty_alpha_presence: float = 0.4

    Positive values penalize new tokens based on whether they appear in the text so far, increasing the model’s likelihood to talk about new topics..

  field rwkv_verbose: bool = True

    Print debug information.

  field strategy: str = 'cpu fp32'

    Token context window.

  field temperature: float = 1.0

    The temperature to use for sampling.

  field tokens_path: str [Required]

    Path to the RWKV tokens file.

  field top_p: float = 0.5

    The top-p value to use for sampling.

  __call__(prompt: str, stop: Optional[List[str]] = None) → str

    Check Cache and run the LLM on the given prompt and input.

  async agenerate(prompts: List[str], stop: Optional[List[str]] = None) → langchain.schema.LLMResult

    Run the LLM on the given prompt and input.

  async agenerate_prompt(prompts: List[langchain.schema.PromptValue], stop: Optional[List[str]] = None) → langchain.schema.LLMResult

    Take in a list of prompt values and return an LLMResult.

  classmethod construct(_fields_set: Optional[SetStr] = None, **values: Any) → Model

    Creates a new model setting __dict__ and __fields_set__ from trusted or pre-validated data. Default values are respected, but no other validation is performed. Behaves as if Config.extra = ‘allow’ was set since it adds all passed values

  copy(*, include: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, exclude: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, update: Optional[DictStrAny] = None, deep: bool = False) → Model

    Duplicate a model, optionally choose which fields to include, exclude and change.

    Parameters

      - include – fields to include in new model

      - exclude – fields to exclude from new model, as with values this takes precedence over include

      - update – values to change/add in the new model. Note: the data is not validated before creating the new model: you should trust this data

      - deep – set to True to make a deep copy of the model

    Returns

      new model instance

  dict(**kwargs: Any) → Dict

    Return a dictionary of the LLM.

  generate(prompts: List[str], stop: Optional[List[str]] = None) → langchain.schema.LLMResult

    Run the LLM on the given prompt and input.

  generate_prompt(prompts: List[langchain.schema.PromptValue], stop: Optional[List[str]] = None) → langchain.schema.LLMResult

    Take in a list of prompt values and return an LLMResult.

  get_num_tokens(text: str) → int

    Get the number of tokens present in the text.

  get_num_tokens_from_messages(messages: List[langchain.schema.BaseMessage]) → int

    Get the number of tokens in the message.

  json(*, include: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, exclude: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, by_alias: bool = False, skip_defaults: Optional[bool] = None, exclude_unset: bool = False, exclude_defaults: bool = False, exclude_none: bool = False, encoder: Optional[Callable[[Any], Any]] = None, models_as_dict: bool = True, **dumps_kwargs: Any) → unicode

    Generate a JSON representation of the model, include and exclude arguments as per dict().

    encoder is an optional function to supply as default to json.dumps(), other arguments as per json.dumps().

  save(file_path: Union[pathlib.Path, str]) → None

    Save the LLM.

    Parameters

      file_path – Path to file to save the LLM to.

    Example: .. code-block:: python

    > llm.save(file_path=”path/llm.yaml”)

  classmethod update_forward_refs(**localns: Any) → None

    Try to update ForwardRefs on fields based on this Model, globalns and localns.

pydantic model langchain.llms.Replicate

  Wrapper around Replicate models.

  To use, you should have the `replicate` python package installed, and the environment variable `REPLICATE_API_TOKEN` set with your API token. You can find your token here: https://replicate.com/account

  The model param is required, but any other model parameters can also be passed in with the format input={model_param: value, …}

  Example

  Validators

    - `build_extra` » `all fields`

    - `set_callback_manager` » `callback_manager`

    - `set_verbose` » `verbose`

    - `validate_environment` » `all fields`

  __call__(prompt: str, stop: Optional[List[str]] = None) → str

    Check Cache and run the LLM on the given prompt and input.

  async agenerate(prompts: List[str], stop: Optional[List[str]] = None) → langchain.schema.LLMResult

    Run the LLM on the given prompt and input.

  async agenerate_prompt(prompts: List[langchain.schema.PromptValue], stop: Optional[List[str]] = None) → langchain.schema.LLMResult

    Take in a list of prompt values and return an LLMResult.

  classmethod construct(_fields_set: Optional[SetStr] = None, **values: Any) → Model

    Creates a new model setting __dict__ and __fields_set__ from trusted or pre-validated data. Default values are respected, but no other validation is performed. Behaves as if Config.extra = ‘allow’ was set since it adds all passed values

  copy(*, include: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, exclude: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, update: Optional[DictStrAny] = None, deep: bool = False) → Model

    Duplicate a model, optionally choose which fields to include, exclude and change.

    Parameters

      - include – fields to include in new model

      - exclude – fields to exclude from new model, as with values this takes precedence over include

      - update – values to change/add in the new model. Note: the data is not validated before creating the new model: you should trust this data

      - deep – set to True to make a deep copy of the model

    Returns

      new model instance

  dict(**kwargs: Any) → Dict

    Return a dictionary of the LLM.

  generate(prompts: List[str], stop: Optional[List[str]] = None) → langchain.schema.LLMResult

    Run the LLM on the given prompt and input.

  generate_prompt(prompts: List[langchain.schema.PromptValue], stop: Optional[List[str]] = None) → langchain.schema.LLMResult

    Take in a list of prompt values and return an LLMResult.

  get_num_tokens(text: str) → int

    Get the number of tokens present in the text.

  get_num_tokens_from_messages(messages: List[langchain.schema.BaseMessage]) → int

    Get the number of tokens in the message.

  json(*, include: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, exclude: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, by_alias: bool = False, skip_defaults: Optional[bool] = None, exclude_unset: bool = False, exclude_defaults: bool = False, exclude_none: bool = False, encoder: Optional[Callable[[Any], Any]] = None, models_as_dict: bool = True, **dumps_kwargs: Any) → unicode

    Generate a JSON representation of the model, include and exclude arguments as per dict().

    encoder is an optional function to supply as default to json.dumps(), other arguments as per json.dumps().

  save(file_path: Union[pathlib.Path, str]) → None

    Save the LLM.

    Parameters

      file_path – Path to file to save the LLM to.

    Example: .. code-block:: python

    > llm.save(file_path=”path/llm.yaml”)

  classmethod update_forward_refs(**localns: Any) → None

    Try to update ForwardRefs on fields based on this Model, globalns and localns.

pydantic model langchain.llms.SagemakerEndpoint

  Wrapper around custom Sagemaker Inference Endpoints.

  To use, you must supply the endpoint name from your deployed Sagemaker model & the region where it is deployed.

  To authenticate, the AWS client uses the following methods to automatically load credentials: https://boto3.amazonaws.com/v1/documentation/api/latest/guide/credentials.html

  If a specific credential profile should be used, you must pass the name of the profile from the ~/.aws/credentials file that is to be used.

  Make sure the credentials / roles used have the required policies to access the Sagemaker endpoint. See: https://docs.aws.amazon.com/IAM/latest/UserGuide/access_policies.html

  Validators

    - `set_callback_manager` » `callback_manager`

    - `set_verbose` » `verbose`

    - `validate_environment` » `all fields`

  field content_handler: langchain.llms.sagemaker_endpoint.LLMContentHandler [Required]

    The content handler class that provides an input and output transform functions to handle formats between LLM and the endpoint.

  field credentials_profile_name: Optional[str] = None

    The name of the profile in the ~/.aws/credentials or ~/.aws/config files, which has either access keys or role information specified. If not specified, the default credential profile or, if on an EC2 instance, credentials from IMDS will be used. See: https://boto3.amazonaws.com/v1/documentation/api/latest/guide/credentials.html

  field endpoint_kwargs: Optional[Dict] = None

    Optional attributes passed to the invoke_endpoint function. See `boto3`_. docs for more info. .. _boto3: <https://boto3.amazonaws.com/v1/documentation/api/latest/index.html>

  field endpoint_name: str = ''

    The name of the endpoint from the deployed Sagemaker model. Must be unique within an AWS Region.

  field model_kwargs: Optional[Dict] = None

    Key word arguments to pass to the model.

  field region_name: str = ''

    The aws region where the Sagemaker model is deployed, eg. us-west-2.

  __call__(prompt: str, stop: Optional[List[str]] = None) → str

    Check Cache and run the LLM on the given prompt and input.

  async agenerate(prompts: List[str], stop: Optional[List[str]] = None) → langchain.schema.LLMResult

    Run the LLM on the given prompt and input.

  async agenerate_prompt(prompts: List[langchain.schema.PromptValue], stop: Optional[List[str]] = None) → langchain.schema.LLMResult

    Take in a list of prompt values and return an LLMResult.

  classmethod construct(_fields_set: Optional[SetStr] = None, **values: Any) → Model

    Creates a new model setting __dict__ and __fields_set__ from trusted or pre-validated data. Default values are respected, but no other validation is performed. Behaves as if Config.extra = ‘allow’ was set since it adds all passed values

  copy(*, include: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, exclude: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, update: Optional[DictStrAny] = None, deep: bool = False) → Model

    Duplicate a model, optionally choose which fields to include, exclude and change.

    Parameters

      - include – fields to include in new model

      - exclude – fields to exclude from new model, as with values this takes precedence over include

      - update – values to change/add in the new model. Note: the data is not validated before creating the new model: you should trust this data

      - deep – set to True to make a deep copy of the model

    Returns

      new model instance

  dict(**kwargs: Any) → Dict

    Return a dictionary of the LLM.

  generate(prompts: List[str], stop: Optional[List[str]] = None) → langchain.schema.LLMResult

    Run the LLM on the given prompt and input.

  generate_prompt(prompts: List[langchain.schema.PromptValue], stop: Optional[List[str]] = None) → langchain.schema.LLMResult

    Take in a list of prompt values and return an LLMResult.

  get_num_tokens(text: str) → int

    Get the number of tokens present in the text.

  get_num_tokens_from_messages(messages: List[langchain.schema.BaseMessage]) → int

    Get the number of tokens in the message.

  json(*, include: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, exclude: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, by_alias: bool = False, skip_defaults: Optional[bool] = None, exclude_unset: bool = False, exclude_defaults: bool = False, exclude_none: bool = False, encoder: Optional[Callable[[Any], Any]] = None, models_as_dict: bool = True, **dumps_kwargs: Any) → unicode

    Generate a JSON representation of the model, include and exclude arguments as per dict().

    encoder is an optional function to supply as default to json.dumps(), other arguments as per json.dumps().

  save(file_path: Union[pathlib.Path, str]) → None

    Save the LLM.

    Parameters

      file_path – Path to file to save the LLM to.

    Example: .. code-block:: python

    > llm.save(file_path=”path/llm.yaml”)

  classmethod update_forward_refs(**localns: Any) → None

    Try to update ForwardRefs on fields based on this Model, globalns and localns.

pydantic model langchain.llms.SelfHostedHuggingFaceLLM

  Wrapper around HuggingFace Pipeline API to run on self-hosted remote hardware.

  Supported hardware includes auto-launched instances on AWS, GCP, Azure, and Lambda, as well as servers specified by IP address and SSH credentials (such as on-prem, or another cloud like Paperspace, Coreweave, etc.).

  To use, you should have the `runhouse` python package installed.

  Only supports text-generation and text2text-generation for now.

  Example using from_model_id:

        from langchain.llms import SelfHostedHuggingFaceLLM
        import runhouse as rh
        gpu = rh.cluster(name="rh-a10x", instance_type="A100:1")
        hf = SelfHostedHuggingFaceLLM(
            model_id="google/flan-t5-large", task="text2text-generation",
            hardware=gpu
        )

  Example passing fn that generates a pipeline (bc the pipeline is not serializable):

        from langchain.llms import SelfHostedHuggingFaceLLM
        from transformers import AutoModelForCausalLM, AutoTokenizer, pipeline
        import runhouse as rh

        def get_pipeline():
            model_id = "gpt2"
            tokenizer = AutoTokenizer.from_pretrained(model_id)
            model = AutoModelForCausalLM.from_pretrained(model_id)
            pipe = pipeline(
                "text-generation", model=model, tokenizer=tokenizer
            )
            return pipe
        hf = SelfHostedHuggingFaceLLM(
            model_load_fn=get_pipeline, model_id="gpt2", hardware=gpu)

  Validators

    - `set_callback_manager` » `callback_manager`

    - `set_verbose` » `verbose`

  field device: int = 0

    Device to use for inference. -1 for CPU, 0 for GPU, 1 for second GPU, etc.

  field hardware: Any = None

    Remote hardware to send the inference function to.

  field inference_fn: Callable = <function _generate_text>

    Inference function to send to the remote hardware.

  field load_fn_kwargs: Optional[dict] = None

    Key word arguments to pass to the model load function.

  field model_id: str = 'gpt2'

    Hugging Face model_id to load the model.

  field model_kwargs: Optional[dict] = None

    Key word arguments to pass to the model.

  field model_load_fn: Callable = <function _load_transformer>

    Function to load the model remotely on the server.

  field model_reqs: List[str] = ['./', 'transformers', 'torch']

    Requirements to install on hardware to inference the model.

  field task: str = 'text-generation'

    Hugging Face task (either “text-generation” or “text2text-generation”).

  __call__(prompt: str, stop: Optional[List[str]] = None) → str

    Check Cache and run the LLM on the given prompt and input.

  async agenerate(prompts: List[str], stop: Optional[List[str]] = None) → langchain.schema.LLMResult

    Run the LLM on the given prompt and input.

  async agenerate_prompt(prompts: List[langchain.schema.PromptValue], stop: Optional[List[str]] = None) → langchain.schema.LLMResult

    Take in a list of prompt values and return an LLMResult.

  classmethod construct(_fields_set: Optional[SetStr] = None, **values: Any) → Model

    Creates a new model setting __dict__ and __fields_set__ from trusted or pre-validated data. Default values are respected, but no other validation is performed. Behaves as if Config.extra = ‘allow’ was set since it adds all passed values

  copy(*, include: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, exclude: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, update: Optional[DictStrAny] = None, deep: bool = False) → Model

    Duplicate a model, optionally choose which fields to include, exclude and change.

    Parameters

      - include – fields to include in new model

      - exclude – fields to exclude from new model, as with values this takes precedence over include

      - update – values to change/add in the new model. Note: the data is not validated before creating the new model: you should trust this data

      - deep – set to True to make a deep copy of the model

    Returns

      new model instance

  dict(**kwargs: Any) → Dict

    Return a dictionary of the LLM.

  classmethod from_pipeline(pipeline: Any, hardware: Any, model_reqs: Optional[List[str]] = None, device: int = 0, **kwargs: Any) → langchain.llms.base.LLM

    Init the SelfHostedPipeline from a pipeline object or string.

  generate(prompts: List[str], stop: Optional[List[str]] = None) → langchain.schema.LLMResult

    Run the LLM on the given prompt and input.

  generate_prompt(prompts: List[langchain.schema.PromptValue], stop: Optional[List[str]] = None) → langchain.schema.LLMResult

    Take in a list of prompt values and return an LLMResult.

  get_num_tokens(text: str) → int

    Get the number of tokens present in the text.

  get_num_tokens_from_messages(messages: List[langchain.schema.BaseMessage]) → int

    Get the number of tokens in the message.

  json(*, include: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, exclude: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, by_alias: bool = False, skip_defaults: Optional[bool] = None, exclude_unset: bool = False, exclude_defaults: bool = False, exclude_none: bool = False, encoder: Optional[Callable[[Any], Any]] = None, models_as_dict: bool = True, **dumps_kwargs: Any) → unicode

    Generate a JSON representation of the model, include and exclude arguments as per dict().

    encoder is an optional function to supply as default to json.dumps(), other arguments as per json.dumps().

  save(file_path: Union[pathlib.Path, str]) → None

    Save the LLM.

    Parameters

      file_path – Path to file to save the LLM to.

    Example: .. code-block:: python

    > llm.save(file_path=”path/llm.yaml”)

  classmethod update_forward_refs(**localns: Any) → None

    Try to update ForwardRefs on fields based on this Model, globalns and localns.

pydantic model langchain.llms.SelfHostedPipeline

  Run model inference on self-hosted remote hardware.

  Supported hardware includes auto-launched instances on AWS, GCP, Azure, and Lambda, as well as servers specified by IP address and SSH credentials (such as on-prem, or another cloud like Paperspace, Coreweave, etc.).

  To use, you should have the `runhouse` python package installed.

  Example for custom pipeline and inference functions:

        from langchain.llms import SelfHostedPipeline
        from transformers import AutoModelForCausalLM, AutoTokenizer, pipeline
        import runhouse as rh

        def load_pipeline():
            tokenizer = AutoTokenizer.from_pretrained("gpt2")
            model = AutoModelForCausalLM.from_pretrained("gpt2")
            return pipeline(
                "text-generation", model=model, tokenizer=tokenizer,
                max_new_tokens=10
            )
        def inference_fn(pipeline, prompt, stop = None):
            return pipeline(prompt)[0]["generated_text"]

        gpu = rh.cluster(name="rh-a10x", instance_type="A100:1")
        llm = SelfHostedPipeline(
            model_load_fn=load_pipeline,
            hardware=gpu,
            model_reqs=model_reqs, inference_fn=inference_fn
        )

  Example for <2GB model (can be serialized and sent directly to the server):

        from langchain.llms import SelfHostedPipeline
        import runhouse as rh
        gpu = rh.cluster(name="rh-a10x", instance_type="A100:1")
        my_model = ...
        llm = SelfHostedPipeline.from_pipeline(
            pipeline=my_model,
            hardware=gpu,
            model_reqs=["./", "torch", "transformers"],
        )

  Example passing model path for larger models:

        from langchain.llms import SelfHostedPipeline
        import runhouse as rh
        import pickle
        from transformers import pipeline

        generator = pipeline(model="gpt2")
        rh.blob(pickle.dumps(generator), path="models/pipeline.pkl"
            ).save().to(gpu, path="models")
        llm = SelfHostedPipeline.from_pipeline(
            pipeline="models/pipeline.pkl",
            hardware=gpu,
            model_reqs=["./", "torch", "transformers"],
        )

  Validators

    - `set_callback_manager` » `callback_manager`

    - `set_verbose` » `verbose`

  field hardware: Any = None

    Remote hardware to send the inference function to.

  field inference_fn: Callable = <function _generate_text>

    Inference function to send to the remote hardware.

  field load_fn_kwargs: Optional[dict] = None

    Key word arguments to pass to the model load function.

  field model_load_fn: Callable [Required]

    Function to load the model remotely on the server.

  field model_reqs: List[str] = ['./', 'torch']

    Requirements to install on hardware to inference the model.

  __call__(prompt: str, stop: Optional[List[str]] = None) → str

    Check Cache and run the LLM on the given prompt and input.

  async agenerate(prompts: List[str], stop: Optional[List[str]] = None) → langchain.schema.LLMResult

    Run the LLM on the given prompt and input.

  async agenerate_prompt(prompts: List[langchain.schema.PromptValue], stop: Optional[List[str]] = None) → langchain.schema.LLMResult

    Take in a list of prompt values and return an LLMResult.

  classmethod construct(_fields_set: Optional[SetStr] = None, **values: Any) → Model

    Creates a new model setting __dict__ and __fields_set__ from trusted or pre-validated data. Default values are respected, but no other validation is performed. Behaves as if Config.extra = ‘allow’ was set since it adds all passed values

  copy(*, include: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, exclude: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, update: Optional[DictStrAny] = None, deep: bool = False) → Model

    Duplicate a model, optionally choose which fields to include, exclude and change.

    Parameters

      - include – fields to include in new model

      - exclude – fields to exclude from new model, as with values this takes precedence over include

      - update – values to change/add in the new model. Note: the data is not validated before creating the new model: you should trust this data

      - deep – set to True to make a deep copy of the model

    Returns

      new model instance

  dict(**kwargs: Any) → Dict

    Return a dictionary of the LLM.

  classmethod from_pipeline(pipeline: Any, hardware: Any, model_reqs: Optional[List[str]] = None, device: int = 0, **kwargs: Any) → langchain.llms.base.LLM

    Init the SelfHostedPipeline from a pipeline object or string.

  generate(prompts: List[str], stop: Optional[List[str]] = None) → langchain.schema.LLMResult

    Run the LLM on the given prompt and input.

  generate_prompt(prompts: List[langchain.schema.PromptValue], stop: Optional[List[str]] = None) → langchain.schema.LLMResult

    Take in a list of prompt values and return an LLMResult.

  get_num_tokens(text: str) → int

    Get the number of tokens present in the text.

  get_num_tokens_from_messages(messages: List[langchain.schema.BaseMessage]) → int

    Get the number of tokens in the message.

  json(*, include: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, exclude: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, by_alias: bool = False, skip_defaults: Optional[bool] = None, exclude_unset: bool = False, exclude_defaults: bool = False, exclude_none: bool = False, encoder: Optional[Callable[[Any], Any]] = None, models_as_dict: bool = True, **dumps_kwargs: Any) → unicode

    Generate a JSON representation of the model, include and exclude arguments as per dict().

    encoder is an optional function to supply as default to json.dumps(), other arguments as per json.dumps().

  save(file_path: Union[pathlib.Path, str]) → None

    Save the LLM.

    Parameters

      file_path – Path to file to save the LLM to.

    Example: .. code-block:: python

    > llm.save(file_path=”path/llm.yaml”)

  classmethod update_forward_refs(**localns: Any) → None

    Try to update ForwardRefs on fields based on this Model, globalns and localns.

pydantic model langchain.llms.StochasticAI

  Wrapper around StochasticAI large language models.

  To use, you should have the environment variable `STOCHASTICAI_API_KEY` set with your API key.

  Example

      from langchain.llms import StochasticAI
      stochasticai = StochasticAI(api_url="")

  Validators

    - `build_extra` » `all fields`

    - `set_callback_manager` » `callback_manager`

    - `set_verbose` » `verbose`

    - `validate_environment` » `all fields`

  field api_url: str = ''

    Model name to use.

  field model_kwargs: Dict[str, Any] [Optional]

    Holds any model parameters valid for create call not explicitly specified.

  __call__(prompt: str, stop: Optional[List[str]] = None) → str

    Check Cache and run the LLM on the given prompt and input.

  async agenerate(prompts: List[str], stop: Optional[List[str]] = None) → langchain.schema.LLMResult

    Run the LLM on the given prompt and input.

  async agenerate_prompt(prompts: List[langchain.schema.PromptValue], stop: Optional[List[str]] = None) → langchain.schema.LLMResult

    Take in a list of prompt values and return an LLMResult.

  classmethod construct(_fields_set: Optional[SetStr] = None, **values: Any) → Model

    Creates a new model setting __dict__ and __fields_set__ from trusted or pre-validated data. Default values are respected, but no other validation is performed. Behaves as if Config.extra = ‘allow’ was set since it adds all passed values

  copy(*, include: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, exclude: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, update: Optional[DictStrAny] = None, deep: bool = False) → Model

    Duplicate a model, optionally choose which fields to include, exclude and change.

    Parameters

      - include – fields to include in new model

      - exclude – fields to exclude from new model, as with values this takes precedence over include

      - update – values to change/add in the new model. Note: the data is not validated before creating the new model: you should trust this data

      - deep – set to True to make a deep copy of the model

    Returns

      new model instance

  dict(**kwargs: Any) → Dict

    Return a dictionary of the LLM.

  generate(prompts: List[str], stop: Optional[List[str]] = None) → langchain.schema.LLMResult

    Run the LLM on the given prompt and input.

  generate_prompt(prompts: List[langchain.schema.PromptValue], stop: Optional[List[str]] = None) → langchain.schema.LLMResult

    Take in a list of prompt values and return an LLMResult.

  get_num_tokens(text: str) → int

    Get the number of tokens present in the text.

  get_num_tokens_from_messages(messages: List[langchain.schema.BaseMessage]) → int

    Get the number of tokens in the message.

  json(*, include: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, exclude: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, by_alias: bool = False, skip_defaults: Optional[bool] = None, exclude_unset: bool = False, exclude_defaults: bool = False, exclude_none: bool = False, encoder: Optional[Callable[[Any], Any]] = None, models_as_dict: bool = True, **dumps_kwargs: Any) → unicode

    Generate a JSON representation of the model, include and exclude arguments as per dict().

    encoder is an optional function to supply as default to json.dumps(), other arguments as per json.dumps().

  save(file_path: Union[pathlib.Path, str]) → None

    Save the LLM.

    Parameters

      file_path – Path to file to save the LLM to.

    Example: .. code-block:: python

    > llm.save(file_path=”path/llm.yaml”)

  classmethod update_forward_refs(**localns: Any) → None

    Try to update ForwardRefs on fields based on this Model, globalns and localns.

pydantic model langchain.llms.Writer

  Wrapper around Writer large language models.

  To use, you should have the environment variable `WRITER_API_KEY` set with your API key.

  Example

      from langchain import Writer
      writer = Writer(model_id="palmyra-base")

  Validators

    - `set_callback_manager` » `callback_manager`

    - `set_verbose` » `verbose`

    - `validate_environment` » `all fields`

  field base_url: Optional[str] = None

    Base url to use, if None decides based on model name.

  field beam_search_diversity_rate: float = 1.0

    Only applies to beam search, i.e. when the beam width is >1. A higher value encourages beam search to return a more diverse set of candidates

  field beam_width: Optional[int] = None

    The number of concurrent candidates to keep track of during beam search

  field length: int = 256

    The maximum number of tokens to generate in the completion.

  field length_pentaly: float = 1.0

    Only applies to beam search, i.e. when the beam width is >1. Larger values penalize long candidates more heavily, thus preferring shorter candidates

  field logprobs: bool = False

    Whether to return log probabilities.

  field model_id: str = 'palmyra-base'

    Model name to use.

  field random_seed: int = 0

    The model generates random results. Changing the random seed alone will produce a different response with similar characteristics. It is possible to reproduce results by fixing the random seed (assuming all other hyperparameters are also fixed)

  field repetition_penalty: float = 1.0

    Penalizes repeated tokens according to frequency.

  field stop: Optional[List[str]] = None

    Sequences when completion generation will stop

  field temperature: float = 1.0

    What sampling temperature to use.

  field tokens_to_generate: int = 24

    Max number of tokens to generate.

  field top_k: int = 1

    The number of highest probability vocabulary tokens to keep for top-k-filtering.

  field top_p: float = 1.0

    Total probability mass of tokens to consider at each step.

  __call__(prompt: str, stop: Optional[List[str]] = None) → str

    Check Cache and run the LLM on the given prompt and input.

  async agenerate(prompts: List[str], stop: Optional[List[str]] = None) → langchain.schema.LLMResult

    Run the LLM on the given prompt and input.

  async agenerate_prompt(prompts: List[langchain.schema.PromptValue], stop: Optional[List[str]] = None) → langchain.schema.LLMResult

    Take in a list of prompt values and return an LLMResult.

  classmethod construct(_fields_set: Optional[SetStr] = None, **values: Any) → Model

    Creates a new model setting __dict__ and __fields_set__ from trusted or pre-validated data. Default values are respected, but no other validation is performed. Behaves as if Config.extra = ‘allow’ was set since it adds all passed values

  copy(*, include: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, exclude: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, update: Optional[DictStrAny] = None, deep: bool = False) → Model

    Duplicate a model, optionally choose which fields to include, exclude and change.

    Parameters

      - include – fields to include in new model

      - exclude – fields to exclude from new model, as with values this takes precedence over include

      - update – values to change/add in the new model. Note: the data is not validated before creating the new model: you should trust this data

      - deep – set to True to make a deep copy of the model

    Returns

      new model instance

  dict(**kwargs: Any) → Dict

    Return a dictionary of the LLM.

  generate(prompts: List[str], stop: Optional[List[str]] = None) → langchain.schema.LLMResult

    Run the LLM on the given prompt and input.

  generate_prompt(prompts: List[langchain.schema.PromptValue], stop: Optional[List[str]] = None) → langchain.schema.LLMResult

    Take in a list of prompt values and return an LLMResult.

  get_num_tokens(text: str) → int

    Get the number of tokens present in the text.

  get_num_tokens_from_messages(messages: List[langchain.schema.BaseMessage]) → int

    Get the number of tokens in the message.

  json(*, include: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, exclude: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, by_alias: bool = False, skip_defaults: Optional[bool] = None, exclude_unset: bool = False, exclude_defaults: bool = False, exclude_none: bool = False, encoder: Optional[Callable[[Any], Any]] = None, models_as_dict: bool = True, **dumps_kwargs: Any) → unicode

    Generate a JSON representation of the model, include and exclude arguments as per dict().

    encoder is an optional function to supply as default to json.dumps(), other arguments as per json.dumps().

  save(file_path: Union[pathlib.Path, str]) → None

    Save the LLM.

    Parameters

      file_path – Path to file to save the LLM to.

    Example: .. code-block:: python

    > llm.save(file_path=”path/llm.yaml”)

  classmethod update_forward_refs(**localns: Any) → None

    Try to update ForwardRefs on fields based on this Model, globalns and localns.