Toggle Menu
Prev Up Next

# `langchain_community.embeddings.xinference`.XinferenceEmbeddings

class langchain_community.embeddings.xinference.XinferenceEmbeddings(server_url: Optional[str] = None, model_uid: Optional[str] = None)

  Xinference embedding models.

  To use, you should have the xinference library installed:

      pip install xinference

  Check out: https://github.com/xorbitsai/inference To run, you need to start a Xinference supervisor on one server and Xinference workers on the other servers.

  Example

  To start a local instance of Xinference, run

      $ xinference

  You can also deploy Xinference in a distributed cluster. Here are the steps:

  Starting the supervisor:

      $ xinference-supervisor

  Starting the worker:

      $ xinference-worker

  Then, launch a model using command line interface (CLI).

  Example:

      $ xinference launch -n orca -s 3 -q q4_0

  It will return a model UID. Then you can use Xinference Embedding with LangChain.

  Example:

      from langchain_community.embeddings import XinferenceEmbeddings

      xinference = XinferenceEmbeddings(
          server_url="http://0.0.0.0:9997",
          model_uid = {model_uid} # replace model_uid with the model UID return from launching the model
      )

  Attributes

  `client` |
  `server_url` | URL of the xinference server
  `model_uid` | UID of the launched model

  Methods

  `__init__`([server_url, model_uid]) |
  `aembed_documents`(texts) | Asynchronous Embed search docs.
  `aembed_query`(text) | Asynchronous Embed query text.
  `embed_documents`(texts) | Embed a list of documents using Xinference.
  `embed_query`(text) | Embed a query of documents using Xinference.

  Parameters

    - server_url (Optional[str]) –

    - model_uid (Optional[str]) –

  __init__(server_url: Optional[str] = None, model_uid: Optional[str] = None)
    Parameters

      - server_url (Optional[str]) –

      - model_uid (Optional[str]) –

  async aembed_documents(texts: List[str]) → List[List[float]]

    Asynchronous Embed search docs.

    Parameters

      texts (List[str]) –

    Return type

      List[List[float]]

  async aembed_query(text: str) → List[float]

    Asynchronous Embed query text.

    Parameters

      text (str) –

    Return type

      List[float]

  embed_documents(texts: List[str]) → List[List[float]]

    Embed a list of documents using Xinference. :param texts: The list of texts to embed.

    Returns

      List of embeddings, one for each text.

    Parameters

      texts (List[str]) –

    Return type

      List[List[float]]

  embed_query(text: str) → List[float]

    Embed a query of documents using Xinference. :param text: The text to embed.

    Returns

      Embeddings for the text.

    Parameters

      text (str) –

    Return type

      List[float]

# Examples using XinferenceEmbeddings

- xinference.md