langchain_nvidia_ai_endpoints.reranking.NVIDIARerank

class langchain_nvidia_ai_endpoints.reranking.NVIDIARerank[source]

Bases: BaseDocumentCompressor

LangChain Document Compressor that uses the NVIDIA NeMo Retriever Reranking API.

Create a new NVIDIARerank document compressor.

Unless you plan to use the “nim” mode, you need to provide an API key. Your options are -

  1. Pass the key as the nvidia_api_key parameter.

  2. Pass the key as the api_key parameter.

  3. Set the NVIDIA_API_KEY environment variable, recommended.

Precedence is in the order listed above.

param max_batch_size: int = 32

The maximum batch size.

Constraints

  • minimum = 1

param model: str = 'ai-rerank-qa-mistral-4b'

The model to use for reranking.

param top_n: int = 5

The number of documents to return.

Constraints

  • minimum = 0

async acompress_documents(documents: Sequence[Document], query: str, callbacks: Optional[Union[List[BaseCallbackHandler], BaseCallbackManager]] = None) → Sequence[Document]

Compress retrieved documents given the query context.

Parameters
Return type

Sequence[Document]

compress_documents(documents: Sequence[Document], query: str, callbacks: Optional[Union[List[BaseCallbackHandler], BaseCallbackManager]] = None) → Sequence[Document][source]

Compress documents using the NVIDIA NeMo Retriever Reranking microservice API.

Parameters

  • documents (Sequence[Document]) – A sequence of documents to compress.

  • query (str) – The query to use for compressing the documents.

  • callbacks (Optional[Union[List[BaseCallbackHandler], BaseCallbackManager]]) – Callbacks to run during the compression process.

Returns

A sequence of compressed documents.

Return type

Sequence[Document]

classmethod construct(_fields_set: Optional[SetStr] = None, **values: Any) → Model

Creates a new model setting __dict__ and __fields_set__ from trusted or pre-validated data. Default values are respected, but no other validation is performed. Behaves as if Config.extra = ‘allow’ was set since it adds all passed values

Parameters

  • _fields_set (Optional[SetStr]) –

  • values (Any) –

Return type

Model

copy(*, include: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, exclude: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, update: Optional[DictStrAny] = None, deep: bool = False) → Model

Duplicate a model, optionally choose which fields to include, exclude and change.

Parameters

  • include (Optional[Union[AbstractSetIntStr, MappingIntStrAny]]) – fields to include in new model

  • exclude (Optional[Union[AbstractSetIntStr, MappingIntStrAny]]) – fields to exclude from new model, as with values this takes precedence over include

  • update (Optional[DictStrAny]) – values to change/add in the new model. Note: the data is not validated before creating the new model: you should trust this data

  • deep (bool) – set to True to make a deep copy of the model

  • self (Model) –

Returns

new model instance

Return type

Model

dict(*, include: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, exclude: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, by_alias: bool = False, skip_defaults: Optional[bool] = None, exclude_unset: bool = False, exclude_defaults: bool = False, exclude_none: bool = False) → DictStrAny

Generate a dictionary representation of the model, optionally specifying which fields to include or exclude.

Parameters

  • include (Optional[Union[AbstractSetIntStr, MappingIntStrAny]]) –

  • exclude (Optional[Union[AbstractSetIntStr, MappingIntStrAny]]) –

  • by_alias (bool) –

  • skip_defaults (Optional[bool]) –

  • exclude_unset (bool) –

  • exclude_defaults (bool) –

  • exclude_none (bool) –

Return type

DictStrAny

classmethod from_orm(obj: Any) → Model
Parameters

obj (Any) –

Return type

Model

classmethod get_available_models(mode: Optional[Literal['catalog', 'nvidia', 'nim', 'open', 'openai']] = None, list_all: bool = False, **kwargs: Any) → List[Model][source]

Get a list of available models. These models will work with the NVIDIARerank interface.

Use the mode parameter to specify the mode to use. See the docs for mode() to understand additional keyword arguments required when setting mode.

It is possible to get a list of all models, including those that are not chat models, by setting the list_all parameter to True.

Parameters

  • mode (Optional[Literal['catalog', 'nvidia', 'nim', 'open', 'openai']]) –

  • list_all (bool) –

  • kwargs (Any) –

Return type

List[Model]

json(*, include: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, exclude: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, by_alias: bool = False, skip_defaults: Optional[bool] = None, exclude_unset: bool = False, exclude_defaults: bool = False, exclude_none: bool = False, encoder: Optional[Callable[[Any], Any]] = None, models_as_dict: bool = True, **dumps_kwargs: Any) → unicode

Generate a JSON representation of the model, include and exclude arguments as per dict().

encoder is an optional function to supply as default to json.dumps(), other arguments as per json.dumps().

Parameters

  • include (Optional[Union[AbstractSetIntStr, MappingIntStrAny]]) –

  • exclude (Optional[Union[AbstractSetIntStr, MappingIntStrAny]]) –

  • by_alias (bool) –

  • skip_defaults (Optional[bool]) –

  • exclude_unset (bool) –

  • exclude_defaults (bool) –

  • exclude_none (bool) –

  • encoder (Optional[Callable[[Any], Any]]) –

  • models_as_dict (bool) –

  • dumps_kwargs (Any) –

Return type

unicode

mode(mode: Optional[Literal['catalog', 'nvidia', 'nim', 'open', 'openai']] = 'nvidia', base_url: Optional[str] = None, model: Optional[str] = None, api_key: Optional[str] = None, **kwargs: Any) → NVIDIARerank[source]

Change the mode.

There are two modes, “nvidia” and “nim”. The “nvidia” mode is the default mode and is used to interact with hosted NVIDIA AI endpoints. The “nim” mode is used to interact with NVIDIA NIM endpoints, which are typically hosted on-premises.

For the “nvidia” mode, the “api_key” parameter is available to specify your API key. If not specified, the NVIDIA_API_KEY environment variable will be used.

For the “nim” mode, the “base_url” and “model” parameters are required. Set base_url to the url of your NVIDIA NIM endpoint. For instance, “https://localhost:9999/v1”. Additionally, the “model” parameter must be set to the name of the model inside the NIM.

Parameters

  • mode (Optional[Literal['catalog', 'nvidia', 'nim', 'open', 'openai']]) –

  • base_url (Optional[str]) –

  • model (Optional[str]) –

  • api_key (Optional[str]) –

  • kwargs (Any) –

Return type

NVIDIARerank

classmethod parse_file(path: Union[str, Path], *, content_type: unicode = None, encoding: unicode = 'utf8', proto: Protocol = None, allow_pickle: bool = False) → Model
Parameters

  • path (Union[str, Path]) –

  • content_type (unicode) –

  • encoding (unicode) –

  • proto (Protocol) –

  • allow_pickle (bool) –

Return type

Model

classmethod parse_obj(obj: Any) → Model
Parameters

obj (Any) –

Return type

Model

classmethod parse_raw(b: Union[str, bytes], *, content_type: unicode = None, encoding: unicode = 'utf8', proto: Protocol = None, allow_pickle: bool = False) → Model
Parameters

  • b (Union[str, bytes]) –

  • content_type (unicode) –

  • encoding (unicode) –

  • proto (Protocol) –

  • allow_pickle (bool) –

Return type

Model

classmethod schema(by_alias: bool = True, ref_template: unicode = '#/definitions/{model}') → DictStrAny
Parameters

  • by_alias (bool) –

  • ref_template (unicode) –

Return type

DictStrAny

classmethod schema_json(*, by_alias: bool = True, ref_template: unicode = '#/definitions/{model}', **dumps_kwargs: Any) → unicode
Parameters

  • by_alias (bool) –

  • ref_template (unicode) –

  • dumps_kwargs (Any) –

Return type

unicode

classmethod update_forward_refs(**localns: Any) → None

Try to update ForwardRefs on fields based on this Model, globalns and localns.

Parameters

localns (Any) –

Return type

None

classmethod validate(value: Any) → Model
Parameters

value (Any) –

Return type

Model

property available_models: List[Model]

Get a list of available models that work with NVIDIARerank.