Toggle Menu
Prev Up Next

# `langchain_community.document_loaders.diffbot`.DiffbotLoader

class langchain_community.document_loaders.diffbot.DiffbotLoader(api_token: str, urls: List[str], continue_on_failure: bool = True)

  Load Diffbot json file.

  Initialize with API token, ids, and key.

  Parameters

    - api_token (str) – Diffbot API token.

    - urls (List[str]) – List of URLs to load.

    - continue_on_failure (bool) – Whether to continue loading other URLs if one fails. Defaults to True.

  Methods

  `__init__`(api_token, urls[, continue_on_failure]) | Initialize with API token, ids, and key.
  `alazy_load`() | A lazy loader for Documents.
  `aload`() | Load data into Document objects.
  `lazy_load`() | A lazy loader for Documents.
  `load`() | Extract text from Diffbot on all the URLs and return Documents
  `load_and_split`([text_splitter]) | Load Documents and split into chunks.

  __init__(api_token: str, urls: List[str], continue_on_failure: bool = True)

    Initialize with API token, ids, and key.

    Parameters

      - api_token (str) – Diffbot API token.

      - urls (List[str]) – List of URLs to load.

      - continue_on_failure (bool) – Whether to continue loading other URLs if one fails. Defaults to True.

  async alazy_load() → AsyncIterator[Document]

    A lazy loader for Documents.

    Return type

      AsyncIterator[Document]

  async aload() → List[Document]

    Load data into Document objects.

    Return type

      List[Document]

  lazy_load() → Iterator[Document]

    A lazy loader for Documents.

    Return type

      Iterator[Document]

  load() → List[Document]

    Extract text from Diffbot on all the URLs and return Documents

    Return type

      List[Document]

  load_and_split(text_splitter: Optional[TextSplitter] = None) → List[Document]

    Load Documents and split into chunks. Chunks are returned as Documents.

    Do not override this method. It should be considered to be deprecated!

    Parameters

      text_splitter (Optional[TextSplitter]) – TextSplitter instance to use for splitting documents. Defaults to RecursiveCharacterTextSplitter.

    Returns

      List of Documents.

    Return type

      List[Document]

# Examples using DiffbotLoader

- Diffbot

- diffbot.md