Toggle Menu
Prev Up Next

# `langchain_community.document_loaders.wikipedia`.WikipediaLoader

class langchain_community.document_loaders.wikipedia.WikipediaLoader(query: str, lang: str = 'en', load_max_docs: Optional[int] = 25, load_all_available_meta: Optional[bool] = False, doc_content_chars_max: Optional[int] = 4000)

  Load from Wikipedia.

  The hard limit on the length of the query is 300 for now.

  Each wiki page represents one Document.

  Initializes a new instance of the WikipediaLoader class.

  Parameters

    - query (str) – The query string to search on Wikipedia.

    - lang (str, optional) – The language code for the Wikipedia language edition. Defaults to “en”.

    - load_max_docs (int, optional) – The maximum number of documents to load. Defaults to 100.

    - load_all_available_meta (bool, optional) – Indicates whether to load all available metadata for each document. Defaults to False.

    - doc_content_chars_max (int, optional) – The maximum number of characters for the document content. Defaults to 4000.

  Methods

  `__init__`(query[, lang, load_max_docs, ...]) | Initializes a new instance of the WikipediaLoader class.
  `alazy_load`() | A lazy loader for Documents.
  `aload`() | Load data into Document objects.
  `lazy_load`() | Loads the query result from Wikipedia into a list of Documents.
  `load`() | Load data into Document objects.
  `load_and_split`([text_splitter]) | Load Documents and split into chunks.

  __init__(query: str, lang: str = 'en', load_max_docs: Optional[int] = 25, load_all_available_meta: Optional[bool] = False, doc_content_chars_max: Optional[int] = 4000)

    Initializes a new instance of the WikipediaLoader class.

    Parameters

      - query (str) – The query string to search on Wikipedia.

      - lang (str, optional) – The language code for the Wikipedia language edition. Defaults to “en”.

      - load_max_docs (int, optional) – The maximum number of documents to load. Defaults to 100.

      - load_all_available_meta (bool, optional) – Indicates whether to load all available metadata for each document. Defaults to False.

      - doc_content_chars_max (int, optional) – The maximum number of characters for the document content. Defaults to 4000.

  async alazy_load() → AsyncIterator[Document]

    A lazy loader for Documents.

    Return type

      AsyncIterator[Document]

  async aload() → List[Document]

    Load data into Document objects.

    Return type

      List[Document]

  lazy_load() → Iterator[Document]

    Loads the query result from Wikipedia into a list of Documents.

    Returns

      A list of Document objects representing the loaded

        Wikipedia pages.

    Return type

      Iterator[Document]

  load() → List[Document]

    Load data into Document objects.

    Return type

      List[Document]

  load_and_split(text_splitter: Optional[TextSplitter] = None) → List[Document]

    Load Documents and split into chunks. Chunks are returned as Documents.

    Do not override this method. It should be considered to be deprecated!

    Parameters

      text_splitter (Optional[TextSplitter]) – TextSplitter instance to use for splitting documents. Defaults to RecursiveCharacterTextSplitter.

    Returns

      List of Documents.

    Return type

      List[Document]

# Examples using WikipediaLoader

- Wikipedia

- diffbot.md

- wikipedia.md