Toggle Menu
Prev Up Next

# `langchain_community.document_loaders.url_selenium`.SeleniumURLLoader

class langchain_community.document_loaders.url_selenium.SeleniumURLLoader(urls: List[str], continue_on_failure: bool = True, browser: Literal['chrome', 'firefox'] = 'chrome', binary_location: Optional[str] = None, executable_path: Optional[str] = None, headless: bool = True, arguments: List[str] = [])

  Load HTML pages with Selenium and parse with Unstructured.

  This is useful for loading pages that require javascript to render.

  Parameters

    - urls (List[str]) –

    - continue_on_failure (bool) –

    - browser (Literal['chrome', 'firefox']) –

    - binary_location (Optional[str]) –

    - executable_path (Optional[str]) –

    - headless (bool) –

    - arguments (List[str]) –

  urls

    List of URLs to load.

    Type

      List[str]

  continue_on_failure

    If True, continue loading other URLs on failure.

    Type

      bool

  browser

    The browser to use, either ‘chrome’ or ‘firefox’.

    Type

      str

  binary_location

    The location of the browser binary.

    Type

      Optional[str]

  executable_path

    The path to the browser executable.

    Type

      Optional[str]

  headless

    If True, the browser will run in headless mode.

    Type

      bool

  arguments [List[str]]

    List of arguments to pass to the browser.

  Load a list of URLs using Selenium and unstructured.

  Methods

  `__init__`(urls[, continue_on_failure, ...]) | Load a list of URLs using Selenium and unstructured.
  `alazy_load`() | A lazy loader for Documents.
  `aload`() | Load data into Document objects.
  `lazy_load`() | A lazy loader for Documents.
  `load`() | Load the specified URLs using Selenium and create Document instances.
  `load_and_split`([text_splitter]) | Load Documents and split into chunks.

  __init__(urls: List[str], continue_on_failure: bool = True, browser: Literal['chrome', 'firefox'] = 'chrome', binary_location: Optional[str] = None, executable_path: Optional[str] = None, headless: bool = True, arguments: List[str] = [])

    Load a list of URLs using Selenium and unstructured.

    Parameters

      - urls (List[str]) –

      - continue_on_failure (bool) –

      - browser (Literal['chrome', 'firefox']) –

      - binary_location (Optional[str]) –

      - executable_path (Optional[str]) –

      - headless (bool) –

      - arguments (List[str]) –

  async alazy_load() → AsyncIterator[Document]

    A lazy loader for Documents.

    Return type

      AsyncIterator[Document]

  async aload() → List[Document]

    Load data into Document objects.

    Return type

      List[Document]

  lazy_load() → Iterator[Document]

    A lazy loader for Documents.

    Return type

      Iterator[Document]

  load() → List[Document]

    Load the specified URLs using Selenium and create Document instances.

    Returns

      A list of Document instances with loaded content.

    Return type

      List[Document]

  load_and_split(text_splitter: Optional[TextSplitter] = None) → List[Document]

    Load Documents and split into chunks. Chunks are returned as Documents.

    Do not override this method. It should be considered to be deprecated!

    Parameters

      text_splitter (Optional[TextSplitter]) – TextSplitter instance to use for splitting documents. Defaults to RecursiveCharacterTextSplitter.

    Returns

      List of Documents.

    Return type

      List[Document]

# Examples using SeleniumURLLoader

- url.md