Toggle Menu
Prev Up Next

# `langchain_community.document_loaders.html_bs`.BSHTMLLoader

class langchain_community.document_loaders.html_bs.BSHTMLLoader(file_path: Union[str, Path], open_encoding: Optional[str] = None, bs_kwargs: Optional[dict] = None, get_text_separator: str = '')

  Load HTML files and parse them with beautiful soup.

  initialize with path, and optionally, file encoding to use, and any kwargs to pass to the BeautifulSoup object.

  Parameters

    - file_path (Union[str, Path]) – The path to the file to load.

    - open_encoding (Optional[str]) – The encoding to use when opening the file.

    - bs_kwargs (Optional[dict]) – Any kwargs to pass to the BeautifulSoup object.

    - get_text_separator (str) – The separator to use when calling get_text on the soup.

  Methods

  `__init__`(file_path[, open_encoding, ...]) | initialize with path, and optionally, file encoding to use, and any kwargs to pass to the BeautifulSoup object.
  `alazy_load`() | A lazy loader for Documents.
  `aload`() | Load data into Document objects.
  `lazy_load`() | Load HTML document into document objects.
  `load`() | Load data into Document objects.
  `load_and_split`([text_splitter]) | Load Documents and split into chunks.

  __init__(file_path: Union[str, Path], open_encoding: Optional[str] = None, bs_kwargs: Optional[dict] = None, get_text_separator: str = '') → None

    initialize with path, and optionally, file encoding to use, and any kwargs to pass to the BeautifulSoup object.

    Parameters

      - file_path (Union[str, Path]) – The path to the file to load.

      - open_encoding (Optional[str]) – The encoding to use when opening the file.

      - bs_kwargs (Optional[dict]) – Any kwargs to pass to the BeautifulSoup object.

      - get_text_separator (str) – The separator to use when calling get_text on the soup.

    Return type

      None

  async alazy_load() → AsyncIterator[Document]

    A lazy loader for Documents.

    Return type

      AsyncIterator[Document]

  async aload() → List[Document]

    Load data into Document objects.

    Return type

      List[Document]

  lazy_load() → Iterator[Document]

    Load HTML document into document objects.

    Return type

      Iterator[Document]

  load() → List[Document]

    Load data into Document objects.

    Return type

      List[Document]

  load_and_split(text_splitter: Optional[TextSplitter] = None) → List[Document]

    Load Documents and split into chunks. Chunks are returned as Documents.

    Do not override this method. It should be considered to be deprecated!

    Parameters

      text_splitter (Optional[TextSplitter]) – TextSplitter instance to use for splitting documents. Defaults to RecursiveCharacterTextSplitter.

    Returns

      List of Documents.

    Return type

      List[Document]

# Examples using BSHTMLLoader

- Download the content

- HTML