Toggle Menu
Prev Up Next

# `langchain_community.document_loaders.text`.TextLoader

class langchain_community.document_loaders.text.TextLoader(file_path: Union[str, Path], encoding: Optional[str] = None, autodetect_encoding: bool = False)

  Load text file.

  Parameters

    - file_path (Union[str, Path]) – Path to the file to load.

    - encoding (Optional[str]) – File encoding to use. If None, the file will be loaded

    - encoding. (with the default system) –

    - autodetect_encoding (bool) – Whether to try to autodetect the file encoding if the specified encoding fails.

  Initialize with file path.

  Methods

  `__init__`(file_path[, encoding, ...]) | Initialize with file path.
  `alazy_load`() | A lazy loader for Documents.
  `aload`() | Load data into Document objects.
  `lazy_load`() | Load from file path.
  `load`() | Load data into Document objects.
  `load_and_split`([text_splitter]) | Load Documents and split into chunks.

  __init__(file_path: Union[str, Path], encoding: Optional[str] = None, autodetect_encoding: bool = False)

    Initialize with file path.

    Parameters

      - file_path (Union[str, Path]) –

      - encoding (Optional[str]) –

      - autodetect_encoding (bool) –

  async alazy_load() → AsyncIterator[Document]

    A lazy loader for Documents.

    Return type

      AsyncIterator[Document]

  async aload() → List[Document]

    Load data into Document objects.

    Return type

      List[Document]

  lazy_load() → Iterator[Document]

    Load from file path.

    Return type

      Iterator[Document]

  load() → List[Document]

    Load data into Document objects.

    Return type

      List[Document]

  load_and_split(text_splitter: Optional[TextSplitter] = None) → List[Document]

    Load Documents and split into chunks. Chunks are returned as Documents.

    Do not override this method. It should be considered to be deprecated!

    Parameters

      text_splitter (Optional[TextSplitter]) – TextSplitter instance to use for splitting documents. Defaults to RecursiveCharacterTextSplitter.

    Returns

      List of Documents.

    Return type

      List[Document]

# Examples using TextLoader

- # Automatically restart kernel after installs so that your environment can access the new packages

- # if you plan to use bson serialization, install also:

- %pip install –upgrade –quiet surrealdb langchain langchain-community

- 1. Vanilla RAG {#vanilla-rag-1}

- Create a bes instance and index docs.

- Create collection if running for the first time. If the collection

- Dependencies {#dependencies}

- Document loaders

- DocumentDB connection string

- Establishing a connection to the database is facilitated through the singlestoredb Python connector.

- File Directory

- Get an OpenAI token: https://platform.openai.com/account/api-keys

- Helper function for printing docs

- Here we useimport getpass

- Here we want to make sure the answer is minimally relevant

- If using the default Docker installation, use this instantiation instead:

- Load in document to retrieve over

- Load the document and split it into chunks

- Metadata {#metadata}

- OR

- OR (depending on Python version)

- Option 1: use an OpenAI account

- Pip install necessary package

- Pip install necessary package {#pip-install-necessary-package}

- Pip install necessary packages

- Refresh is required for server use

- Retrievers

- Run tests with shell:

- Set up the OpenAI Environment Variables

- Setup {#setup}

- Swapping the `ByteStore` {#swapping-the-bytestore}

- The vectorstore to use to index the child chunks

- This text splitter is used to create the child documents

- This will only get documents for Ankush

- Use Meilisearch vector store to store texts & associated embeddings as vector

- VDMS

- Vector stores

- Wait until the cluster is ready for use.

- You need to install sqlite-vss as a dependency.

- add texts

- analyticdb.md

- atlas.md

- awadb.md

- azure_ai_search.md

- baiduvectordb.md

- clickhouse.md

- conversational_retrieval_agents.md

- cosine: distance metric

- create cluster and add texts

- databricks_vector_search.md

- default metric is angular

- docs[0].metadata[“id”] == “id:testapp:testapp::32”

- drop first if index already exists

- duckdb.md

- ecloud_vector_search.md

- epsilla.md

- for example

- from langchain_community.embeddings.openai import OpenAIEmbeddings

- hologres.md

- import

- initialize marqo

- install package

- lancedb.md

- openai

- or install latest:

- or shorter

- os.environ[“LANGCHAIN_TRACING_V2”] = “true”

- output length: 4

- pinecone.md

- qdrant.md

- replace

- scann.md

- tigris.md

- tiledb.md

- typesense.md

- use directly a `where_str` to delete

- usearch.md

- vectorstore.md

- vikingdb.md

- with pip

- xata.md