Toggle Menu
Prev Up Next

# `langchain_community.document_loaders.lakefs`.LakeFSLoader

class langchain_community.document_loaders.lakefs.LakeFSLoader(lakefs_access_key: str, lakefs_secret_key: str, lakefs_endpoint: str, repo: Optional[str] = None, ref: Optional[str] = 'main', path: Optional[str] = '')

  Load from lakeFS.

  Parameters

    - lakefs_access_key (str) – [required] lakeFS server’s access key

    - lakefs_secret_key (str) – [required] lakeFS server’s secret key

    - lakefs_endpoint (str) – [required] lakeFS server’s endpoint address, ex: https://example.my-lakefs.com

    - repo (str) – [optional, default = ‘’] target repository

    - ref (str) – [optional, default = ‘main’] target ref (branch name, tag, or commit ID)

    - path (str) – [optional, default = ‘’] target path

  Attributes

  `repo` |
  `ref` |
  `path` |

  Methods

  `__init__`(lakefs_access_key, ...[, repo, ...]) | param lakefs_access_key [required] lakeFS server's access key
  `alazy_load`() | A lazy loader for Documents.
  `aload`() | Load data into Document objects.
  `lazy_load`() | A lazy loader for Documents.
  `load`() | Load data into Document objects.
  `load_and_split`([text_splitter]) | Load Documents and split into chunks.
  `set_path`(path) |
  `set_ref`(ref) |
  `set_repo`(repo) |

  __init__(lakefs_access_key: str, lakefs_secret_key: str, lakefs_endpoint: str, repo: Optional[str] = None, ref: Optional[str] = 'main', path: Optional[str] = '')
    Parameters

      - lakefs_access_key (str) – [required] lakeFS server’s access key

      - lakefs_secret_key (str) – [required] lakeFS server’s secret key

      - lakefs_endpoint (str) – [required] lakeFS server’s endpoint address, ex: https://example.my-lakefs.com

      - repo (Optional[str]) – [optional, default = ‘’] target repository

      - ref (Optional[str]) – [optional, default = ‘main’] target ref (branch name, tag, or commit ID)

      - path (Optional[str]) – [optional, default = ‘’] target path

  async alazy_load() → AsyncIterator[Document]

    A lazy loader for Documents.

    Return type

      AsyncIterator[Document]

  async aload() → List[Document]

    Load data into Document objects.

    Return type

      List[Document]

  lazy_load() → Iterator[Document]

    A lazy loader for Documents.

    Return type

      Iterator[Document]

  load() → List[Document]

    Load data into Document objects.

    Return type

      List[Document]

  load_and_split(text_splitter: Optional[TextSplitter] = None) → List[Document]

    Load Documents and split into chunks. Chunks are returned as Documents.

    Do not override this method. It should be considered to be deprecated!

    Parameters

      text_splitter (Optional[TextSplitter]) – TextSplitter instance to use for splitting documents. Defaults to RecursiveCharacterTextSplitter.

    Returns

      List of Documents.

    Return type

      List[Document]

  set_path(path: str) → None
    Parameters

      path (str) –

    Return type

      None

  set_ref(ref: str) → None
    Parameters

      ref (str) –

    Return type

      None

  set_repo(repo: str) → None
    Parameters

      repo (str) –

    Return type

      None

# Examples using LakeFSLoader

- lakeFS

- lakefs.md