Skip to main content
`Ctrl`+`K`

# ragoon.datasets

Functions

`dataset_loader`(name[, streaming, split]) | Helper function to load a single dataset in parallel.
`load_datasets`(req[, streaming]) | Downloads datasets specified in a list and creates a list of loaded datasets.

ragoon.datasets.dataset_loader(name: str, streaming: bool | None = True, split: str | List[str] | None = None) → Dataset

  Helper function to load a single dataset in parallel.

  Parameters:

    - name (str) – Name of the dataset to be loaded.

    - streaming (bool, optional) – Determines if datasets are streamed. Default is True.

    - split (Optional[Union[str, List[str]]], optional) – Which split of the data to load. If None, will return a dict with all splits (typically datasets.Split.TRAIN and datasets.Split.TEST). If given, will return a single Dataset. Splits can be combined and specified like in tensorflow-datasets.

  Returns:

    dataset – Loaded dataset object.

  Return type:

    datasets.Dataset

  Raises:

    Exception – If an error occurs during dataset loading.

ragoon.datasets.load_datasets(req: list, streaming: bool | None = False) → list

  Downloads datasets specified in a list and creates a list of loaded datasets.

  Parameters:

    - req (list) – A list containing the names of datasets to be downloaded.

    - streaming (bool, optional) – Determines if datasets are streamed. Default is False.

  Returns:

    datasets_list – A list containing loaded datasets as per the requested names provided in ‘req’.

  Return type:

    list

  Raises:

    Exception – If an error occurs during dataset loading or processing.

  Examples

      >>> req = [
      ...    "louisbrulenaudet/code-artisanat",
      ...    "louisbrulenaudet/code-action-sociale-familles",
      ... # ...
      ]

      >>> datasets_list = load_datasets(
      ...    req=req,
      ...    streaming=True
      )

      >>> dataset = datasets.concatenate_datasets(
      ...    datasets_list
      )