Class●Since v0.3

HuggingFaceDatasetLoader

HuggingFaceDatasetLoader(
  self,
  path: str,
  page_content_column: str = 'text',
  name: Optional

Bases

BaseLoader

Name	Type	Description
`path`*	`str`	Path or name of the dataset.
`page_content_column`	`str`	Default:`'text'` Page content column name. Default is "text".
`name`	`Optional[str]`	Default:`None` Name of the dataset configuration.
`data_dir`	`Optional[str]`	Default:`None`
`data_files`	`Optional[Union[str, Sequence[str], Mapping[str, Union[str, Sequence[str`
`cache_dir`	`Optional[str]`	Default:`None`
`keep_in_memory`	`Optional[bool]`	Default:`None`
`save_infos`	`bool`	Default:`False`
`use_auth_token`	`Optional[Union[bool, str]]`	Default:`None`
`num_proc`	`Optional[int]`	Default:`None`

Name	Type
path	str
page_content_column	str
name	Optional[str]
data_dir	Optional[str]
data_files	Optional[Union[str, Sequence[str], Mapping[str, Union[str, Sequence[str]]]]]
cache_dir	Optional[str]
keep_in_memory	Optional[bool]
save_infos	bool
use_auth_token	Optional[Union[bool, str]]
num_proc	Optional[int]