Showing content from https://crawlee.dev/python/api/class/Dataset below:
Dataset | API | Crawlee for Python ยท Fast, reliable Python web crawlers.
Dataset Index Methods
- __init__(client, id, name): None
- async export_to(key: str, content_type?: Literal[json, csv], to_kvs_id?: str | None, to_kvs_name?: str | None, to_kvs_storage_client?: StorageClient | None, to_kvs_configuration?: Configuration | None, kwargs: Any): None
- async export_to(key: str, content_type: Literal[json], to_kvs_id?: str | None, to_kvs_name?: str | None, to_kvs_storage_client?: StorageClient | None, to_kvs_configuration?: Configuration | None, *: , skipkeys: NotRequired[bool], ensure_ascii: NotRequired[bool], check_circular: NotRequired[bool], allow_nan: NotRequired[bool], cls: NotRequired[type[json.JSONEncoder]], indent: NotRequired[int], separators: NotRequired[tuple[str, str]], default: NotRequired[Callable], sort_keys: NotRequired[bool]): None
- async export_to(key: str, content_type: Literal[csv], to_kvs_id?: str | None, to_kvs_name?: str | None, to_kvs_storage_client?: StorageClient | None, to_kvs_configuration?: Configuration | None, *: , dialect: NotRequired[str], delimiter: NotRequired[str], doublequote: NotRequired[bool], escapechar: NotRequired[str], lineterminator: NotRequired[str], quotechar: NotRequired[str], quoting: NotRequired[int], skipinitialspace: NotRequired[bool], strict: NotRequired[bool]): None
- Parameters
- key: str
- optionalcontent_type: Literal[json, csv] = 'json'
- optionalto_kvs_id: str | None = None
- optionalto_kvs_name: str | None = None
- optionalto_kvs_storage_client: StorageClient | None = None
- optionalto_kvs_configuration: Configuration | None = None
- kwargs: Any
Returns None
- async get_data(*, offset, limit, clean, desc, fields, omit, unwind, skip_empty, skip_hidden, flatten, view): DatasetItemsListPage
- Parameters
- optionalkeyword-onlyoffset: int = 0
- optionalkeyword-onlylimit: int | None = 999_999_999_999
- optionalkeyword-onlyclean: bool = False
- optionalkeyword-onlydesc: bool = False
- optionalkeyword-onlyfields: list[str] | None = None
- optionalkeyword-onlyomit: list[str] | None = None
- optionalkeyword-onlyunwind: list[str] | None = None
- optionalkeyword-onlyskip_empty: bool = False
- optionalkeyword-onlyskip_hidden: bool = False
- optionalkeyword-onlyflatten: list[str] | None = None
- optionalkeyword-onlyview: str | None = None
Returns DatasetItemsListPage
- async iterate_items(*, offset, limit, clean, desc, fields, omit, unwind, skip_empty, skip_hidden): AsyncIterator[dict[str, Any]]
- Parameters
- optionalkeyword-onlyoffset: int = 0
- optionalkeyword-onlylimit: int | None = 999_999_999_999
- optionalkeyword-onlyclean: bool = False
- optionalkeyword-onlydesc: bool = False
- optionalkeyword-onlyfields: list[str] | None = None
- optionalkeyword-onlyomit: list[str] | None = None
- optionalkeyword-onlyunwind: list[str] | None = None
- optionalkeyword-onlyskip_empty: bool = False
- optionalkeyword-onlyskip_hidden: bool = False
Returns AsyncIterator[dict[str, Any]]
- async list_items(*, offset, limit, clean, desc, fields, omit, unwind, skip_empty, skip_hidden): list[dict[str, Any]]
- Parameters
- optionalkeyword-onlyoffset: int = 0
- optionalkeyword-onlylimit: int | None = 999_999_999_999
- optionalkeyword-onlyclean: bool = False
- optionalkeyword-onlydesc: bool = False
- optionalkeyword-onlyfields: list[str] | None = None
- optionalkeyword-onlyomit: list[str] | None = None
- optionalkeyword-onlyunwind: list[str] | None = None
- optionalkeyword-onlyskip_empty: bool = False
- optionalkeyword-onlyskip_hidden: bool = False
Returns list[dict[str, Any]]
- async open(*, id, name, configuration, storage_client): Storage
-
Overrides Storage.open
Parameters
- optionalkeyword-onlyid: str | None = None
- optionalkeyword-onlyname: str | None = None
- optionalkeyword-onlyconfiguration: Configuration | None = None
- optionalkeyword-onlystorage_client: StorageClient | None = None
Returns Storage
- async push_data(data): None
- Parameters
- data: list[dict[str, Any]] | dict[str, Any]
Returns None
Properties
RetroSearch is an open source project built by @garambo
| Open a GitHub Issue
Search and Browse the WWW like it's 1997 | Search results from DuckDuckGo
HTML:
3.2
| Encoding:
UTF-8
| Version:
0.7.4