Home - News ( United States | United Kingdom | Italy | Germany ) - Football scores

Showing content from https://github.com/developmentseed/obstore below:

developmentseed/obstore: The simplest, highest-throughput Python interface to S3, GCS & Azure Storage, powered by Rust.

The simplest, highest-throughput ¹ Python interface to Amazon S3, Google Cloud Storage, Azure Storage, & other S3-compliant APIs, powered by Rust.

• One interface for all backends with no required Python dependencies.
• Sync and async API with full type hinting.
• Streaming downloads with configurable chunking.
• Streaming uploads from files or async or sync iterators.
• Streaming list, with no need to paginate.
• Automatic multipart uploads for large file objects.
• Automatic credential refresh before expiration.
• File-like object API and fsspec integration.
• Optionally return list results in Apache Arrow format, which is faster and more memory-efficient than materializing Python dicts.
• Zero-copy data exchange between Rust and Python via the buffer protocol.

For Rust developers looking to add object_store support to their own Python packages, refer to pyo3-object_store.

To install obstore using pip:

Obstore is on conda-forge and can be installed using conda, mamba, or pixi. To install obstore using conda:

conda install -c conda-forge obstore

Full documentation is available on the website.

Head to Getting Started to dig in.

1. Benchmarking is ongoing, but preliminary results indicate roughly 9x higher throughput than fsspec and 2.8x higher throughput than aioboto3 for many concurrent, small, get requests from an async context. ↩

RetroSearch is an open source project built by @garambo | Open a GitHub Issue

Search and Browse the WWW like it's 1997 | Search results from DuckDuckGo

HTML: 3.2 | Encoding: UTF-8 | Version: 0.7.4