RetroSearch Browse

Home - News ( United States | United Kingdom | Italy | Germany ) - Football scores

Showing content from https://github.com/codarcode/libpressio below:

CODARcode/libpressio: Pressio is latin for compression. Libpressio is a C++ library with C compatible bindings to abstract between different lossless and lossy compressors and their configurations. It solves the problem of having to having to write separate application level code for each lossy compressor that is developed. Instead, users write application level code using libpressio, and the library will make the correct underlying calls to the compressors. It provides interfaces to represent data, compressors settings, and compressors.

Pressio is latin for compression. LibPressio is a C++ library with C compatible bindings to abstract between different lossless and lossy compressors and their configurations. It solves the problem of having to having to write separate application level code for each lossy compressor that is developed. Instead, users write application level code using LibPressio, and the library will make the correct underlying calls to the compressors. It provides interfaces to represent data, compressors settings, and compressors.

Documentation for the master branch can be found here

Example using the CLI from pressio-tools We also have C, C++, Rust, Julia, and Python bindings.

pressio -i ~/git/datasets/hurricane/100x500x500/CLOUDf48.bin.f32 \
    -b compressor=sz3 -o abs=1e-4 -O all \
    -m time -m size -m error_stat -M all \
    -w /path/to/output.dec

The reccomended way to learn LibPressio is with self-paced LibPressio Tutorial. Here you will find examples of how to use LibPressio in a series of lessons for several common languages.

You can also find a recording of the tutorial on YouTube.

After skimming the example, LibPressio has 6 major headers that you will need to use:

Type Use pressio.h Error reporting and aquiring handles to compressors pressio_compressor.h Used to compress and decompress data, provided by plugins pressio_data.h Represents data and associated metadata (size, type, dimentionality, memory ownership) pressio_options.h Maps between names and values, used for options for compressors and metrics results pressio_metrics.h A set of metrics to run while compressors run pressio_io.h An extension header that provides methods to load or store data from/to persistent storage

All of these are included by the convience header libpressio.h.

You can pick up the more advanced features as you need them.

You can also find more examples in test/ or in the LibPressio intresting scripts collection which catalogs intresting higher-level use cases.

Supported Compressors and Metrics

Libpressio provides a number of builtin compressor and metrics modules. All of these are disabled by default. They can be enabled by passing the corresponding LIBPRESSIO_HAS_* variable to CMake.

Additionally, Libpressio is extensible. For information on writing a compressor plugin see Writing a Compressor Plugin For information on writing a metrics plugin see Writing a Metrics Plugin

1st party compressors plugins can be found in src/plugins/compressors

See the compressor settings page for information on how to configure them.

1st party compressors plugins can be found in src/plugins/metrics

See the metrics results page for information on what they produce

1st party compressors plugins can be found in src/plugins/io

See the io settings page for information on how to configure them

Installing LibPressio using Spack

LibPressio can be built using spack. This example will install libpressio with only the SZ3 plugin.

git clone https://github.com/spack/spack
source ./spack/share/spack/setup-env.sh
spack install libpressio+sz3

More information on spack can be found in the spack documentation or my quick start guides for systems that I use

You can see the other available versions and compilation options by calling spack info libpressio

The following language bindings are in this repository.

C -- (default) if you need a stable interface
C++ -- (default) if you want a more productive interface, or want to extend LibPressio
Python -- (+python; BUILD_PYTHON_WRAPPER) if you know or want to intergate Python
HDF5 -- (+hdf5+json; LIBPRESSIO_HAS_HDF AND LIBPRESSIO_HAS_JSON) you already use HDF5

The following bindings must be installed seperately:

R -- r-libpressio if you know or want to integrate with R
Bash/CLI -- libpressio-tools if you want to quickly prototype from the CLI

The following bindings are experimental and can be installed manually:

Julia -- libpressio-jl if you know or want to integrate with Julia
Rust -- libpressio-rs if you know or want to integrate with Rust

Doing a development build with spack

The easiest way to do a development build of libpressio is to use Spack envionments.

# one time setup: create an envionment
spack env create -d mydevenviroment
spack env activate mydevenvionment

# one time setup: tell spack to set LD_LIBRARY_PATH with the spack envionment's library paths
spack config add modules:prefix_inspections:lib64:[LD_LIBRARY_PATH]
spack config add modules:prefix_inspections:lib:[LD_LIBRARY_PATH]

# one time setup: install libpressio-tools and checkout 
# libpressio for development
spack add libpressio-tools
spack develop libpressio@git.master

# compile and install (repeat as needed)
spack install

Libpressio unconditionally requires:

cmake
pkg-config
std_compat
either:
- gcc-4.8.5 or later
- clang-7.0.0 or later using either libc++ or libstdc++. Beware that system libraries may need to be recompiled with libc++ if using libc++

Dependency versions and optional dependencies are documented in the spack package.

Configuring LibPressio Manually

LibPressio uses a fairly standard CMake buildsystem. For more information on CMake refer to these docs

The set of configuration options for LibPressio can be found using cmake -L $BUILD_DIR. For information on what these settings do, see the spack package

Please refer to docs/stability.md.

Please refer to CONTRIBUTORS.md for a list of contributors, sponsors, and contribution guidelines.

Please files bugs to the Github Issues page on the CODARCode libpressio repository.

Please read this post on how to file a good bug report. After reading this post, please provide the following information specific to libpressio:

Your OS version and distribution information, usually this can be found in /etc/os-release
the output of cmake -L $BUILD_DIR
the version of each of libpressio's dependencies listed in the README that you have installed. Where possible, please provide the commit hashes.

If you find LibPressio useful, please cite this paper:

@inproceedings{underwood2021productive,
  title={Productive and Performant Generic Lossy Data Compression with LibPressio},
  author={Underwood, Robert and Malvoso, Victoriana and Calhoun, Jon C and Di, Sheng and Cappello, Franck},
  booktitle={2021 7th International Workshop on Data Analysis and Reduction for Big Scientific Data (DRBSD-7)},
  pages={1--10},
  year={2021},
  organization={IEEE}
}

RetroSearch is an open source project built by @garambo | Open a GitHub Issue

Search and Browse the WWW like it's 1997 | Search results from DuckDuckGo

HTML: 3.2 | Encoding: UTF-8 | Version: 0.7.4