Showing content from https://github.com/cf-convention/cf-conventions/issues/204 below:
A convention for complex numbers? · Issue #369 · cf-convention/discuss · GitHub
This may not come up often for Climate and Forecast use-cases, but many physical scientists are interested in storing arrays of complex numbers. This has come up with some regularity for xarray users, e.g., see pydata/xarray#3297 for the most recent issue.
There does not appear to be a standard for how to store complex numbers in netCDF files, so scientists are currently presented with two poor options:
- Use their own de-facto convention, unsupported by software, or
- Rely on non-standard extensions of netCDF, e.g.,
invalid_netcdf=True with h5netcdf: https://github.com/shoyer/h5netcdf#invalid-netcdf-files
I would love to resolve this with a standard way to store complex values in netCDF file, so xarray doesn't have to invent its own standard or encourage writing HDF5 files that aren't valid in netCDF.
Some questions:
- Does it make sense to put this in CF-Conventions (the de-facto standard for how to write netCDF files), even though there do not appear to be climate/forecast use-cases for this?
- What should this look like? I can think of three reasonable approaches:
a. Use compound data types in a single array, like h5py and HDF5.jl (see complex number support JuliaIO/HDF5.jl#558). Advantages: this is an existing standard already in use. Disadvantages: requires the HDF5 data model; netCDF-C lacks support for reading some types compound data types ( NetCDF unable to read some HDF5 enums Unidata/netcdf-c#267), including the convention used by h5py.
b. Store real and imaginary parts in separate arrays, with some sort of metadata convention for indicating that they are the same logical array. Advantages: easy to interpret merely by inspection; easy to access real or imaginary parts separately. Disadvantages: a new convention; reading data into complex dtypes in memory will likely require an additional copy to combine real/imaginary parts.
c. Store real and imaginary parts in a single array with an extra dimension of length 2 at the end for real and imaginary parts. Advantages: still backwards compatible with old netCDF formats; can be memory mapped without a copy. Disadvantages: slightly less self-explanatory (user needs to understand the dimension mapping); adds an extra dangling dimension of length two in the dataset.
JimBiardCics, jhamman, prisae, iuryt and ocefpaf
RetroSearch is an open source project built by @garambo
| Open a GitHub Issue
Search and Browse the WWW like it's 1997 | Search results from DuckDuckGo
HTML:
3.2
| Encoding:
UTF-8
| Version:
0.7.4