Hello, For the record there are number of initiatives currently to boost the usefulness and efficiency of multi-process computation in Python. One of them is PEP 574 (zero-copy pickling with out-of-band buffers), which I'm working on. Another is Pierre Glaser's work on allowing pickling of dynamic functions and classes with the C-accelerated _pickle module (rather than the slow pure Python implementation): https://bugs.python.org/issue35900 https://bugs.python.org/issue35911 Another is Davin's work on shared memory managers. There are also emerging standards like Apache Arrow that provide a shared, runtime-agnostic, compute-friendly representation for in-memory tabular data, and third-party frameworks like Dask which are potentially able to work on top of that and expose nice end-user APIs. For maximum synergy between these initiatives and the resulting APIs, it is better if things are done in the open ;-) Regards Antoine.
RetroSearch is an open source project built by @garambo | Open a GitHub Issue
Search and Browse the WWW like it's 1997 | Search results from DuckDuckGo
HTML:
3.2
| Encoding:
UTF-8
| Version:
0.7.4