RetroSearch Browse

Thu Jun 1 04:23:04 EDT 2017 · https://mail.python.org/pipermail/python-dev/2017-June/148110.html

Hello.

AFAIK, allocating arena doesn't eat real (physical) memory.

* On Windows, VirtualAlloc is used for arena.  Real memory page is assigned
  when the page is used first time.
* On Linux and some other *nix, anonymous mmap is used.  Real page is
  assigned when first touch, like Windows.

Arena size is more important for **freeing** memory.
Python returns memory to system when:

1. When no block in pool is used, it returned to arena.
2. When no pool is used, return the arena to system.

So only one memory block can disturb returning the whole arena.

Some VMs (e.g. mono) uses special APIs to return "real page" from
allocated space.

* On Windows, VirtualFree() + VirtualAlloc() can be used to unassign pages.
* On Linux, madvice(..., MADV_DONTNEED) can be used.
* On other *nix, madvice(..., MADV_DONTNEED) + madvice(..., MADV_FREE)
can be used.

See also:

https://github.com/corngood/mono/blob/ef186403b5e95a5c95c38f1f19d0c8d061f2ac37/mono/utils/mono-mmap.c#L204-L208
(Windows)
https://github.com/corngood/mono/blob/ef186403b5e95a5c95c38f1f19d0c8d061f2ac37/mono/utils/mono-mmap.c#L410-L424
(Unix)

I think we can return not recently used free pools to system in same way.
So more large arena size + more memory efficient can be achieved.

But I need more experiment.

Regards,

Home - News ( United States | United Kingdom | Italy | Germany ) - Football scores

Showing content from https://mail.python.org/pipermail/python-dev/2017-June/148110.html below:

[Python-Dev] The untuned tunable parameter ARENA_SIZE