A RetroSearch Logo

Home - News ( United States | United Kingdom | Italy | Germany ) - Football scores

Search Query:

Showing content from https://mail.python.org/pipermail/python-dev/2008-December/084475.html below:

[Python-Dev] extremely slow exit for program having huge (45G) dict (python 2.5.2)

[Python-Dev] extremely slow exit for program having huge (45G) dict (python 2.5.2)Tim Peters tim.peters at gmail.com
Sat Dec 20 22:03:11 CET 2008
[Mike Coleman]
>> ... Regarding interning, I thought this only worked with strings.

Implementation details.  Recent versions of CPython also, e.g.,
"intern" the empty tuple, and very small integers.

>> Is there some way to intern integers?  I'm probably creating 300M
>> integers more or less uniformly distributed across range(10000)?

Interning would /vastly/ reduce memory use for ints in that case, from
gigabytes down to less than half a megabyte.


[Scott David Daniels]
> held = list(range(10000))
> ...
>    troublesome_dict[string] = held[number_to_hold]
> ...

More generally, but a bit slower, for objects usable as dict keys,
change code of the form:

    x = whatever_you_do_to_get_a_new_object()
    use(x)

to:

    x = whatever_you_do_to_get_a_new_object()
    x = intern_it(x, x)
    use(x)

where `intern_it` is defined like so once at the start of the program:

    intern_it = {}.setdefault

This snippet may make the mechanism clearer:

>>> intern_it = {}.setdefault
>>> x = 3000
>>> id(intern_it(x, x))
36166156
>>> x = 1000 + 2000
>>> id(intern_it(x, x))
36166156
>>> x = "works for computed strings too"
>>> id(intern_it(x, x))
27062696
>>> x = "works for computed strings t" + "o" * 2
>>> id(intern_it(x, x))
27062696
More information about the Python-Dev mailing list

RetroSearch is an open source project built by @garambo | Open a GitHub Issue

Search and Browse the WWW like it's 1997 | Search results from DuckDuckGo

HTML: 3.2 | Encoding: UTF-8 | Version: 0.7.4