A RetroSearch Logo

Home - News ( United States | United Kingdom | Italy | Germany ) - Football scores

Search Query:

Showing content from https://mail.python.org/pipermail/python-dev/2011-September/113533.html below:

[Python-Dev] Multigigabyte memory usage in the OpenIndiana Buildbot

[Python-Dev] Multigigabyte memory usage in the OpenIndiana BuildbotAntoine Pitrou solipsis at pitrou.net
Wed Sep 7 14:32:59 CEST 2011
On Wed, 07 Sep 2011 13:38:23 +0200
Jesus Cea <jcea at jcea.es> wrote:
> 
> So, the problem is that a) "make test" takes quite a bit of RAM and b)
> the buildbot forks some "big" processes, so the virtual memory needed
> is BIG.

Note that buildbots run "make buildbottest", not "make test".

> So I have two questions:
> 
> 1. Can we reduce the memory footprint of the tests?. I can't
> understand why the python test process is taking so much memory.

Because the test suite will by construction load all the stdlib (minus
the few modules which don't have a test suite), and creates numerous
test scenarios. Depending on the memory allocator, fragmentation can
make it difficult to reclaim memory that has been formally freed after
a test is run.

If "-j" is used, tests get run in a separate process each, so that
approach might be an answer.

> 2. Why buildbot is "forking()" big processes?. Can we do something to
> change this?.

Because we need to test for various functionalities, such as os.fork()
and os.exec*(), but also the command-line behaviour of the interpreter,
the distutils module, the packaging module, the subprocess module, the
multiprocessing module... (this list is not exhaustive).

Regards

Antoine.


More information about the Python-Dev mailing list

RetroSearch is an open source project built by @garambo | Open a GitHub Issue

Search and Browse the WWW like it's 1997 | Search results from DuckDuckGo

HTML: 3.2 | Encoding: UTF-8 | Version: 0.7.4