On Fri, May 16, 2003 at 03:09:31PM -0400, Barry Warsaw wrote:
> Skip, you're going about this all wrong. We already have the technology
> to start Python up blazingly fast. All you have to do <wink> is port
> XEmacs's unexec code. Then you load up Python with all the modules you
> think you're going to need, unexec it, then the next time it starts up
> like lightening. Disk space is cheap!
I gave it a try, starting with 2.3b1 and using FSF Emacs 21.3's unexelf.c.
An unexec'd binary loads faster than 'python -S -c pass', and seems to
work properly with two exceptions and a few limitations.
The only change to Python is in main(): I use mallopt() to force all
allocations to go through brk() instead of through mmap(), because unexec
doesn't support mmap'd memory. I also used Modules/Setup.local to make
some normally-shared modules not shared (for the same reason).
dump.py loads the requested modules (-<module> forces the module to *not*
be found) and then calls unexec(), producing a new binary with the given
name.
$ time ./python -S -c pass # best 'real' of 5 runs
real 0m0.054s
user 0m0.040s
sys 0m0.010s
$ time ./python -c 'import cgi' # best 'real' of 5 runs
real 0m0.127s
user 0m0.110s
sys 0m0.010s
$ strace -e open ./python -c 'import cgi' 2>&1 | grep -v ENOENT | wc -l
88
$ ./python dump.py cgipython -_ssl cgi
$ time ./cgipython -c 'import cgi' # best 'real' of 5 runs
real 0m0.039s
user 0m0.020s
sys 0m0.020s
$ strace -e open ./cgipython -c 'import cgi' 2>&1 | grep -v ENOENT | wc -l
9
$ ./python dump.py dython
-rwxrwxr-x 1 jepler jepler 4983713 May 18 19:42 cgipython
-rwxrwxr-x 1 jepler jepler 3603737 May 18 19:39 python
-rwxrwxr-x 1 jepler jepler 4541345 May 18 19:55 dython
(a minimal unexec'd python is about 90k bigger than the regular Python
binary)
I'm running the test suite now .. it hangs in test_signal for some reason.
test_thread seems to hang too, which may be related. (but test_threading
completes?)
$ ./dython Lib/test/regrtest.py -x test_signal -x test_thread
[...]
225 tests OK.
26 tests skipped:
test_aepack test_al test_bsddb3 test_bz2 test_cd test_cl
test_curses test_email_codecs test_gl test_imgfile
test_linuxaudiodev test_macfs test_macostools test_nis
test_normalization test_ossaudiodev test_pep277 test_plistlib
test_scriptpackages test_socket_ssl test_socketserver
test_sunaudiodev test_timeout test_urllibnet test_winreg
test_winsound
1 skip unexpected on linux2:
test_bz2
Well, if it worked right it'd sure be interesting. OTOH, unexelf.c is
GPL'd and there's also the nightmare of different unex* for different
platforms.
Jeff
########################################################################
# dump.py
import unexec, sys
for m in sys.argv[2:]:
if m[0] == "-":
sys.modules[m[1:]] = None
continue
__import__(m)
for m in sys.modules.keys():
mod = sys.modules[m]
if mod is None:
continue # negatively cached entry
if not hasattr(mod, "__file__"):
continue # builtin module
if mod.__file__.endswith(".so"):
raise RuntimeError, "Cannot dump with shared module %s" % m
unexec.dump(sys.argv[1], sys.executable)
/**********************************************************************/
/* unexecmodule.c (needs unexec() eg from unexelf.c) */
#include <Python.h>
extern void unexec (char *new_name, char *old_name, unsigned data_start, unsigned bss_start, unsigned entry_address);
static PyObject *dump_python(PyObject *self, PyObject *args) {
char *filename, *symfile;
if(!PyArg_ParseTuple(args, "ss", &filename, &symfile))
return NULL;
unexec(filename, symfile, 0, 0, (unsigned)Py_Main);
_exit(99);
}
static PyMethodDef dump_methods[] = {
{"dump", dump_python, METH_VARARGS,
PyDoc_STR("dump(filename, symfile) -> None")},
{NULL, NULL}
};
PyDoc_STRVAR(module_doc,
"Support for undumping the Python executable, a la Emacs");
PyMODINIT_FUNC
initunexec(void)
{
Py_InitModule3("unexec", dump_methods, module_doc);
}
########################################################################
# Setup.local
# Edit this file for local setup changes
unexec unexecmodule.c unexelf.c
time timemodule.c
_socket socketmodule.c
_random _randommodule.c
math mathmodule.c
fcntl fcntlmodule.c
RetroSearch is an open source project built by @garambo | Open a GitHub Issue
Search and Browse the WWW like it's 1997 | Search results from DuckDuckGo
HTML:
3.2
| Encoding:
UTF-8
| Version:
0.7.4