[martin@v.loewis.de]
> That's what I mean (I'm *really* confused about memory family APIs,
> ever since everything changed :-)
Here's the in-depth course:
PyMem_xyz calls the platform malloc/realloc/free (fiddled for
x-platform uniformity in NULL and 0 handling)
PyObject_xyz calls pymalloc's malloc/realloc/free
and instead of a dozen layers of indirection we've now got crushingly
straightforward WYSIWYG preprocessor blocks like:
#ifdef WITH_PYMALLOC
#ifdef PYMALLOC_DEBUG
#define PyObject_MALLOC _PyObject_DebugMalloc
#define PyObject_Malloc _PyObject_DebugMalloc
#define PyObject_REALLOC _PyObject_DebugRealloc
#define PyObject_Realloc _PyObject_DebugRealloc
#define PyObject_FREE _PyObject_DebugFree
#define PyObject_Free _PyObject_DebugFree
#else /* WITH_PYMALLOC && ! PYMALLOC_DEBUG */
#define PyObject_MALLOC PyObject_Malloc
#define PyObject_REALLOC PyObject_Realloc
#define PyObject_FREE PyObject_Free
#endif
#else /* ! WITH_PYMALLOC */
#define PyObject_MALLOC PyMem_MALLOC
#define PyObject_REALLOC PyMem_REALLOC
#define PyObject_FREE PyMem_FREE
#endif /* WITH_PYMALLOC */
#define PyObject_Del PyObject_Free
#define PyObject_DEL PyObject_FREE
/* for source compatibility with 2.2 */
#define _PyObject_Del PyObject_Free
All the names you love are still there, it's just that most of them are
redundant now <wink>.
> ...
> I do think that the Unicode data should be managed by pymalloc as
> well.
Well, that largely depends on how big these suckers are. Calling
PyObject_XYZ adds real overhead if pymalloc can't handle the requested size:
all the overhead of the system routines, + the overhead of pymalloc figuring
out it can't handle it. I expect it's also not good to mix pymalloc with
custom free lists: you hold on to one object from a pymalloc pool, and it
prevents the entire pool from getting recycled for another size class. So
if you want to investigate using pymalloc more heavily for Unicode objects,
I suggest two things:
1. Get rid of the Unicode-specific free list.
2. Change the object layout to embed the str member storage, just as
PyStringObject does.
#1 is pretty localized, but #2 would require changing a lot of code.
> Of course, DecodeUTF8 would then raise the same issue: decoding
> UTF-8 doesn't know how many characters you'll get, either. This
> currently does not try to be clever, but allocates enough memory for
> the worst case.
I just put a patch up on SourceForge that's *less* clever, but shouldn't
waste any memory in the end. I expect you'll be happy with it, or rant
inconsolably. It's all the same to me <wink>.
RetroSearch is an open source project built by @garambo | Open a GitHub Issue
Search and Browse the WWW like it's 1997 | Search results from DuckDuckGo
HTML:
3.2
| Encoding:
UTF-8
| Version:
0.7.4