[Martin v. Loewis] > You should consider that malloc overhead is often 16 bytes per > object. Given that PyUnicodeObject is 24 bytes in 2.2, system malloc > will allocate 48 bytes per Unicode object on modern architectures. I > would think 100% overhead *is* a big argument. > > If you relate this to the actual data, it gets worse: A Unicode string > of length 1 would still require 32 bytes on an allocator that aligns > to 16. I think that's unusual -- 8-byte alignment is most common even on 64-bit boxes. KSR had to align to 128-byte boundaries, but there's a reason KSR died <wink -- alas, gross alignment requirements wasn't really it>. > Therefore, to store 2 bytes of real data, you need 80 bytes of > memory. > > I don't know how much overhead pymalloc adds, though; I believe it is > significantly less expensive. Yes, much less. On a 32-bit box, using the current #define's, and ignoring "arena" overhead(*), pymalloc uses 32 bytes per 4096 for bookkeeping. The remaining 4064 bytes can all be user data, but subject to 8-byte alignment, and to how many whole chunks of a given size can fit in 4064 bytes. For the PyUnicodeObject example, 8-byte alignment is without cost, and for the rest >>> divmod(4096 - 32, 24) (169, 8) >>> That is, pymalloc can get 169 PyUnicodeObjects out of a 4KB "page", with 32 bytes for bookkeeping, and 8 bytes left over (unused) -- total overhead is about 1%. (*) pymalloc gets "arenas" from the system malloc, where an arena is currently 256KB. Up to (worst case) 4KB of that is lost to align the start address to a 4KB boundary, and there's also the comparatively trivial (compared to 4KB!) overhead from the system malloc.
RetroSearch is an open source project built by @garambo | Open a GitHub Issue
Search and Browse the WWW like it's 1997 | Search results from DuckDuckGo
HTML:
3.2
| Encoding:
UTF-8
| Version:
0.7.4