As it turns out, it's not memory related, but has to do with tokenize not supporting coding cookies in files. Mark picked up on this and linked it to an issue already in roundup that was raised way back in 2003: http://bugs.python.org/issue71988.
I've just finished patching test_tokenizer.py to better represent this test case -- the current implementation doesn't lend itself very well to being debugged when things go wrong (I think Mark and I both felt like we were on a bit of a wild goose chase). I've fixed that and have a bunch of text files with various utf-8/bom sig permutations that are now used to test tokenizer's compliance with PEP 0263. I'll upload that now then move on to actually patching tokenizer.py.
Trent "wishes-there-was-somewhere-to-get-some-food-after-11pm-at-pycon" Nelson.
________________________________________
From: python-dev-bounces+tnelson=onresolve.com at python.org [python-dev-bounces+tnelson=onresolve.com at python.org] On Behalf Of ocean [ocean at m2.ccsnet.ne.jp]
Sent: 17 March 2008 01:34
To: Neal Norwitz; Mark Dickinson
Cc: Python Dev
Subject: Re: [Python-Dev] 3.0 buildbots all red
> Yeah, sounds like a memory issue. Did you try running with valgrind
> or purify? I haven't done so for a long time, perhaps never on 3k
> branch. It would be a good thing to run a tool soon.
Maybe is this related?
[Potential overflows due to incorrect usage of PyUnicode_AsString]
http://bugs.python.org/issue1950
Thank you.
_______________________________________________
Python-Dev mailing list
Python-Dev at python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/tnelson%40onresolve.com
RetroSearch is an open source project built by @garambo | Open a GitHub Issue
Search and Browse the WWW like it's 1997 | Search results from DuckDuckGo
HTML:
3.2
| Encoding:
UTF-8
| Version:
0.7.4