On Fri, Apr 3, 2009 at 10:50 AM, Antoine Pitrou <solipsis at pitrou.net> wrote: > Collin Winter <collinw <at> gmail.com> writes: >> >> - I wish PyBench actually did more isolation. >> Call.py:ComplexPythonFunctionCalls is on my mind right now; I wish it >> didn't put keyword arguments and **kwargs in the same microbenchmark. > > Well, there is a balance to be found between having more subtests and keeping a > reasonable total running time :-) > (I have to plead guilty for ComplexPythonFunctionCalls, btw) Sure, there's definitely a balance to maintain. With perf.py, we're going down the road of having different tiers of benchmarks: the default set is the one we pay the most attention to, with other benchmarks available for benchmarking certain specific subsystems or workloads (like pickling list-heavy input data). Something similar could be done for PyBench, giving the user the option of increasing the level of detail (and run-time) as appropriate. >> - I would like to see PyBench incorporate better statistics for >> indicating the significance of the observed performance difference. > > I see you already have this kind of measurement in your perf.py script, would it > be easy to port it? Yes, it should be straightforward to incorporate these statistics into PyBench. In the same directory as perf.py, you'll find test_perf.py which includes tests for the stats functions we're using. Collin
RetroSearch is an open source project built by @garambo | Open a GitHub Issue
Search and Browse the WWW like it's 1997 | Search results from DuckDuckGo
HTML:
3.2
| Encoding:
UTF-8
| Version:
0.7.4