Hunter is a flexible code tracing toolkit, not for measuring coverage, but for debugging, logging, inspection and other nefarious purposes. It has a simple Python API, a convenient terminal API and a CLI tool to attach to processes.
pip install hunter
https://python-hunter.readthedocs.io/
Basic use involves passing various filters to the trace option. An example:
import hunter
hunter.trace(module='posixpath', action=hunter.CallPrinter)
import os
os.path.join('a', 'b')
That would result in:
>>> os.path.join('a', 'b')
/usr/lib/python3.6/posixpath.py:75 call => join(a='a')
/usr/lib/python3.6/posixpath.py:80 line a = os.fspath(a)
/usr/lib/python3.6/posixpath.py:81 line sep = _get_sep(a)
/usr/lib/python3.6/posixpath.py:41 call => _get_sep(path='a')
/usr/lib/python3.6/posixpath.py:42 line if isinstance(path, bytes):
/usr/lib/python3.6/posixpath.py:45 line return '/'
/usr/lib/python3.6/posixpath.py:45 return <= _get_sep: '/'
/usr/lib/python3.6/posixpath.py:82 line path = a
/usr/lib/python3.6/posixpath.py:83 line try:
/usr/lib/python3.6/posixpath.py:84 line if not p:
/usr/lib/python3.6/posixpath.py:86 line for b in map(os.fspath, p):
/usr/lib/python3.6/posixpath.py:87 line if b.startswith(sep):
/usr/lib/python3.6/posixpath.py:89 line elif not path or path.endswith(sep):
/usr/lib/python3.6/posixpath.py:92 line path += sep + b
/usr/lib/python3.6/posixpath.py:86 line for b in map(os.fspath, p):
/usr/lib/python3.6/posixpath.py:96 line return path
/usr/lib/python3.6/posixpath.py:96 return <= join: 'a/b'
'a/b'
In a terminal it would look like:
Another useful scenario is to ignore all standard modules and force colors to make them stay even if the output is redirected to a file.
import hunter hunter.trace(stdlib=False, action=hunter.CallPrinter(force_colors=True))
Output format can be controlled with "actions". There's an alternative CodePrinter action that doesn't handle nesting (it was the default action until Hunter 2.0).
If filters match then action will be run. Example:
import hunter
hunter.trace(module='posixpath', action=hunter.CodePrinter)
import os
os.path.join('a', 'b')
That would result in:
>>> os.path.join('a', 'b')
/usr/lib/python3.6/posixpath.py:75 call def join(a, *p):
/usr/lib/python3.6/posixpath.py:80 line a = os.fspath(a)
/usr/lib/python3.6/posixpath.py:81 line sep = _get_sep(a)
/usr/lib/python3.6/posixpath.py:41 call def _get_sep(path):
/usr/lib/python3.6/posixpath.py:42 line if isinstance(path, bytes):
/usr/lib/python3.6/posixpath.py:45 line return '/'
/usr/lib/python3.6/posixpath.py:45 return return '/'
... return value: '/'
/usr/lib/python3.6/posixpath.py:82 line path = a
/usr/lib/python3.6/posixpath.py:83 line try:
/usr/lib/python3.6/posixpath.py:84 line if not p:
/usr/lib/python3.6/posixpath.py:86 line for b in map(os.fspath, p):
/usr/lib/python3.6/posixpath.py:87 line if b.startswith(sep):
/usr/lib/python3.6/posixpath.py:89 line elif not path or path.endswith(sep):
/usr/lib/python3.6/posixpath.py:92 line path += sep + b
/usr/lib/python3.6/posixpath.py:86 line for b in map(os.fspath, p):
/usr/lib/python3.6/posixpath.py:96 line return path
/usr/lib/python3.6/posixpath.py:96 return return path
... return value: 'a/b'
'a/b'
Another useful action is the VarsPrinter:
import hunter
# note that this kind of invocation will also use the default `CallPrinter` action
hunter.trace(hunter.Q(module='posixpath', action=hunter.VarsPrinter('path')))
import os
os.path.join('a', 'b')
That would result in:
>>> os.path.join('a', 'b')
/usr/lib/python3.6/posixpath.py:75 call => join(a='a')
/usr/lib/python3.6/posixpath.py:80 line a = os.fspath(a)
/usr/lib/python3.6/posixpath.py:81 line sep = _get_sep(a)
/usr/lib/python3.6/posixpath.py:41 call [path => 'a']
/usr/lib/python3.6/posixpath.py:41 call => _get_sep(path='a')
/usr/lib/python3.6/posixpath.py:42 line [path => 'a']
/usr/lib/python3.6/posixpath.py:42 line if isinstance(path, bytes):
/usr/lib/python3.6/posixpath.py:45 line [path => 'a']
/usr/lib/python3.6/posixpath.py:45 line return '/'
/usr/lib/python3.6/posixpath.py:45 return [path => 'a']
/usr/lib/python3.6/posixpath.py:45 return <= _get_sep: '/'
/usr/lib/python3.6/posixpath.py:82 line path = a
/usr/lib/python3.6/posixpath.py:83 line [path => 'a']
/usr/lib/python3.6/posixpath.py:83 line try:
/usr/lib/python3.6/posixpath.py:84 line [path => 'a']
/usr/lib/python3.6/posixpath.py:84 line if not p:
/usr/lib/python3.6/posixpath.py:86 line [path => 'a']
/usr/lib/python3.6/posixpath.py:86 line for b in map(os.fspath, p):
/usr/lib/python3.6/posixpath.py:87 line [path => 'a']
/usr/lib/python3.6/posixpath.py:87 line if b.startswith(sep):
/usr/lib/python3.6/posixpath.py:89 line [path => 'a']
/usr/lib/python3.6/posixpath.py:89 line elif not path or path.endswith(sep):
/usr/lib/python3.6/posixpath.py:92 line [path => 'a']
/usr/lib/python3.6/posixpath.py:92 line path += sep + b
/usr/lib/python3.6/posixpath.py:86 line [path => 'a/b']
/usr/lib/python3.6/posixpath.py:86 line for b in map(os.fspath, p):
/usr/lib/python3.6/posixpath.py:96 line [path => 'a/b']
/usr/lib/python3.6/posixpath.py:96 line return path
/usr/lib/python3.6/posixpath.py:96 return [path => 'a/b']
/usr/lib/python3.6/posixpath.py:96 return <= join: 'a/b'
'a/b'
In a terminal it would look like:
You can give it a tree-like configuration where you can optionally configure specific actions for parts of the tree (like dumping variables or a pdb set_trace):
from hunter import trace, Q, Debugger
from pdb import Pdb
trace(
# drop into a Pdb session if ``foo.bar()`` is called
Q(module="foo", function="bar", kind="call", action=Debugger(klass=Pdb))
| # or
Q(
# show code that contains "mumbo.jumbo" on the current line
lambda event: event.locals.get("mumbo") == "jumbo",
# and it's not in Python's stdlib
stdlib=False,
# and it contains "mumbo" on the current line
source__contains="mumbo"
)
)
import foo
foo.func()
With a foo.py like this:
def bar():
execution_will_get_stopped # cause we get a Pdb session here
def func():
mumbo = 1
mumbo = "jumbo"
print("not shown in trace")
print(mumbo)
mumbo = 2
print(mumbo) # not shown in trace
bar()
We get:
>>> foo.func()
not shown in trace
/home/ionel/osp/python-hunter/foo.py:8 line print(mumbo)
jumbo
/home/ionel/osp/python-hunter/foo.py:9 line mumbo = 2
2
/home/ionel/osp/python-hunter/foo.py:1 call def bar():
> /home/ionel/osp/python-hunter/foo.py(2)bar()
-> execution_will_get_stopped # cause we get a Pdb session here
(Pdb)
In a terminal it would look like:
In similar fashion to strace Hunter can trace other processes, eg:
hunter-trace --gdb -p 123
If you wanna play it safe (no messy GDB) then add this in your code:
from hunter import remote remote.install()
Then you can do:
hunter-trace -p 123
See docs on the remote feature.
Note: Windows ain't supported.
Environment variable activationFor your convenience environment variable activation is available. Just run your app like this:
PYTHONHUNTER="module='os.path'" python yourapp.py
On Windows you'd do something like:
set PYTHONHUNTER=module='os.path' python yourapp.py
The activation works with a clever .pth file that checks for that env var presence and before your app runs does something like this:
from hunter import * trace(<whatever-you-had-in-the-PYTHONHUNTER-env-var>)
Note that Hunter is activated even if the env var is empty, eg: PYTHONHUNTER="".
Sometimes you always use the same options (like stdlib=False or force_colors=True). To save typing you can set something like this in your environment:
PYTHONHUNTERCONFIG="stdlib=False,force_colors=True"
This is the same as PYTHONHUNTER="stdlib=False,action=CallPrinter(force_colors=True)".
Notes:
Setting PYTHONHUNTERCONFIG alone doesn't activate hunter.
All the options for the builtin actions are supported.
Although using predicates is supported it can be problematic. Example of setup that won't trace anything:
PYTHONHUNTERCONFIG="Q(module_startswith='django')" PYTHONHUNTER="Q(module_startswith='celery')"
which is the equivalent of:
PYTHONHUNTER="Q(module_startswith='django'),Q(module_startswith='celery')"
which is the equivalent of:
PYTHONHUNTER="Q(module_startswith='django')&Q(module_startswith='celery')"
Hunter supports a flexible query DSL, see the introduction.
To run the all tests run:
tox
Hunter doesn't do everything. As a design goal of this library some things are made intentionally austere and verbose (to avoid complexity, confusion and inconsistency). This has few consequences:
~Q(module='re').You should look at it like it's a tool to help you understand and debug big applications, or a framework ridding you of the boring parts of settrace, not something that helps you learn Python.
There's some obvious overlap with smiley but there are few fundamental differences:
Complexity. Smiley is simply over-engineered:
Why do you need all that just to debug some stuff in a terminal? Simply put, it's a nice idea but the design choices work against you when you're already neck-deep into debugging your own code. In my experience Smiley has been very buggy and unreliable. Your mileage may vary of course.
Tracing long running code. This will make Smiley record lots of data, making it unusable.
Now because Smiley records everything, you'd think it's better suited for short programs. But alas, if your program runs quickly then it's pointless to record the execution. You can just run it again.
It seems there's only one situation where it's reasonable to use Smiley: tracing io-bound apps remotely. Those apps don't execute lots of code, they just wait on network so Smiley's storage won't blow out of proportion and tracing overhead might be acceptable.
Use-cases. It seems to me Smiley's purpose is not really debugging code, but more of a "non interactive monitoring" tool.
In contrast, Hunter is very simple:
Few dependencies.
Low overhead (tracing/filtering code has an optional Cython extension).
No storage. This simplifies lots of things.
The only cost is that you might need to run the code multiple times to get the filtering/actions right. This means Hunter is not really suited for "post-mortem" debugging. If you can't reproduce the problem anymore then Hunter won't be of much help.
Pytrace is another tracer tool. It seems quite similar to Smiley - it uses a sqlite database for the events, threads and IPC, thus it's reasonable to expect the same kind of problems.
Why not PySnooper or snoop?snoop is a refined version of PySnooper. Both are more suited to tracing small programs or functions as the output is more verbose and less suited to the needs of tracing a big application where Hunter provides more flexible setup, filtering capabilities, speed and brevity.
For purposes of debugging coverage is a great tool but only as far as "debugging by looking at what code is (not) run". Checking branch coverage is good but it will only get you as far.
From the other perspective, you'd be wondering if you could use Hunter to measure coverage-like things. You could do it but for that purpose Hunter is very "rough": it has no builtin storage. You'd have to implement your own storage. You can do it but it wouldn't give you any advantage over making your own tracer if you don't need to "pre-filter" whatever you're recording.
In other words, filtering events is the main selling point of Hunter - it's fast (cython implementation) and the query API is flexible enough.
Noteworthy usages or Hunter (submit a PR with your project if you built a tool that relies on hunter):
More projects using it at https://github.com/ionelmc/python-hunter/network/dependents
RetroSearch is an open source project built by @garambo | Open a GitHub Issue
Search and Browse the WWW like it's 1997 | Search results from DuckDuckGo
HTML:
3.2
| Encoding:
UTF-8
| Version:
0.7.4