Showing content from https://github.com/facebook/jest/issues/11188 below:
Document some of the tradeoffs of V8 coverage (vs Babel/Istanbul coverage) · Issue #11188 · jestjs/jest · GitHub
🐛 Bug Report
Jest supports two code coverage providers: the default babel provider based on Istanbul, and a newer experimental-but-encouraged v8 provider (#8596) based on V8's built-in coverage features.
When I tried switching to V8 coverage, I noticed a few subtle-but-important differences that don't seem to be explicitly documented. And when I investigated further, I found that some of those differences are not a result of V8 coverage being experimental, but are caused by deeper limitations and deliberate tradeoffs that might not change in the foreseeable future.
(V8 coverage still has plenty of merit, especially in terms of performance; I'd just like to help people understand the decision they're making when they use it.)
The main differences I want to point out are:
- V8 coverage is not a 1:1 replacement for Babel/Istanbul coverage
- Switching between the two will usually change the reported coverage statistics, which can change whether coverage thresholds are reached or not
- Switching between the two can cause regions of uncovered code to be discovered or ignored
- (This is mostly just an overall summary of the other points)
- V8 coverage works by taking a coverage report for output/transpiled code, and then using
v8-to-istanbul and source maps to convert that report into an Istanbul-compatible format
- This is an inherently imprecise and heuristic-driven process, though in many cases it works well enough for practical purposes
- In some cases this can give confusing or misleading results, or fail to distinguish between user code and generated code (e.g. uncovered branches introduced by
__esModule detection shims)
- Babel/Istanbul is able to be more precise because it usually operates directly on the user's original source code
- V8 coverage tracks “blocks”, not individual statements
- In particular, if you have a sequence of statements, and the middle statements always throw an exception, V8 coverage will mark the later statements as “covered” even though they never ran
- This is a deliberate tradeoff made by the V8 developers who implemented coverage
- Babel/Istanbul is able to be more precise because it explicitly instruments every source statement
- V8 coverage does not track the else branch of an if-statement without an explicit else
- So if a one-sided if-statement's condition is always true, V8 will not warn about an uncovered branch
- Babel/Istanbul is able to track these by artificially inserting an else with a branch counter
- V8 and Babel/Istanbul use different comment conventions for ignoring uncovered code
- V8 coverage uses
c8 ignore comments that are processed on a line-by-line basis without parsing
- Babel/Istanbul uses
istanbul ignore comments that are handled during AST processing
I'd like to find a way to update the coverageProvider documentation to mention some or all of these tradeoffs, hopefully in a way that isn't too intimidating or discouraging.
thymikee, lillallol, viceice, Stephanemw, lencioni and 3 more
RetroSearch is an open source project built by @garambo
| Open a GitHub Issue
Search and Browse the WWW like it's 1997 | Search results from DuckDuckGo
HTML:
3.2
| Encoding:
UTF-8
| Version:
0.7.4