Integration tests need to be updates, they are failing right now: https://github.com/pmd/pmd-regression-tester/actions/workflows/manual-integration-tests.yml
The update of the test projects (see pmd/pmd#3640) is applied already, since this integration test just runs on pmd from master branch. We probably need to update our patch files, which simulate some changes.
In detail:
2021-11-29T10:12:00.4054476Z {:errors=>{:changed=>0, :new=>0, :removed=>1, :base_total=>40, :patch_total=>39}, :violations=>{:changed=>0, :new=>0, :removed=>512, :base_total=>512, :patch_total=>0}, :configerrors=>{:changed=>0, :new=>0, :removed=>3, :base_total=>3, :patch_total=>0}}
2021-11-29T10:12:00.4055486Z #############################
2021-11-29T10:12:00.4056013Z F
2021-11-29T10:12:00.4056599Z ===============================================================================
2021-11-29T10:12:00.4057716Z Failure: test_case_1_single_java_rule_changed(ManualIntegrationTests)
2021-11-29T10:12:00.4059145Z test/manual_integration_tests.rb:48:in `test_case_1_single_java_rule_changed'
2021-11-29T10:12:00.4060281Z 45: assert_equal(0, @summary[:violations][:changed], 'found changed violations')
2021-11-29T10:12:00.4061383Z 46: assert_equal(0, @summary[:violations][:new], 'found new violations')
2021-11-29T10:12:00.4063044Z 47: # These are the artificially created false-negatives for AbstractClassWithoutAbstractMethod rule
2021-11-29T10:12:00.4064646Z => 48: assert_equal(34 + 234, @summary[:violations][:removed], 'found removed violations')
2021-11-29T10:12:00.4065449Z 49:
2021-11-29T10:12:00.4066137Z 50: # errors might have been caused in the baseline for other rules (only visible in the stacktrace)
2021-11-29T10:12:00.4066976Z 51: # hence they might appear as removed
2021-11-29T10:12:00.4067543Z found removed violations
2021-11-29T10:12:00.4068034Z <268> expected but was
2021-11-29T10:12:00.4068433Z <512>
We have now many more removed violations (512 instead of 268). This could be just because of the changed projects - the baseline probably has now more violations that are affected...
2021-11-29T10:21:56.1686223Z �[0;0m#############################: test_case_2_single_xpath_rule_changed
2021-11-29T10:21:56.1687267Z {:errors=>{:changed=>0, :new=>0, :removed=>1, :base_total=>40, :patch_total=>39}, :violations=>{:changed=>0, :new=>15, :removed=>477, :base_total=>64130, :patch_total=>63668}, :configerrors=>{:changed=>0, :new=>0, :removed=>3, :base_total=>3, :patch_total=>0}}
2021-11-29T10:21:56.1688397Z #############################
2021-11-29T10:21:56.1688730Z F
2021-11-29T10:21:56.1689251Z ===============================================================================
2021-11-29T10:21:56.1689942Z Failure: test_case_2_single_xpath_rule_changed(ManualIntegrationTests)
2021-11-29T10:21:56.1691218Z test/manual_integration_tests.rb:81:in `test_case_2_single_xpath_rule_changed'
2021-11-29T10:21:56.1691974Z 78: print "#############################: test_case_2_single_xpath_rule_changed\n" \
2021-11-29T10:21:56.1692664Z 79: "#{@summary}\n#############################\n"
2021-11-29T10:21:56.1693470Z 80: assert_equal(0, @summary[:violations][:changed], 'found changed violations')
2021-11-29T10:21:56.1694413Z => 81: assert_equal(0, @summary[:violations][:new], 'found new violations')
2021-11-29T10:21:56.1695266Z 82: # There are no violations, that have been removed for AvoidMessageDigestField
2021-11-29T10:21:56.1696337Z 83: assert_equal(0, @summary[:violations][:removed], 'found removed violations')
2021-11-29T10:21:56.1696872Z 84:
2021-11-29T10:21:56.1697229Z found new violations
2021-11-29T10:21:56.1697645Z <0> expected but was
2021-11-29T10:21:56.1697997Z <15>
That's weird - by disabled one xpath rule we got new violations - that shouldn't happen.
2021-11-29T10:46:27.0962509Z �[0;0m#############################: test_case_3_change_in_core
2021-11-29T10:46:27.0964183Z {:errors=>{:changed=>0, :new=>0, :removed=>0, :base_total=>40, :patch_total=>40}, :violations=>{:changed=>0, :new=>36, :removed=>474, :base_total=>826504, :patch_total=>826066}, :configerrors=>{:changed=>0, :new=>0, :removed=>0, :base_total=>3, :patch_total=>3}}
2021-11-29T10:46:27.0965298Z #############################
2021-11-29T10:46:27.0965958Z F
2021-11-29T10:46:27.0966875Z ===============================================================================
2021-11-29T10:46:27.0967612Z Failure: test_case_3_change_in_core(ManualIntegrationTests)
2021-11-29T10:46:27.0969303Z test/manual_integration_tests.rb:114:in `test_case_3_change_in_core'
2021-11-29T10:46:27.0971638Z 111: print "#############################: test_case_3_change_in_core\n" \
2021-11-29T10:46:27.0972240Z 112: "#{@summary}\n#############################\n"
2021-11-29T10:46:27.0973910Z 113: assert_equal(0, @summary[:violations][:changed], 'found changed violations')
2021-11-29T10:46:27.0975061Z => 114: assert_equal(0, @summary[:violations][:new], 'found new violations')
2021-11-29T10:46:27.0977058Z 115: assert_equal(0, @summary[:violations][:removed], 'found removed violations')
2021-11-29T10:46:27.0978172Z 116: assert_equal(0, @summary[:errors][:removed], 'found removed errors')
2021-11-29T10:46:27.0979827Z 117: assert_equal(0, @summary[:errors][:changed], 'found changed errors')
2021-11-29T10:46:27.0980502Z found new violations
2021-11-29T10:46:27.0980987Z <0> expected but was
2021-11-29T10:46:27.0981916Z <36>
Here we got again new violations, while we expected no change at all.
That's fine, this test case still works.
RetroSearch is an open source project built by @garambo | Open a GitHub Issue
Search and Browse the WWW like it's 1997 | Search results from DuckDuckGo
HTML:
3.2
| Encoding:
UTF-8
| Version:
0.7.4