The clFFT test suite is based on the googletest framework. The test sources are located in ./src/tests/library. When the test code is built, there is a single test executable that is created called Test. This is the program to run in order to verify the functional correctness of the library.
Setting up the tests to runThe cmake generated build projects define an INSTALL target that can be built, which in addition to compiling the sources also goes through the extra work of creating a ./bin/clFFT/develop/vs10x64/package subdirectory and copying all built executables and libraries together into the same directory. This is convenient for performance measurement and testing, as typically the build tree has the executables and libraries scattered and built in their own build directories.
The test executable uses the FFTW library to validate answers. FFTW needs to be installed to successfully build and run the tests. Find below the help message given by the test program. The test program takes in various parameters to control the scope of test cases to be executed.
C:\clFFT\bin\package\bin64>Test.exe -h
clFFT client API version: 2.1.0
clFFT runtime version: 2.1.0
OpenCL platform [ 0 ]:
. . .
OpenCL devices [ 0 ]:
. . .
clFFT Runtime Test command line options:
-h [ --help ] produces this help message
-v [ --verbose ] print out detailed information for the
tests
--noVersion Don't print version information from
the clFFT library
--noInfoCL Don't print information from the OpenCL
runtime
-c [ --cpu ] Run tests on a CPU device
-g [ --gpu ] Run tests on a GPU device (default)
-p [ --pointwise ] Do a pointwise comparison to determine
test correctness (default: use root
mean square)
-t [ --tolerance ] arg (=0.00100000005)
tolerance level to use when determining
test pass/fail
-r [ --numRandom ] arg (=2000) number of random tests to run
--seed arg (=66780601) seed to use for the random test.
defaults to time(NULL)
-s [ --short ] Run radix 2 tests; no random testing
-m [ --medium ] Run all radices; no random testing
The exact time it takes for the test executables to finish is dependent on the hardware under test, and varies widely. Since the test framework is based on gtest, gtest filters can be applied to narrow testing to only a fraction of the overall tests. The gtest_filter flag takes a regular expression that it matches to the test name, and each test name is unique.
There are 2 groups of tests that the program runs. The first group is the individually named unit tests that tests exactly the same transform problem every time. The second group is the random tests that we will cover later. The first group of tests can be filtered using gtest filters. Example of running only tests that run power-of-2 transform lengths is shown below.
C:\clFFT\bin\package\bin64>Test.exe --gtest_filter=*pow2* clFFT client API version: 2.1.0 clFFT runtime version: 2.1.0 OpenCL platform [ 0 ]: . . . OpenCL devices [ 0 ]: . . . Result comparison tolerance is 0.001 Random test's seed is 66783095 Note: Google Test filter = *pow2* [==========] Running 436 tests from 2 test cases. [----------] Global test environment set-up. [----------] 218 tests from accuracy_test_pow2_single [ RUN ] accuracy_test_pow2_single.normal_1D_forward_in_place_complex_planar_to_complex_planar [ OK ] accuracy_test_pow2_single.normal_1D_forward_in_place_complex_planar_to_complex_planar (424 ms) [ RUN ] accuracy_test_pow2_single.normal_1D_backward_in_place_complex_planar_to_complex_planar [ OK ] accuracy_test_pow2_single.normal_1D_backward_in_place_complex_planar_to_complex_planar (356 ms) [ RUN ] accuracy_test_pow2_single.normal_1D_forward_in_place_complex_interleaved_to_complex_interleaved [ OK ] accuracy_test_pow2_single.normal_1D_forward_in_place_complex_interleaved_to_complex_interleaved (329 ms) [ RUN ] accuracy_test_pow2_single.len65536_1D_forward_in_place_complex_interleaved_to_complex_interleaved [ OK ] accuracy_test_pow2_single.len65536_1D_forward_in_place_complex_interleaved_to_complex_interleaved (327 ms) [ RUN ] accuracy_test_pow2_single.normal_1D_backward_in_place_complex_interleaved_to_complex_interleaved [ OK ] accuracy_test_pow2_single.normal_1D_backward_in_place_complex_interleaved_to_complex_interleaved (328 ms) [ RUN ] accuracy_test_pow2_single.normal_1D_forward_out_of_place_complex_planar_to_complex_planar [ OK ] accuracy_test_pow2_single.normal_1D_forward_out_of_place_complex_planar_to_complex_planar (362 ms) [ RUN ] accuracy_test_pow2_single.normal_1D_backward_out_of_place_complex_planar_to_complex_planar [ OK ] accuracy_test_pow2_single.normal_1D_backward_out_of_place_complex_planar_to_complex_planar (360 ms) [ RUN ] accuracy_test_pow2_single.normal_1D_forward_out_of_place_complex_interleaved_to_complex_interleaved [ OK ] accuracy_test_pow2_single.normal_1D_forward_out_of_place_complex_interleaved_to_complex_interleaved (334 ms) [ RUN ] accuracy_test_pow2_single.normal_1D_backward_out_of_place_complex_interleaved_to_complex_interleaved
Negative test filters are also available with the '-' operator to filter out tests. A filter that avoids running power-of-3 tests would look like --gtest_filter=-pow3 with similar output as above. A complicated expression can be created using both positive and negative filters, separated by the ':' character. This is just using standard googletest filter notation.
When the test program is run without any options, it runs standard unit tests available and also a set of random test cases. These random tests choose the parameters of FFT transform randomly and tests them. By default, the program runs 2000 random tests. But the number of random tests can be controlled. Also, using the verbose switch gives useful information on the kind of transform that is tested. Here is an example:
C:\clFFT\bin\package\bin64>Test.exe -r 5 -v --gtest_filte clFFT client API version: 2.1.0 clFFT runtime version: 2.1.0 OpenCL platform [ 0 ]: . . . OpenCL devices [ 0 ]: . . . Result comparison tolerance is 0.001 Random test's seed is 66783469 Note: Google Test filter = *random* [==========] Running 5 tests from 1 test case. [----------] Global test environment set-up. [----------] 5 tests from clAmdFft_RandomTest/accuracy_test_random [ RUN ] clAmdFft_RandomTest/accuracy_test_random.random_transform/0 transform parameters as seen by clAmdFft: 2 dimension(s): 108 750 batch: 5 double precision in-place hermitian interleaved -> real input stride(s): 1 166 output stride(s): 3 332 input distance: 124582 output distance: 249164 [ OK ] clAmdFft_RandomTest/accuracy_test_random.random_transform/0 (4520 ms) [ RUN ] clAmdFft_RandomTest/accuracy_test_random.random_transform/1 transform parameters as seen by clAmdFft: 3 dimension(s): 12 3750 2 batch: 1 double precision out-of-place complex interleaved -> complex interleaved input stride(s): 4 50 187500 output stride(s): 1 15 56250 input distance: 375011 output distance: 112560 [ OK ] clAmdFft_RandomTest/accuracy_test_random.random_transform/1 (2099 ms) [ RUN ] clAmdFft_RandomTest/accuracy_test_random.random_transform/2 transform parameters as seen by clAmdFft: 2 dimension(s): 1200 540 batch: 2 double precision out-of-place real -> hermitian planar input stride(s): 1 1200 output stride(s): 2 2401 input distance: 648000 output distance: 1296662 [ OK ] clAmdFft_RandomTest/accuracy_test_random.random_transform/2 (10036 ms) [ RUN ] clAmdFft_RandomTest/accuracy_test_random.random_transform/3 transform parameters as seen by clAmdFft: 3 dimension(s): 5400 18 2 batch: 1 single precision out-of-place complex interleaved -> complex interleaved input stride(s): 4 21601 388820 output stride(s): 1 5400 97200 input distance: 777757 output distance: 194400 [ OK ] clAmdFft_RandomTest/accuracy_test_random.random_transform/3 (3813 ms) [ RUN ] clAmdFft_RandomTest/accuracy_test_random.random_transform/4 transform parameters as seen by clAmdFft: 3 dimension(s): 15 2025 24 batch: 1 double precision in-place complex interleaved -> complex interleaved input stride(s): 1 18 36454 output stride(s): 1 18 36454 input distance: 874989 output distance: 874989 [ OK ] clAmdFft_RandomTest/accuracy_test_random.random_transform/4 (6868 ms) [----------] 5 tests from clAmdFft_RandomTest/accuracy_test_random (27342 ms total) [----------] Global test environment tear-down [==========] 5 tests from 1 test case ran. (27347 ms total) [ PASSED ] 5 tests.
RetroSearch is an open source project built by @garambo | Open a GitHub Issue
Search and Browse the WWW like it's 1997 | Search results from DuckDuckGo
HTML:
3.2
| Encoding:
UTF-8
| Version:
0.7.4