322 lines · plain
1=====================2Test-Suite Extensions3=====================4 5.. contents::6 :depth: 17 :local:8 9Abstract10========11 12These are ideas for additional programs, benchmarks, applications and13algorithms that could be added to the LLVM Test-Suite.14The test-suite could be much larger than it is now, which would help us15detecting compiler errors (crashes, miscompiles) during development.16 17Most probably, the reason why the programs below have not been added to18the test-suite yet is that nobody has found time to do it. But there19might be other issues as well, such as20 21 * Licensing (Support can still be added as external module,22 like for the SPEC benchmarks)23 24 * Language (in particular, there is no official LLVM frontend25 for FORTRAN yet)26 27 * Parallelism (currently, all programs in test-suite use28 one thread only)29 30Benchmarks31==========32 33SPEC CPU 201734-------------35https://www.spec.org/cpu2017/36 37The following have not been included yet because they contain Fortran38code.39 40In case of cactuBSSN only a small portion is Fortran. The hosts's41Fortran compiler could be used for these parts.42 43Note that CMake's Ninja generator has difficulties with Fortran. See the44`CMake documentation <https://cmake.org/cmake/help/v3.13/generator/Ninja.html#fortran-support>`_45for details.46 47 * 503.bwaves_r/603.bwaves_s48 * 507.cactuBSSN_r49 * 521.wrf_r/621.wrf_s50 * 527.cam4_r/627.cam4_s51 * 628.pop2_s52 * 548.exchange2_r/648.exchange2_s53 * 549.fotonik3d_r/649.fotonik3d_s54 * 554.roms_r/654.roms_s55 56SPEC OMP201257------------58https://www.spec.org/omp2012/59 60 * 350.md61 * 351.bwaves62 * 352.nab63 * 357.bt33164 * 358.botsalgn65 * 359.botsspar66 * 360.ilbdc67 * 362.fma3d68 * 363.swim69 * 367.imagick70 * 370.mgrid33171 * 371.applu33172 * 372.smithwa73 * 376.kdtree74 75OpenCV76------77https://opencv.org/78 79OpenMP 4.x SIMD Benchmarks80--------------------------81https://github.com/flwende/simd_benchmarks82 83PWM-benchmarking84----------------85https://github.com/tbepler/PWM-benchmarking86 87SLAMBench88---------89https://github.com/pamela-project/slambench90 91FireHose92--------93http://firehose.sandia.gov/94 95A Benchmark for the C/C++ Standard Library96------------------------------------------97https://github.com/hiraditya/std-benchmark98 99OpenBenchmarking.org CPU / Processor Suite100------------------------------------------101https://openbenchmarking.org/suite/pts/cpu102 103This is a subset of the104`Phoronix Test Suite <https://github.com/phoronix-test-suite/phoronix-test-suite/>`_105and is itself a collection of benchmark suites106 107Parboil Benchmarks108------------------109http://impact.crhc.illinois.edu/parboil/parboil.aspx110 111MachSuite112---------113https://breagen.github.io/MachSuite/114 115Rodinia116-------117http://lava.cs.virginia.edu/Rodinia/download_links.htm118 119Rodinia has already been partially included in120MultiSource/Benchmarks/Rodinia. Benchmarks still missing are:121 122 * streamcluster123 * particlefilter124 * nw125 * nn126 * myocyte127 * mummergpu128 * lud129 * leukocyte130 * lavaMD131 * kmeans132 * hotspot3D133 * heartwall134 * cfd135 * bfs136 * b+tree137 138vecmathlib tests harness139------------------------140https://bitbucket.org/eschnett/vecmathlib/wiki/Home141 142PARSEC143------144http://parsec.cs.princeton.edu/145 146Graph500 reference implementations147----------------------------------148https://github.com/graph500/graph500/tree/v2-spec149 150NAS Parallel Benchmarks151-----------------------152https://www.nas.nasa.gov/publications/npb.html153 154The official benchmark is written in Fortran, but an unofficial155C-translation is available as well:156https://github.com/benchmark-subsetting/NPB3.0-omp-C157 158DARPA HPCS SSCA#2 C/OpenMP reference implementation159---------------------------------------------------160http://www.highproductivity.org/SSCABmks.htm161 162This web site does not exist any more, but there seems to be a copy of163some of the benchmarks164https://github.com/gtcasl/hpc-benchmarks/tree/master/SSCA2v2.2165 166Kokkos167------168https://github.com/kokkos/kokkos-kernels/tree/master/perf_test169https://github.com/kokkos/kokkos/tree/master/benchmarks170 171PolyMage172--------173https://github.com/bondhugula/polymage-benchmarks174 175PolyBench176---------177https://sourceforge.net/projects/polybench/178 179A modified version of Polybench 3.2 is already presented in180SingleSource/Benchmarks/Polybench. A newer version 4.2.1 is available.181 182High Performance Geometric Multigrid183------------------------------------184https://crd.lbl.gov/departments/computer-science/PAR/research/hpgmg/185 186RAJA Performance Suite187----------------------188https://github.com/LLNL/RAJAPerf189 190CORAL-2 Benchmarks191------------------192https://asc.llnl.gov/coral-2-benchmarks/193 194Many of its programs have already been integrated in195MultiSource/Benchmarks/DOE-ProxyApps-C and196MultiSource/Benchmarks/DOE-ProxyApps-C++.197 198 * Nekbone199 * QMCPack200 * LAMMPS201 * Kripke202 * Quicksilver203 * PENNANT204 * Big Data Analytic Suite205 * Deep Learning Suite206 * Stream207 * Stride208 * ML/DL micro-benchmark209 * Pynamic210 * ACME211 * VPIC212 * Laghos213 * Parallel Integer Sort214 * Havoq215 216NWChem217------218http://www.nwchem-sw.org/index.php/Benchmarks219 220TVM221----222https://github.com/dmlc/tvm/tree/main/apps/benchmark223 224HydroBench225----------226https://github.com/HydroBench/Hydro227 228ParRes229------230https://github.com/ParRes/Kernels/tree/default/Cxx11231 232Applications/Libraries233======================234 235GnuPG236-----237https://gnupg.org/238 239Blitz++240-------241https://sourceforge.net/projects/blitz/242 243FFmpeg244------245https://ffmpeg.org/246 247FreePOOMA248---------249http://www.nongnu.org/freepooma/250 251FTensors252--------253http://www.wlandry.net/Projects/FTensor254 255rawspeed256--------257https://github.com/darktable-org/rawspeed258 259Its test dataset is 756 MB in size, which is too large to be included260into the test-suite repository.261 262C++ Performance Benchmarks263--------------------------264https://gitlab.com/chriscox/CppPerformanceBenchmarks265 266Generic Algorithms267==================268 269Image processing270----------------271 272Resampling273``````````274 275 * Bilinear276 * Bicubic277 * Lanczos278 279Dither280``````281 282 * Threshold283 * Random284 * Halftone285 * Bayer286 * Floyd-Steinberg287 * Jarvis288 * Stucki289 * Burkes290 * Sierra291 * Atkinson292 * Gradient-based293 294Feature detection295`````````````````296 297 * Harris298 * Histogram of Oriented Gradients299 300Color conversion301````````````````302 303 * RGB to grayscale304 * HSL to RGB305 306Graph307-----308 309Search Algorithms310`````````````````311 312 * Breadth-First-Search313 * Depth-First-Search314 * Dijkstra's algorithm315 * A-Star316 317Spanning Tree318`````````````319 320 * Kruskal's algorithm321 * Prim's algorithm322