HPC Challenge Awards Competition at SC14

The HPC Challenge committee is pleased to announce the results of the annual HPC Challenge Award Competition. The goal of the competition is to focus the HPC community's attention on developing a broad set of HPC hardware and HPC software capabilities that are necessary to productively use HPC systems.

The core of the HPC Challenge Award Competition is the HPC Challenge benchmark suite developed at the University of Tennessee under the DARPA HPCS program with contributions from a wide range of organizations from around the world (see http://icl.cs.utk.edu/hpcc/).

The Competition focuses on four of the most challenging benchmarks in the suite:

  1. Global HPL
  2. Global RandomAccess
  3. EP STREAM (Triad) per system
  4. Global FFT
There are two classes of awards.

Class 1: Best Performance (4 awards)

The figure of merit is the best performance on a base or optimized run submitted to the HPC Challenge website. The particular tests that are judged are: Global HPL, Global RandomAccess, EP STREAM (Triad) per system and Global FFT.

Class 2: Most Productivity (at least 1 award)

The most "elegant" implementation of at least four and at most five computationally intensive kernels. At least 3 tests of the Class 1 have to be included (choose from Global HPL, Global RandomAccess, EP STREAM Triad per system and Global FFT). At most 2 kernels may be selected that are not currently included in HPC Challenge (this allows more flexibility for the participants to show the strength of their system and its implementation). The kernels not present in HPCC need to conform to HPCC's rigor: they need to include testing, verification, and performance reporting components. Also, justification needs to be provided as to why the new kernels were chosen for submission: the stress is on relevance for computational science and the difficulty of parallel implementation and optimization. An example set of computational kernels is included in the following publications:

  1. The Landscape of Parallel Computing Research: A View from Berkeley (EECS-2006-183)
  2. The Parallel Computing Laboratory at U.C. Berkeley: A Research Agenda Based on the Berkeley View (EECS-2008-23)
  3. Communications of the ACM, October 2009 (Vol. 52, No. 10)

Submissions that include additional kernels are welcome as well.

This award is weighted 50% on performance and 50% on code elegance, clarity, and size. Both will be determined by an evaluation committee. For this award, the implementer must submit to hpcc-awards AT cs.utk.edu (by October 24th, 2014) a short description of:

  • the implementation,
  • the performance achieved,
  • lines-of-code, and
  • the actual source code of their implementation.
  • optionally: justification for choosing a computational kernels not present in HPCC

Evaluation and Scoring

The evaluation committee selects a set of finalists who are invited to give a short presentation at BOF session. The last one occured at SC14 under the title HPC Challenge Awards BOF at SC14 on Tuesday, November 18, 2014, Room: 273 at 12:15pm-1:15pm. The presentations are judged by the evaluation committee to select the winner. Multiple winners could be selected.

The Class 1 awards are decided based on benchmark results and should be a clear cut. Benchmark results are accepted up to the last day before the conference.

The Class 2 award is more subjective. The submission procedure works as follows:

  • "Early bird" entry to get feedback on the submission is by October 1st. We provide feedback by October 7th so the submission could be improved.
    • This is to help for compliance of the rules
    • Only one attempt at an early bird submission is allowed (the process is not iterative)
  • The final deadline is October 24th, for the Class 2 submission.
  • Awards Committee chooses finalists who make a presentation at the SC13 session and winners are chosen at the session.

For more information or questions on the HPCC Challenge Awards, please contact:
hpcc-awards AT cs.utk.edu.

Awards Committee:

  • David Bailey, LBNL NERSC
  • Jack Dongarra, (Co-Chair) Univeristy of Tennesse/ORNL
  • Jeremy Kepner, (Co-Chair) MIT Lincoln Lab
  • Bob Lucas, University of Southern California and ISI
  • Rusty Lusk, Argonne National Lab
  • Piotr Luszczek, Univeristy of Tennessee
  • John McCalpin, TACC
  • Rolf Rabenseifner, HLRS, Stuttgart
  • Daisuke Takahashi, Univeristy of Tsukuba
  • Jeff Vetter, ORNL

See also:

In 2014, the evaluation committee selected a set of finalists who were invited to give a short presentation during the HPC Challenge Award BOF at SC14. This presentation was judged by the evaluation committee to select the winner.


The following are the winners of the 2014 HPC Challenge Class 1 Awards:

HPL Achieved System Affiliation Submitter
1st place 9,796 Tflop/s K computer RIKEN AICS Mitsuo Yokokawa
1st runner up 5,709 Tflop/s MIRA, IBM BlueGene/Q Argonne Leadership Computing Facility, ANL Khaled Ibrahim

RandomAccess Achieved System Affiliation Submitter
1st place 2,021 GUPS IBM Power 775 IBM Development Engineering Ramakrishnan Rajamony
1st runner up 472 GUPS K computer RIKEN AICS Mitsuo Yokokawa
2nd runner up 418 GUPS MIRA, IBM BlueGene/Q Argonne Leadership Computing Facility, ANL Khaled Ibrahim

FFT Achieved System Affiliation Submitter
1st place 226 Tflop/s MIRA, IBM BlueGene/Q Argonne Leadership Computing Facility, ANL Khaled Ibrahim
1st runner up 206 Tflop/s K computer RIKEN AICS Mitsuo Yokokawa

STREAM Achieved System Affiliation Submitter
1st place 3857 TB/s K computer RIKEN AICS Mitsuo Yokokawa
1st runner up 1426 TB/s MIRA, IBM BlueGene/Q Argonne Leadership Computing Facility, ANL Khaled Ibrahim

The following is the winner of the 2014 HPC Challenge Class 2 Awards:

Award Presenter Affiliation Language PDF
Best Performance Masahiro Nakao University of Tsukuba, RIKEN AICS XcalableMP and XcalableACC slides
Most Elegant Piotr Bała ICM University of Warsaw PCJ slides


Latest HPC Challenge Award Competition News
2013-11-25
2012-06-12
2011-11-16
2010-11-18
2010-11-17
Nov 25 2014 Contact: Admin Login