Books about Parallel Computing#

This is a very incomplete list, permanently under construction, of books about parallel computing.


  • G. Hager and G. Wellein. Introduction to high performance computing for scientists and engineers. Chapman & Hall, 2010. This book first introduces the architecture of modern cache-based microprocessors and discusses their inherent performance limitations, before describing general optimization strategies for serial code on cache-based architectures. It next covers shared- and distributed-memory parallel computer architectures and the most relevant network topologies. After discussing parallel computing on a theoretical level, the authors show how to avoid or ameliorate typical performance problems connected with OpenMP. They then present cache-coherent nonuniform memory access (ccNUMA) optimization techniques, examine distributed-memory parallel programming with message passing interface (MPI), and explain how to write efficient MPI code. The final chapter focuses on hybrid programming with MPI and OpenMP.

  • V. Eijkhout. Introduction to high performance scientific computing. 2011. This is a textbook that teaches the bridging topics between numerical analysis, parallel computing, code performance, large scale applications. It can be freely downloaded from the author’s page on the book (though you have to respect the copyright of course).

  • A. Grama, A. Gupta, G. Kapyris, and V. Kumar. Introduction to parallel computing (2nd edition). Pearson Addison Wesley, 2003. ISBN 978-0-201-64865-2. A somewhat older book, but still used a lot as textbook in academic courses on parallel computing.

  • C. Lin and L. Snyder. Principles of parallel programming. Pearson Addison Wesley, 2008. ISBN 978-0-32148790-2. This books discusses parallel programming both from a more abstract level and a more practical level, touching briefly threads programming, OpenMP, MPI and PGAS-languages (using ZPL).

  • M. McCool, A.D. Robinson, and J. Reinders. Structured parallel programming: patterns for efficient computation. Morgan Kaufmann, 2012. ISBN 978-0-12-415993-8

Grid computing#



GPU computing#

  • M. Scarpino. OpenCL in action. Manning Publications Co., 2012. ISBN 978-1-617290-17-6

  • D.R. Kaeli, P. Mistry, D. Schaa, and D.P. Zhang. Heterogeneous computing with OpenCL 2.0, 1st Edition. Morgan Kaufmann, 2015. ISBN 978-0-12-801414-1 (print) or 978-0-12-801649-7 (eBook). A thourough rewrite of the earlier well-selling book for OpenCL 1.2 that saw 2 editions.

Xeon Phi computing#

Case studies and examples of programming paradigms#

Please mail further suggestions to