How to cite this WIREs title:
WIREs Comp Stat

Parallel computing with R: A brief review

Advanced Review

Dirk Eddelbuettel

Published Online: Jun 15 2020

DOI: 10.1002/wics.1515

Full article on Wiley Online Library: HTML PDF

Can't access this content? Tell your librarian.

Abstract Parallel computing has established itself as another standard method for applied research and data analysis. The R system, being internally constrained to mostly singly‐threaded operations, can nevertheless be used along with different parallel computing approaches. This brief review covers OpenMP and Intel TBB at the CPU‐ and compiler level, moves to process‐parallel approaches before discussing message‐passing parallelism and big data technologies for parallel processing such as Spark, Docker and Kubernetes before concluding with a focus on the future package integrating many of these approaches. This article is categorized under: Algorithms and Computational Methods > Methods for High Performance Computing Software for Computational Statistics > Software/Statistical Software Software for Computational Statistics > High Performance Software

References

Abadi,, M., Agarwal,, A., Barham,, P., Brevdo,, E., Chen,, Z., Citro,, C., …, Zheng,, X. (2015). TensorFlow: Large‐scale machine learning on heterogeneous systems. Software available from https://www.tensorflow.org/.
Allaire,, J., Francois,, R., Ushey,, K., Vandenbrouck,, G., Geelnard,, M., & Intel. (2019). RcppParallel: Parallel programming tools for ‘Rcpp’. R package version 4.4.4. Retrieved from https://CRAN.R-project.org/package=RcppParallel.
Balamuta,, J., & van den Berg,, T. (2019). sitmo: Parallel pseudo random number generator (PPRNG) ‘sitmo’ header files. R package version 2.0.1. Retrieved from https://CRAN.R-project.org/package=sitmo.
Bengtsson,, H. (2019a). Future: Unified parallel and distributed processing in R for everyone. R package version 1.15.1. Retrieved from https://CRAN.R-project.org/package=future.
Bengtsson,, H. (2019b). future.apply: Apply function to elements in parallel using futures. R package version 1.3.0. Retrieved from https://CRAN.R-project.org/package=future.apply.
Boettiger,, C., & Eddelbuettel,, D. (2017). An introduction to rocker: Docker Containers for R. The R Journal, 9(2), 527–536 Retrieved from https://journal.r-project.org/archive/2017/RJ-2017-065/index.html
Chen,, T., He,, T., Benesty,, M., Khotilovich,, V., Tang,, Y., Cho,, H., …, Li,, Y. (2019). xgboost: Extreme gradient boosting. R package version 0.90.0.2. Retrieved from https://CRAN.R-project.org/package=xgboost.
Clayden,, J., Modat,, M., Presles,, B., Anthopoulis,, T., & Daga,, P. (2019). RNiftyReg: Image registration using the ‘NiftyReg’ library. R package version 2.6.7. Retrieved from https://CRAN.R-project.org/package=RNiftyReg.
Dagum,, L., & Menon,, R. (1998). OpenMP: An industry‐standard API for shared‐memory programming. IEEE Computational Science and Engineering, 5(1), 46–55. https://doi.org/10.1109/99.660313
Determan,, C. Jr. (2019). RViennaCL: ‘ViennaCL’ C++ Header Files. R package version 1.7.1.8. Retrieved from https://CRAN.R-project.org/package=RViennaCL.
Dowle,, M., & Srinivasan,, A. (2019). data.table: Extension of ‘data.frame’. R package version 1.12.6. Retrieved from https://CRAN.R-project.org/package=data.table.
Eddelbuettel,, D., & Balamuta,, J. J. (2018). Extending R with C++: A Brief Introduction to Rcpp. The American Statistician, 72(1), 28–36. https://doi.org/10.1080/00031305.2017.1375990
Eddelbuettel,, D., Francois,, R., Allaire,, J., Ushey,, K., Kou,, Q., Russell,, N., Bates,, D., & Chambers,, J. (2019). Rcpp: Seamless R and C++ Integration. Retrieved from http://www.rcpp.org, http://dirk.eddelbuettel.com/code/rcpp.html, https://github.com/RcppCore/Rcpp.
Eddelbuettel,, D., Francois,, R., Bates,, D., & Ni,, B. (2019). RcppArmadillo: ‘Rcpp’ Integration for the ‘Armadillo’ Templated Linear Algebra Library. R package version 0.9.800.3.0. Retrieved from https://CRAN.R-project.org/package=RcppArmadillo.
Friedman,, D. P., & Wise,, D. S. (1978). Aspects of applicative programming for parallel processing. IEEE Transactions on Computers, C‐27(4), 289–296. https://doi.org/10.1109/tc.1978.1675100
Fukui,, K., & Stubner,, R. (2019). RcppArrayFire: Rcpp integration for ArrayFire. GitHub‐based R package version 0.1.0. Retrieved from https://github.com/daqana/rcpparrayfire.
Goodrich,, B., Gelman,, A., Carpenter,, B., Hoffman,, M., Lee,, D., Betancourt,, M., …, Gao,, Y. (2019). StanHeaders: C++ Header Files for Stan. R package version 2.19.0. Retrieved from https://CRAN.R-project.org/package=StanHeaders.
Hewitt,, C., Baker,, H. G. (1977). Laws for Communicating Parallel Processes. In IFIP Congress, pp. 987–992. Retrieved from https://dblp.uni-trier.de/db/conf/ifip/ifip1977.html#HewittB77.
Hibbard,, P. (1976). Parallel processing facilities. In S. A. Schuman, (Ed.), New directions in algorithmic languages. Rocquencourt, France: IRIA.
Knaus,, J. (2015). snowfall: Easier cluster computing (based on snow). R package version 1.84‐6.1. Retrieved from https://CRAN.R-project.org/package=snowfall.
Kurtzer,, G. M., Sochat,, V., & Bauer,, M. W. (2017). Singularity: Scientific containers for mobility of compute. PLoS One, 12(5), e0177459. https://doi.org/10.1371/journal.pone.0177459
Lang,, M., Bischl,, B., & Surmann,, D. (2018). batchtools: Tools for Computation on Batch Systems. R package version 0.9.11. Retrieved from https://CRAN.R-project.org/package=batchtools.
Leydold,, J. (2017). Rstream: Streams of random number. R package version 1.3.5. Retrieved from https://CRAN.R-project.org/package=rstream.
Luraschi,, J., Kuo,, K., Ushey,, K., Allaire,, J., & Macedo,, S., RStudio, The Apache Software Foundation (2019). sparklyr: R Interface to Apache Spark. R package version 1.0.5. Retrieved from https://CRAN.R-project.org/package=sparklyr.
Marchand,, P., Carroll,, I., Smorul,, M., Blake,, R., & Read,, Q. (2019). rslurm: Submit R Calculations to a ‘Slurm’ Cluster. R package version 0.5.0. Retrieved from https://CRAN.R-project.org/package=rslurm.
Merkel,, D. (2014). “Docker: Lightweight Linux containers for consistent development and deployment.” Linux Journal, 2014(239). Retrieved from http://dl.acm.org/citation.cfm?id=2600239.2600241.
Mersmann,, O. (2019). microbenchmark: Accurate Timing Functions. R package version 1.4‐7. Retrieved from https://CRAN.R-project.org/package=microbenchmark.
Message Passing Interface Forum (1994). MPI: A Message‐Passing Interface Standard. Technical report, Knoxville, TN.
Microsoft Corporation, & Weston,, S. (2019). doParallel: Foreach Parallel Adaptor for the ‘parallel’ Package. R package version 1.0.15. Retrieved from https://CRAN.R-project.org/package=doParallel.
Morandat,, F., Hill,, B., Osvald,, L., & Vitek,, J. (2012). Evaluating the Design of the R Language. In J. Noble, (Ed.), ECOOP 2012—Object‐oriented programming (pp. 104–131). Berlin, Heidelberg: Springer.
Nakano,, J., & Nakama,, E (2018). RhpcBLASctl: Control the Number of Threads on ‘BLAS’. R package version 0.18‐205. Retrieved from https://CRAN.R-project.org/package=RhpcBLASctl.
Ooi,, H., Microsoft Corporation, Weston,, S. (2019a). doMC: Foreach Parallel Adaptor for ‘parallel’. R package version 1.3.6. Retrieved from https://CRAN.R-project.org/package=doMC.
Ooi,, H., Microsoft Corporation, Weston,, S. (2019b). foreach: Provides Foreach Looping Construct. R package version 1.4.7. Retrieved from https://CRAN.R-project.org/package=foreach.
Paszke,, A., Gross,, S., Massa,, F., Lerer,, A., Bradbury,, J., Chanan,, G., … Chintala,, S. (2019). PyTorch: An imperative style, high‐performance deep learning library. In H. Wallach,, H. Larochelle,, A. Beygelzimer,, F. d`Alché‐Buc,, E. Fox,, & R. Garnett, (Eds.), Advances in neural information processing systems 32 (pp. 8024–8035). Red Hook, NY: Curran Associates, Inc Retrieved from http://papers.neurips.cc/paper/9015-pytorch-an-imperative-style-high-performance-deep-learning-library.pdf
R Core Team. (2019). R: A language and environment for statistical computing. Vienna, Austria: R Foundation for Statistical Computing Retrieved from https://www.R-project.org/
Roehl,, T., Treibig,, J., Hager,, G., & Wellein,, G. (2014). Overhead analysis of performance counter measurements. In 43rd International Conference on Parallel Processing Workshops (ICCPW), pp. 176–185. doi: https://doi.org/10.1109/ICPPW.2014.34.
Sartore,, L. (2019). spMC: Continuous‐Lag Spatial Markov Chains. R package version 0.3.10. Retrieved from https://CRAN.R-project.org/package=spMC.
Schmidberger,, M., Morgan,, M., Eddelbuettel,, D., Yu,, H., Tierney,, L., & Mansmann,, U. (2009). State of the art in parallel computing with R. Journal of Statistical Software, 31(1), 1–27. https://doi.org/10.18637/jss.v031.i01 Retrieved from https://www.jstatsoft.org/v031/i01
Sevcikova,, H., Rossini,, T., & L`Ecuyer,, P. (2019). rlecuyer: R Interface to RNG with Multiple Streams. R package version 0.3‐5. Retrieved from https://CRAN.R-project.org/package=rlecuyer.
Shvachko,, K., Kuang,, H., Radia,, S., & Chansler,, R. (2010). The Hadoop distributed file system. In Proceedings of the 2010 IEEE 26th Symposium on Mass Storage Systems and Technologies (MSST), MSST`10, pp. 1–10. IEEE Computer Society, Washington, DC. doi:https://doi.org/10.1109/MSST.2010.5496972
Stubner,, R. (2019). dqrng: Fast Pseudo Random Number Generators. R package version 0.2.1. Retrieved from https://CRAN.R-project.org/package=dqrng.
Tierney,, L, Rossini,, A. J., Li,, N., & Sevcikova,, H. (2018). snow: Simple Network of Workstations. R package version 0.4‐3. Retrieved from https://CRAN.R-project.org/package=snow.
Treibig,, J., Hager,, G., & Wellein,, G. (2010). LIKWID: A lightweight performance‐oriented tool suite for x86 multicore environments. In Proceedings of PSTI2010, the First International Workshop on Parallel Software Tools and Tool Infrastructures. San Diego, CA.
Venkataraman,, S., Meng,, X., & Cheung,, F., The Apache Software Foundation (2019). SparkR: R Front End for ‘Apache Spark’. R package version 2.4.4. Retrieved from https://CRAN.R-project.org/package=SparkR.
Weston,, S. (2017). doMPI: Foreach Parallel Adaptor for Rmpi Package. R package version 0.2.2. Retrieved from https://CRAN.R-project.org/package=doMPI.
Wickham,, H., François,, R., Henry,, L., & Müller,, K. (2019). dplyr: A Grammar of Data Manipulation. R package version 0.8.3. Retrieved from https://CRAN.R-project.org/package=dplyr.
Wikipedia Contributors (2019a). Kubernets. Retrieved from https://en.wikipedia.org/wiki/Kubernetes.
Wikipedia Contributors (2019b). LXC. Retrieved from https://en.wikipedia.org/wiki/LXC.
Wikipedia Contributors (2019c). SIMD. Retrieved from https://en.wikipedia.org/wiki/SIMD.
Yu,, H. (2018). Rmpi: Interface (Wrapper) to MPI (Message‐Passing Interface). R package version 0.6‐9. Retrieved from https://CRAN.R-project.org/package=Rmpi.
Zaharia,, M., Chowdhury,, M., Franklin,, M. J., Shenker,, S., & Stoica,, I. (2010). Spark: Cluster computing with working sets. In Proceedings of the 2nd USENIX Conference on Hot Topics in Cloud Computing, HotCloud`10, pp. 10–10. USENIX Association, Berkeley, CA. Retrieved from http://dl.acm.org/citation.cfm?id=1863103.1863113.
Zaharia,, M., Xin,, R. S., Wendell,, P., Das,, T., Armbrust,, M., Dave,, A., … Stoica,, I. (2016). Apache spark: A unified engine for big data processing. Communications of the ACM, 59(11), 56–65.

Access to this WIREs title is by subscription only.

Recommend to Your
Librarian Now!

Parallel computing with R: A brief review

Browse by Topic

Access to this WIREs title is by subscription only.

The latest WIREs articles in your inbox