HPC With R: The Basics Drew Schmidt November 12, 2016 Slides: - PowerPoint PPT Presentation

Profiling Profiling R Code Performance Profiling Tools: system.time() system.time() is a basic R utility for timing expressions x <- matrix(rnorm (20000*750) , nrow =20000 , ncol =750) 1 2 system.time(t(x) %*% x) 3 # user system elapsed 4 # 2.187 0.032 2.324 5 6 system.time(crossprod(x)) 7 # user system elapsed 8 # 1.009 0.003 1.019 9 10 system.time(cov(x)) 11 # user system elapsed 12 # 6.264 0.026 6.338 13 Slides: wrathematics.github.io/hpcdevcon2016/ Drew Schmidt HPC With R: The Basics 18/86

Profiling Profiling R Code Performance Profiling Tools: system.time() Put more complicated expressions inside of brackets: x <- matrix(rnorm (20000*750) , nrow =20000 , ncol =750) 1 2 system.time ({ 3 y <- x+1 4 z <- y*2 5 }) 6 # user system elapsed 7 # 0.057 0.032 0.089 8 Slides: wrathematics.github.io/hpcdevcon2016/ Drew Schmidt HPC With R: The Basics 19/86

Profiling Profiling R Code Performance Profiling Tools: Rprof() Rprof(filename="Rprof.out", append=FALSE , interval =0.02 , 1 memory.profiling=FALSE , gc.profiling=FALSE , 2 line.profiling=FALSE , numfiles =100L, bufsize =10000L) 3 Slides: wrathematics.github.io/hpcdevcon2016/ Drew Schmidt HPC With R: The Basics 20/86

Profiling Profiling R Code Slides: wrathematics.github.io/hpcdevcon2016/ Drew Schmidt HPC With R: The Basics 21/86

Profiling Profiling R Code Performance Profiling Tools: Rprof() x <- matrix(rnorm (10000*250) , nrow =10000 , ncol =250) 1 2 Rprof () 3 invisible(prcomp(x)) 4 Rprof(NULL) 5 6 summaryRprof () 7 8 Rprof(interval =.99) 9 invisible(prcomp(x)) 10 Rprof(NULL) 11 12 summaryRprof () 13 Slides: wrathematics.github.io/hpcdevcon2016/ Drew Schmidt HPC With R: The Basics 22/86

Profiling Profiling R Code Performance Profiling Tools: Rprof() $by.self 1 self.time self.pct total.time total.pct 2 "La.svd" 0.68 69.39 0.72 73.47 3 "%*%" 0.12 12.24 0.12 12.24 4 "aperm.default" 0.04 4.08 0.04 4.08 5 "array" 0.04 4.08 0.04 4.08 6 "matrix" 0.04 4.08 0.04 4.08 7 "sweep" 0.02 2.04 0.10 10.20 8 ### output truncated by presenter 9 10 11 $by.total total.time total.pct self.time self.pct 12 "prcomp" 0.98 100.00 0.00 0.00 13 "prcomp.default" 0.98 100.00 0.00 0.00 14 "svd" 0.76 77.55 0.00 0.00 15 "La.svd" 0.72 73.47 0.68 69.39 16 ### output truncated by presenter 17 18 $sample.interval 19 [1] 0.02 20 21 $sampling.time 22 [1] 0.98 23 Slides: wrathematics.github.io/hpcdevcon2016/ Drew Schmidt HPC With R: The Basics 23/86

Profiling Profiling R Code Performance Profiling Tools: Rprof() $by.self 1 [1] self.time self.pct total.time total.pct 2 <0 rows > (or 0-length row.names) 3 4 $by.total 5 [1] total.time total.pct self.time self.pct 6 <0 rows > (or 0-length row.names) 7 8 $sample.interval 9 [1] 0.99 10 11 $sampling.time 12 [1] 0 13 Slides: wrathematics.github.io/hpcdevcon2016/ Drew Schmidt HPC With R: The Basics 24/86

Profiling Advanced R Profiling 3 Profiling Why Profile? Profiling R Code Advanced R Profiling Slides: wrathematics.github.io/hpcdevcon2016/ Drew Schmidt HPC With R: The Basics

Profiling Advanced R Profiling Other Profiling Tools perf, PAPI fpmpi, mpiP, TAU pbdPROF pbdPAPI See forthcoming paper Analyzing Analytics: Advanced Performance Analysis Tools for R for more details. Slides: wrathematics.github.io/hpcdevcon2016/ Drew Schmidt HPC With R: The Basics 25/86

Benchmarking 1 Introduction 2 Debugging 3 Profiling 4 Benchmarking Slides: wrathematics.github.io/hpcdevcon2016/ Drew Schmidt HPC With R: The Basics

Benchmarking Benchmarking There’s a lot that goes on when executing an R funciton. Symbol lookup, creating the abstract syntax tree, creating promises for arguments, argument checking, creating environments, . . . Executing a second time can have dramatically different performance over the first execution. Benchmarking several methods fairly requires some care. Slides: wrathematics.github.io/hpcdevcon2016/ Drew Schmidt HPC With R: The Basics 26/86

Benchmarking Benchmarking tools: rbenchmark rbenchmark is a simple package that easily benchmarks different functions: x <- matrix(rnorm (10000*500) , nrow =10000 , ncol =500) 1 2 f <- function(x) t(x) %*% x 3 g <- function(x) crossprod(x) 4 5 library(rbenchmark ) 6 benchmark(f(x), g(x), columns=c("test", " replications ", "elapsed", "relative")) 7 8 # test replications elapsed relative 9 # 1 f(x) 100 13.679 3.588 10 # 2 g(x) 100 3.812 1.000 11 Slides: wrathematics.github.io/hpcdevcon2016/ Drew Schmidt HPC With R: The Basics 27/86

Benchmarking Benchmarking tools: microbenchmark microbenchmark is a separate package with a slightly different philosophy: x <- matrix(rnorm (10000*500) , nrow =10000 , ncol =500) 1 2 f <- function(x) t(x) %*% x 3 g <- function(x) crossprod(x) 4 5 library( microbenchmark ) 6 microbenchmark (f(x), g(x), unit="s") 7 8 # Unit: seconds 9 # expr min lq mean median uq max neval 10 # f(x) 0.11418617 0.11647517 0.12258556 0.11754302 0.12058145 0.17292507 100 11 # g(x) 0.03542552 0.03613772 0.03884497 0.03668231 0.03740173 0.07478309 100 12 Slides: wrathematics.github.io/hpcdevcon2016/ Drew Schmidt HPC With R: The Basics 28/86

Benchmarking Benchmarking tools: microbenchmark bench <- microbenchmark (f(x), g(x), unit="s") 1 boxplot(bench) 2 160 ● ● ● ● ● ● ● 140 ● ● ● ● 120 100 log(time) [t] 80 ● ● ● ● ● 60 ● ● ● ● 40 ● ● f(x) g(x) Expression Slides: wrathematics.github.io/hpcdevcon2016/ Drew Schmidt HPC With R: The Basics 29/86

Part II Improving R Performance Slides: wrathematics.github.io/hpcdevcon2016/ Drew Schmidt HPC With R: The Basics

Free Improvements 5 Free Improvements Packages The Bytecode Compiler Choice of BLAS Library 6 Writing Better R Code Slides: wrathematics.github.io/hpcdevcon2016/ Drew Schmidt HPC With R: The Basics

Free Improvements Packages 5 Free Improvements Packages The Bytecode Compiler Choice of BLAS Library Slides: wrathematics.github.io/hpcdevcon2016/ Drew Schmidt HPC With R: The Basics

Free Improvements Packages Packages Many high-quality “application” packages exist. Data manipulation: dplyr, data.table Modeling/math: Many! Try the CRAN taskviews. Parallelism: Discussed separately. Slides: wrathematics.github.io/hpcdevcon2016/ Drew Schmidt HPC With R: The Basics 30/86

Free Improvements The Bytecode Compiler 5 Free Improvements Packages The Bytecode Compiler Choice of BLAS Library Slides: wrathematics.github.io/hpcdevcon2016/ Drew Schmidt HPC With R: The Basics

Free Improvements The Bytecode Compiler The Compiler Package Released in 2011 (Tierney) Bytecode: sort of like machine code for interpreters. . . Improves R code speed by 2-5% generally. Does best on loops. Slides: wrathematics.github.io/hpcdevcon2016/ Drew Schmidt HPC With R: The Basics 31/86

Free Improvements The Bytecode Compiler Bytecode Compilation Non-core packages not (bytecode) compiled by default. “Base” and “recommended” (core) packages are. Downsides: (slightly) larger install size (much!) longer install process doesn’t fix bad code Upsides: slightly faster. Slides: wrathematics.github.io/hpcdevcon2016/ Drew Schmidt HPC With R: The Basics 32/86

Free Improvements The Bytecode Compiler Compiling a Function test <- function(x) x+1 1 test 2 # function(x) x+1 3 4 library(compiler) 5 6 test <- cmpfun(test) 7 test 8 # function(x) x+1 9 # <bytecode: 0x38c86c8 > 10 11 disassemble (test) 12 # list (.Code , list (7L, GETFUN.OP , 1L, MAKEPROM.OP , 2L, PUSHCONSTARG .OP , 13 # 3L, CALL.OP , 0L, RETURN.OP), list(x + 1, ‘+‘, list (.Code , 14 # list (7L, GETVAR.OP , 0L, RETURN.OP), list(x)), 1)) 15 Slides: wrathematics.github.io/hpcdevcon2016/ Drew Schmidt HPC With R: The Basics 33/86

Free Improvements The Bytecode Compiler Compiling Packages From R install.packages("my_package", type="source", INSTALL_opts="--byte -compile") 1 From The Shell export R_COMPILE_PKGS =1 1 R CMD INSTALL my_package.tar.gz 2 Or add the line: ByteCompile: yes to the package’s DESCRIPTION file. Slides: wrathematics.github.io/hpcdevcon2016/ Drew Schmidt HPC With R: The Basics 34/86

Free Improvements The Bytecode Compiler The Compiler: How much does it help really ? f <- function(n) for (i in 1:n) 2*(3+4) 1 2 3 library(compiler) 4 f_comp <- cmpfun(f) 5 6 7 library(rbenchmark ) 8 9 n <- 100000 10 benchmark(f(n), f_comp(n), columns=c("test", " replications ", "elapsed", 11 "relative"), order="relative") 12 # test replications elapsed relative 13 # 2 f_comp(n) 100 2.604 1.000 14 # 1 f(n) 100 2.845 1.093 15 Slides: wrathematics.github.io/hpcdevcon2016/ Drew Schmidt HPC With R: The Basics 35/86

Free Improvements The Bytecode Compiler The Compiler: How much does it help really ? g <- function(n){ 1 x <- matrix(runif(n*n), nrow=n, ncol=n) 2 min(colSums(x)) 3 } 4 5 library(compiler) 6 g_comp <- cmpfun(g) 7 8 9 library(rbenchmark ) 10 11 n <- 1000 12 benchmark(g(n), g_comp(n), columns=c("test", " replications ", "elapsed", 13 "relative"), order="relative") 14 # test replications elapsed relative 15 # 2 g_comp(n) 100 6.854 1.000 16 # 1 g(n) 100 6.860 1.001 17 Slides: wrathematics.github.io/hpcdevcon2016/ Drew Schmidt HPC With R: The Basics 36/86

Free Improvements Choice of BLAS Library 5 Free Improvements Packages The Bytecode Compiler Choice of BLAS Library Slides: wrathematics.github.io/hpcdevcon2016/ Drew Schmidt HPC With R: The Basics

Free Improvements Choice of BLAS Library The BLAS Basic Linear Algebra Subprograms. Basic numeric matrix operations. Used in linear algebra and many statistical operations. Different implementations available. Several multithreaded BLAS libraries exist. Slides: wrathematics.github.io/hpcdevcon2016/ Drew Schmidt HPC With R: The Basics 37/86

Free Improvements Choice of BLAS Library Reference Atlas OpenBLAS MKL 1000 500 100 50 4000 Log10 Average Wall Clock Time (5 Runs) 1000 500 100 50 8000 RRR D&C RRR D&C RRR D&C RRR D&C Comparing Symmetric Eigenvalue Performance http://bit.ly/2f49Sop Slides: wrathematics.github.io/hpcdevcon2016/ Drew Schmidt HPC With R: The Basics 38/86

Writing Better R Code 5 Free Improvements 6 Writing Better R Code Loops Ply Functions Vectorization Loops, Plys, and Vectorization Slides: wrathematics.github.io/hpcdevcon2016/ Drew Schmidt HPC With R: The Basics

Writing Better R Code Loops 6 Writing Better R Code Loops Ply Functions Vectorization Loops, Plys, and Vectorization Slides: wrathematics.github.io/hpcdevcon2016/ Drew Schmidt HPC With R: The Basics

Writing Better R Code Loops Loops for while No goto ’s or do while ’s. They’re really slow. Slides: wrathematics.github.io/hpcdevcon2016/ Drew Schmidt HPC With R: The Basics 39/86

Writing Better R Code Loops Loops: Best Practices Profile, profile, profile . Mostly try to avoid. Evaluate practicality of rewrite (plys, vectorization, compiled code) Always preallocate storage; don’t grow it dynamically. Slides: wrathematics.github.io/hpcdevcon2016/ Drew Schmidt HPC With R: The Basics 40/86

Writing Better R Code Ply Functions 6 Writing Better R Code Loops Ply Functions Vectorization Loops, Plys, and Vectorization Slides: wrathematics.github.io/hpcdevcon2016/ Drew Schmidt HPC With R: The Basics

Writing Better R Code Ply Functions “Ply” Functions R has functions that apply other functions to data. In a nutshell: loop sugar. Typical *ply’s: apply() : apply function over matrix “margin(s)”. lapply() : apply function over list/vector. mapply() : apply function over multiple lists/vectors. sapply() : same as lapply() , but (possibly) nicer output. Plus some other mostly irrelevant ones. Also Map() and Reduce() . Slides: wrathematics.github.io/hpcdevcon2016/ Drew Schmidt HPC With R: The Basics 41/86

Writing Better R Code Ply Functions Ply Examples: apply() x <- matrix (1:10 , 2) 1 2 x 3 # [,1] [,2] [,3] [,4] [,5] 4 # [1,] 1 3 5 7 9 5 # [2,] 2 4 6 8 10 6 7 apply(X=x, MARGIN =1, FUN=sum) 8 # [1] 25 30 9 10 apply(X=x, MARGIN =2, FUN=sum) 11 # [1] 3 7 11 15 19 12 13 apply(X=x, MARGIN =1:2 , FUN=sum) 14 # [,1] [,2] [,3] [,4] [,5] 15 # [1,] 1 3 5 7 9 16 # [2,] 2 4 6 8 10 17 Slides: wrathematics.github.io/hpcdevcon2016/ Drew Schmidt HPC With R: The Basics 42/86

Writing Better R Code Ply Functions Ply Examples: lapply() and sapply() lapply (1:4 , sqrt) 1 # [[1]] 2 # [1] 1 3 # 4 # [[2]] 5 # [1] 1.414214 6 # 7 # [[3]] 8 # [1] 1.732051 9 # 10 # [[4]] 11 # [1] 2 12 13 sapply (1:4 , sqrt) 14 # [1] 1.000000 1.414214 1.732051 2.000000 15 Slides: wrathematics.github.io/hpcdevcon2016/ Drew Schmidt HPC With R: The Basics 43/86

Writing Better R Code Ply Functions Transforming Loops Into Ply’s vec <- numeric(n) 1 for (i in 1:n){ 2 vec[i] <- my_function(i) 3 } 4 Becomes: sapply (1:n, my_function) 1 Slides: wrathematics.github.io/hpcdevcon2016/ Drew Schmidt HPC With R: The Basics 44/86

Writing Better R Code Ply Functions Ply’s: Best Practices Most ply’s are just shorthand/higher expressions of loops. Generally not much faster (if at all), especially with the compiler. Thinking in terms of lapply() can be useful however. . . Slides: wrathematics.github.io/hpcdevcon2016/ Drew Schmidt HPC With R: The Basics 45/86

Writing Better R Code Ply Functions Ply’s: Best Practices With ply’s and lambdas, can do some fiendishly crafty things. But don’t go crazy. . . cat(sapply(letters , function(a) sapply(letters , function(b) sapply(letters , 1 function(c) sapply(letters , function(d) paste(a, b, c, d, letters , "\n", sep="")))))) Slides: wrathematics.github.io/hpcdevcon2016/ Drew Schmidt HPC With R: The Basics 46/86

Writing Better R Code Vectorization 6 Writing Better R Code Loops Ply Functions Vectorization Loops, Plys, and Vectorization Slides: wrathematics.github.io/hpcdevcon2016/ Drew Schmidt HPC With R: The Basics

Writing Better R Code Vectorization Vectorization x+y x[, 1] <- 0 rnorm(1000) Slides: wrathematics.github.io/hpcdevcon2016/ Drew Schmidt HPC With R: The Basics 47/86

Writing Better R Code Vectorization Vectorization Same in R as in other high-level languages (Matlab, Python, . . . ). Idea: use pre-existing compiled kernels to avoid interpreter overhead. Much faster than loops and plys. ply <- function(x) lapply(rep(1, 1000) , rnorm) 1 vec <- function(x) rnorm (1000) 2 3 library(rbenchmark ) 4 benchmark(ply(x), vec(x)) 5 # test replications elapsed relative 6 # 1 ply(x) 100 0.348 38.667 7 # 2 vec(x) 100 0.009 1.000 8 Slides: wrathematics.github.io/hpcdevcon2016/ Drew Schmidt HPC With R: The Basics 48/86

Writing Better R Code Loops, Plys, and Vectorization 6 Writing Better R Code Loops Ply Functions Vectorization Loops, Plys, and Vectorization Slides: wrathematics.github.io/hpcdevcon2016/ Drew Schmidt HPC With R: The Basics

Writing Better R Code Loops, Plys, and Vectorization Putting It All Together Loops are slow. apply() , Reduce() are just for loops. Map() , lapply() , sapply() , mapply() (and most other core ones) are not for loops. Ply functions are not vectorized . Vectorization is fastest, but often needs lots of memory. Slides: wrathematics.github.io/hpcdevcon2016/ Drew Schmidt HPC With R: The Basics 49/86

Writing Better R Code Loops, Plys, and Vectorization Squares Let’s compute the square of the numbers 1–100000, using for loop without preallocation for loop with preallocation sapply() vectorization Slides: wrathematics.github.io/hpcdevcon2016/ Drew Schmidt HPC With R: The Basics 50/86

Writing Better R Code Loops, Plys, and Vectorization Squares square_sapply <- function(n) sapply (1:n, function(i) i^2) 1 2 square_vec <- function(n) (1:n)*(1:n) 3 library(rbenchmark ) 1 n <- 100000 2 3 benchmark(square_loop_noinit(n), square_loop_withinit(n), square_sapply(n), 4 square_vec(n)) # test replications elapsed relative 5 # 1 square_loop_noinit(n) 100 17.296 2470.857 6 # 2 square_loop_withinit(n) 100 0.933 133.286 7 # 3 square_sapply(n) 100 1.218 174.000 8 # 4 square_vec(n) 100 0.007 1.000 9 Slides: wrathematics.github.io/hpcdevcon2016/ Drew Schmidt HPC With R: The Basics 51/86

Part III Parallelism Slides: wrathematics.github.io/hpcdevcon2016/ Drew Schmidt HPC With R: The Basics

An Overview of Parallelism 7 An Overview of Parallelism 8 Shared Memory Parallelism in R 9 Distributed Memory Parallelism with R 10 Distributed Matrices Slides: wrathematics.github.io/hpcdevcon2016/ Drew Schmidt HPC With R: The Basics

An Overview of Parallelism Parallel Programming Packages for R Shared Memory Distributed Examples: pbdR , Rmpi , RHadoop , RHIPE Examples: parallel , snow , foreach , gputools , HiPLARM CRAN HPC Task View For more examples, see: http://cran.r-project.org/web/views/HighPerformanceComputing.html Slides: wrathematics.github.io/hpcdevcon2016/ Drew Schmidt HPC With R: The Basics 52/86

An Overview of Parallelism snow Rmpi pbdMPI LAPACK Focus on who owns what data and Sockets BLAS MPI what communication is needed Hadoop Distributed Memory Interconnection Network ScaLAPACK RHIPE PBLAS BLACS PROC PROC PROC PROC + cache + cache + cache + cache PETSc Same Task on Mem Mem Mem Mem Co-Processor Trilinos Blocks of data pbdDMAT pbdDMAT pbdDMAT GPU or magma MIC CUDA OpenCL CUBLAS .C Shared Memory OpenACC Local Memory .Call MAGMA Rcpp GPU: Graphical Processing Unit CORE CORE CORE CORE OpenMP OpenCL + cache + cache + cache + cache MIC: Many Integrated Core OpenACC inline MKL ACML Network Focus on which LibSci OpenMP tasks can be parallel Memory HiPLARM HiPLAR Threads DPLASMA fork multicore snow + multicore = parallel PLASMA (fork) Slides: wrathematics.github.io/hpcdevcon2016/ Drew Schmidt HPC With R: The Basics 53/86

An Overview of Parallelism Portability Many parallel R packages break on Windows Slides: wrathematics.github.io/hpcdevcon2016/ Drew Schmidt HPC With R: The Basics 54/86

An Overview of Parallelism RNG’s in Parallel Be careful! Aided by rlecuyer , rsprng , and doRNG packages. Slides: wrathematics.github.io/hpcdevcon2016/ Drew Schmidt HPC With R: The Basics 55/86

An Overview of Parallelism Parallel Programming: In Theory Slides: wrathematics.github.io/hpcdevcon2016/ Drew Schmidt HPC With R: The Basics 56/86

An Overview of Parallelism Parallel Programming: In Practice Slides: wrathematics.github.io/hpcdevcon2016/ Drew Schmidt HPC With R: The Basics 57/86

Shared Memory Parallelism in R 7 An Overview of Parallelism 8 Shared Memory Parallelism in R The parallel Package The foreach Package 9 Distributed Memory Parallelism with R 10 Distributed Matrices Slides: wrathematics.github.io/hpcdevcon2016/ Drew Schmidt HPC With R: The Basics

Shared Memory Parallelism in R The parallel Package 8 Shared Memory Parallelism in R The parallel Package The foreach Package Slides: wrathematics.github.io/hpcdevcon2016/ Drew Schmidt HPC With R: The Basics

Shared Memory Parallelism in R The parallel Package The parallel Package Comes with R ≥ 2.14.0 Has 2 disjoint interfaces. parallel = snow + multicore Slides: wrathematics.github.io/hpcdevcon2016/ Drew Schmidt HPC With R: The Basics 58/86

Shared Memory Parallelism in R The parallel Package The parallel Package: multicore Operates on fork/join paradigm. Slides: wrathematics.github.io/hpcdevcon2016/ Drew Schmidt HPC With R: The Basics 59/86

Shared Memory Parallelism in R The parallel Package The parallel Package: multicore + Data copied to child on write (handled by OS) + Very efficient. - No Windows support. - Not as efficient as threads. Slides: wrathematics.github.io/hpcdevcon2016/ Drew Schmidt HPC With R: The Basics 60/86

Shared Memory Parallelism in R The parallel Package The parallel Package: multicore mclapply(X, FUN , ..., 1 mc. preschedule =TRUE , mc.set.seed=TRUE , 2 mc.silent=FALSE , mc.cores=getOption("mc.cores", 2L), 3 mc.cleanup=TRUE , mc.allow.recursive=TRUE) 4 x <- lapply (1:10 , sqrt) 1 2 library(parallel) 3 x.mc <- mclapply (1:10 , sqrt) 4 5 all.equal(x.mc , x) 6 # [1] TRUE 7 Slides: wrathematics.github.io/hpcdevcon2016/ Drew Schmidt HPC With R: The Basics 61/86

Shared Memory Parallelism in R The parallel Package The parallel Package: multicore simplify2array (mclapply (1:10 , function(i) Sys.getpid (), mc.cores =4)) 1 # [1] 27452 27453 27454 27455 27452 27453 27454 27455 27452 27453 2 3 simplify2array (mclapply (1:2 , function(i) Sys.getpid (), mc.cores =4)) 4 # [1] 27457 2745 5 Slides: wrathematics.github.io/hpcdevcon2016/ Drew Schmidt HPC With R: The Basics 62/86

Shared Memory Parallelism in R The parallel Package The parallel Package: snow ? Uses sockets. + Works on all platforms. - More fiddley than mclapply() . - Not as efficient as forks. Slides: wrathematics.github.io/hpcdevcon2016/ Drew Schmidt HPC With R: The Basics 63/86

Shared Memory Parallelism in R The parallel Package The parallel Package: snow ### Set up the worker processes 1 cl <- makeCluster ( detectCores ()) 2 cl 3 # socket cluster with 4 nodes on host l o c a l h o s t 4 5 parSapply(cl , 1:5, sqrt) 6 7 stopCluster (cl) 8 Slides: wrathematics.github.io/hpcdevcon2016/ Drew Schmidt HPC With R: The Basics 64/86

Shared Memory Parallelism in R The parallel Package The parallel Package: Summary All detectCores() splitIndices() snow multicore makeCluster() mclapply() stopCluster() mcmapply() parLapply() mcparallel() parSapply() mccollect() and others. . . and others. . . Slides: wrathematics.github.io/hpcdevcon2016/ Drew Schmidt HPC With R: The Basics 65/86

Shared Memory Parallelism in R The foreach Package 8 Shared Memory Parallelism in R The parallel Package The foreach Package Slides: wrathematics.github.io/hpcdevcon2016/ Drew Schmidt HPC With R: The Basics

Shared Memory Parallelism in R The foreach Package The foreach Package On Cran (Revolution Analytics). Main package is foreach , which is a single interface for a number of “backend” packages. Backends: doMC , doMPI , doParallel , doRedis , doRNG , doSNOW . Slides: wrathematics.github.io/hpcdevcon2016/ Drew Schmidt HPC With R: The Basics 66/86

Shared Memory Parallelism in R The foreach Package The foreach Package: The Idea Unify the disparate interfaces. Slides: wrathematics.github.io/hpcdevcon2016/ Drew Schmidt HPC With R: The Basics 67/86

Shared Memory Parallelism in R The foreach Package The foreach Package + Works on all platforms (if backend does). + Can even work serial with minor notational change. + Write the code once, use whichever backend you prefer. - Really bizarre, non-R-ish synatx. - Efficiency issues if you aren’t careful! Slides: wrathematics.github.io/hpcdevcon2016/ Drew Schmidt HPC With R: The Basics 68/86

Shared Memory Parallelism in R The foreach Package Coin Flipping with 24 Cores ● ### Bad performance 1 2 ● foreach(i=1: len) %dopar% 2 Log Run Time in Seconds tinyfun(i) ● 3 ● Function ● ### Expected performance ● lapply 4 ● mclapply foreach(i=1: ncores) %dopar% { 5 0 ● foreach ● ● out <- numeric(len/ncores) 6 ● for (j in 1:( len/ncores)) 7 ● ● out[i] <- tinyfun(j) 8 ● ● ● ● out ● 9 } 10 −2 ● ● ● 10 100 1000 10000 1e+05 1e+06 Length of Iterating Set Slides: wrathematics.github.io/hpcdevcon2016/ Drew Schmidt HPC With R: The Basics 69/86

Shared Memory Parallelism in R The foreach Package The foreach Package: General Procedure Load foreach and your backend package. Register your backend. Call foreach Slides: wrathematics.github.io/hpcdevcon2016/ Drew Schmidt HPC With R: The Basics 70/86

Shared Memory Parallelism in R The foreach Package Using foreach: serial library(foreach) 1 2 ### Example 1 3 foreach(i=1:3) %do% sqrt(i) 4 5 ### Example 2 6 n <- 50 7 reps <- 100 8 9 x <- foreach(i=1: reps) %do% { 10 sum(rnorm(n, mean=i)) / (n*reps) 11 } 12 Slides: wrathematics.github.io/hpcdevcon2016/ Drew Schmidt HPC With R: The Basics 71/86