SLIDE 1
UNCvm Prologue Teaching Research Epilogue
UNCvm Prologue Teaching Research Epilogue
Archiving Computational Research in Virtual Machines Sorin Mitran - - PowerPoint PPT Presentation
Archiving Computational Research in Virtual Machines Sorin Mitran - - PowerPoint PPT Presentation
UNCvm Prologue Teaching Research Epilogue Archiving Computational Research in Virtual Machines Sorin Mitran Applied Mathematics University of North Carolina Chapel Hill Applied Mathematics Perspectives Reproducible Research Workshop
SLIDE 2
SLIDE 3
UNCvm Prologue Teaching Research Epilogue
UNCvm
1
Prologue 15 years ago in July ... Store the context UNCvm
2
Teaching
3
Research
4
Epilogue
SLIDE 4
UNCvm Prologue Teaching Research Epilogue 15 years ago in July ...
Implementing Godunov method
fweb Literate programming LaTeX documentation C, Fortran code Matlab mex facility
SLIDE 5
UNCvm Prologue Teaching Research Epilogue 15 years ago in July ...
Implementing Godunov method
fweb Literate programming LaTeX documentation C, Fortran code Matlab mex facility
SLIDE 6
UNCvm Prologue Teaching Research Epilogue 15 years ago in July ...
Implementing Godunov method
fweb Literate programming LaTeX documentation C, Fortran code Matlab mex facility
SLIDE 7
UNCvm Prologue Teaching Research Epilogue 15 years ago in July ...
Implementing Godunov method
fweb Literate programming LaTeX documentation C, Fortran code Matlab mex facility
SLIDE 8
UNCvm Prologue Teaching Research Epilogue 15 years ago in July ...
Aha! A makefile, I’m saved!
... or am I? Watcom 10.0 compiler - this might be difficult f2c, fweb, TeX - no problem Matlab 4.0 - now where’s Cleve’s phone number?
SLIDE 9
UNCvm Prologue Teaching Research Epilogue 15 years ago in July ...
Aha! A makefile, I’m saved!
... or am I? Watcom 10.0 compiler - this might be difficult f2c, fweb, TeX - no problem Matlab 4.0 - now where’s Cleve’s phone number?
SLIDE 10
UNCvm Prologue Teaching Research Epilogue 15 years ago in July ...
Aha! A makefile, I’m saved!
... or am I? Watcom 10.0 compiler - this might be difficult f2c, fweb, TeX - no problem Matlab 4.0 - now where’s Cleve’s phone number?
SLIDE 11
UNCvm Prologue Teaching Research Epilogue 15 years ago in July ...
Aha! A makefile, I’m saved!
... or am I? Watcom 10.0 compiler - this might be difficult f2c, fweb, TeX - no problem Matlab 4.0 - now where’s Cleve’s phone number?
SLIDE 12
UNCvm Prologue Teaching Research Epilogue 15 years ago in July ...
Some tools stick around, some don’t
Public domain tools are a bit more stable Proprietary tools are not Enough story-telling, get to work!
copy the directory make prog.tex fweave prog.web rm *.dvi; latex prog.tex ftangle prog.web; f77 prog.f vim prog.f; f77 prog.f; vim prog.web
SLIDE 13
UNCvm Prologue Teaching Research Epilogue 15 years ago in July ...
Some tools stick around, some don’t
Public domain tools are a bit more stable Proprietary tools are not Enough story-telling, get to work!
copy the directory make prog.tex fweave prog.web rm *.dvi; latex prog.tex ftangle prog.web; f77 prog.f vim prog.f; f77 prog.f; vim prog.web
SLIDE 14
UNCvm Prologue Teaching Research Epilogue 15 years ago in July ...
Some tools stick around, some don’t
Public domain tools are a bit more stable Proprietary tools are not Enough story-telling, get to work!
copy the directory make prog.tex fweave prog.web rm *.dvi; latex prog.tex ftangle prog.web; f77 prog.f vim prog.f; f77 prog.f; vim prog.web
SLIDE 15
UNCvm Prologue Teaching Research Epilogue Store the context
Lessons learned
Panta rei, Quingenti annus abhinc vos auditus mihi diversus p(x) = y0 + [y1, y0](x − x0) + [y2, y1, y0](x − x0)(x − x1) Science is a social endeavour ... ... hence a product of its time Historians can afford to decipher Colleagues shouldn’t
SLIDE 16
UNCvm Prologue Teaching Research Epilogue Store the context
Lessons learned
Panta rei, Quingenti annus abhinc vos auditus mihi diversus p(x) = y0 + [y1, y0](x − x0) + [y2, y1, y0](x − x0)(x − x1) Science is a social endeavour ... ... hence a product of its time Historians can afford to decipher Colleagues shouldn’t
SLIDE 17
UNCvm Prologue Teaching Research Epilogue Store the context
Lessons learned
Panta rei, Quingenti annus abhinc vos auditus mihi diversus p(x) = y0 + [y1, y0](x − x0) + [y2, y1, y0](x − x0)(x − x1) Science is a social endeavour ... ... hence a product of its time Historians can afford to decipher Colleagues shouldn’t
SLIDE 18
UNCvm Prologue Teaching Research Epilogue Store the context
Lessons learned
Panta rei, Quingenti annus abhinc vos auditus mihi diversus p(x) = y0 + [y1, y0](x − x0) + [y2, y1, y0](x − x0)(x − x1) Science is a social endeavour ... ... hence a product of its time Historians can afford to decipher Colleagues shouldn’t
SLIDE 19
UNCvm Prologue Teaching Research Epilogue Store the context
Lessons learned
Panta rei, Quingenti annus abhinc vos auditus mihi diversus p(x) = y0 + [y1, y0](x − x0) + [y2, y1, y0](x − x0)(x − x1) Science is a social endeavour ... ... hence a product of its time Historians can afford to decipher Colleagues shouldn’t
SLIDE 20
UNCvm Prologue Teaching Research Epilogue Store the context
Lessons learned
Panta rei, Quingenti annus abhinc vos auditus mihi diversus p(x) = y0 + [y1, y0](x − x0) + [y2, y1, y0](x − x0)(x − x1) Science is a social endeavour ... ... hence a product of its time Historians can afford to decipher Colleagues shouldn’t
SLIDE 21
UNCvm Prologue Teaching Research Epilogue Store the context
Live without side effects?
Alonzo Church lambda calculus
no side effects all expressions are calculable
Imperative languages have side effects If we use imperative coding
store the context store the state
SLIDE 22
UNCvm Prologue Teaching Research Epilogue Store the context
Live without side effects?
Alonzo Church lambda calculus
no side effects all expressions are calculable
Imperative languages have side effects If we use imperative coding
store the context store the state
SLIDE 23
UNCvm Prologue Teaching Research Epilogue Store the context
Live without side effects?
Alonzo Church lambda calculus
no side effects all expressions are calculable
Imperative languages have side effects If we use imperative coding
store the context store the state
SLIDE 24
UNCvm Prologue Teaching Research Epilogue Store the context
Virtual machines
What should be stored?
Compiler, library versions Tool versions OS idiosyncracies this might be a long list ...
What should be restored, and how?
all the above To install export CLAW=/home/claw What’s export?
SLIDE 25
UNCvm Prologue Teaching Research Epilogue Store the context
Virtual machines
What should be stored?
Compiler, library versions Tool versions OS idiosyncracies this might be a long list ...
What should be restored, and how?
all the above To install export CLAW=/home/claw What’s export?
SLIDE 26
UNCvm Prologue Teaching Research Epilogue Store the context
Virtual machine pros and cons
On the plus side
Hardware support (virtualization bit) Saves entire context
Turbo Pascal 5.5 (1989) Shock code from 1990 still runs (with graphics!)
On the loss side
Large amount of data to store - the entire OS What guarantees that the VM emulator will still run in 20 years? What about hardware essential to code execution (multiple CPUs for MPI, GPUs)
And yet ...
Storage is cheap and always getting cheaper Current strong vendor support for VM emulators MPI, OpenMP, GPU CUDA are all supported
SLIDE 27
UNCvm Prologue Teaching Research Epilogue Store the context
Virtual machine pros and cons
On the plus side
Hardware support (virtualization bit) Saves entire context
Turbo Pascal 5.5 (1989) Shock code from 1990 still runs (with graphics!)
On the loss side
Large amount of data to store - the entire OS What guarantees that the VM emulator will still run in 20 years? What about hardware essential to code execution (multiple CPUs for MPI, GPUs)
And yet ...
Storage is cheap and always getting cheaper Current strong vendor support for VM emulators MPI, OpenMP, GPU CUDA are all supported
SLIDE 28
UNCvm Prologue Teaching Research Epilogue Store the context
Virtual machine pros and cons
On the plus side
Hardware support (virtualization bit) Saves entire context
Turbo Pascal 5.5 (1989) Shock code from 1990 still runs (with graphics!)
On the loss side
Large amount of data to store - the entire OS What guarantees that the VM emulator will still run in 20 years? What about hardware essential to code execution (multiple CPUs for MPI, GPUs)
And yet ...
Storage is cheap and always getting cheaper Current strong vendor support for VM emulators MPI, OpenMP, GPU CUDA are all supported
SLIDE 29
UNCvm Prologue Teaching Research Epilogue UNCvm
Virtual machines for all
Whatever your workflow
store it document it
Use version management
store states store forks
Distribute your code
with preset environment point and click examples
SLIDE 30
UNCvm Prologue Teaching Research Epilogue UNCvm
Virtual machines for all
Whatever your workflow
store it document it
Use version management
store states store forks
Distribute your code
with preset environment point and click examples
SLIDE 31
UNCvm Prologue Teaching Research Epilogue UNCvm
Virtual machines for all
Whatever your workflow
store it document it
Use version management
store states store forks
Distribute your code
with preset environment point and click examples
SLIDE 32
UNCvm Prologue Teaching Research Epilogue UNCvm
Peculiarities of an academic environment
Incoming students are required to have a laptop
SLIDE 33
UNCvm Prologue Teaching Research Epilogue UNCvm
Peculiarities of an academic environment
As for research scientists ...
SLIDE 34
UNCvm Prologue Teaching Research Epilogue UNCvm
How to guide the transition?
Advocate for more research-friendly platforms
OS X (Carnegie-Mellon Mach kernel) Ubuntu (Debian) CentOS (Redhat)
1-click Linux install on Lenovo UNC-contract hardware Difficulties
The Linux environment still has to be configured Windows still needed, dual-boot is not officially supported
Put together a site-specific virtual machine (UNCvm)
SLIDE 35
UNCvm Prologue Teaching Research Epilogue UNCvm
How to guide the transition?
Advocate for more research-friendly platforms
OS X (Carnegie-Mellon Mach kernel) Ubuntu (Debian) CentOS (Redhat)
1-click Linux install on Lenovo UNC-contract hardware Difficulties
The Linux environment still has to be configured Windows still needed, dual-boot is not officially supported
Put together a site-specific virtual machine (UNCvm)
SLIDE 36
UNCvm Prologue Teaching Research Epilogue UNCvm
How to guide the transition?
Advocate for more research-friendly platforms
OS X (Carnegie-Mellon Mach kernel) Ubuntu (Debian) CentOS (Redhat)
1-click Linux install on Lenovo UNC-contract hardware Difficulties
The Linux environment still has to be configured Windows still needed, dual-boot is not officially supported
Put together a site-specific virtual machine (UNCvm)
SLIDE 37
UNCvm Prologue Teaching Research Epilogue UNCvm
How to guide the transition?
Advocate for more research-friendly platforms
OS X (Carnegie-Mellon Mach kernel) Ubuntu (Debian) CentOS (Redhat)
1-click Linux install on Lenovo UNC-contract hardware Difficulties
The Linux environment still has to be configured Windows still needed, dual-boot is not officially supported
Put together a site-specific virtual machine (UNCvm)
SLIDE 38
UNCvm Prologue Teaching Research Epilogue UNCvm
UNCvm
Lubuntu (Lightweight Ubuntu with LXDE) Contains free mathematical tools
Python, Octave, Gnuplot, Maxima TeXmacs
Supports via version forks
MPI, OpenMP (UNCvmMP) rCUDA GPU (UNCvmGPU)
Contains in-house code
BEARCLAW, Diapason UNCyml2tex
SLIDE 39
UNCvm Prologue Teaching Research Epilogue UNCvm
UNCvm
Lubuntu (Lightweight Ubuntu with LXDE) Contains free mathematical tools
Python, Octave, Gnuplot, Maxima TeXmacs
Supports via version forks
MPI, OpenMP (UNCvmMP) rCUDA GPU (UNCvmGPU)
Contains in-house code
BEARCLAW, Diapason UNCyml2tex
SLIDE 40
UNCvm Prologue Teaching Research Epilogue UNCvm
UNCvm
Lubuntu (Lightweight Ubuntu with LXDE) Contains free mathematical tools
Python, Octave, Gnuplot, Maxima TeXmacs
Supports via version forks
MPI, OpenMP (UNCvmMP) rCUDA GPU (UNCvmGPU)
Contains in-house code
BEARCLAW, Diapason UNCyml2tex
SLIDE 41
UNCvm Prologue Teaching Research Epilogue UNCvm
UNCvm
Lubuntu (Lightweight Ubuntu with LXDE) Contains free mathematical tools
Python, Octave, Gnuplot, Maxima TeXmacs
Supports via version forks
MPI, OpenMP (UNCvmMP) rCUDA GPU (UNCvmGPU)
Contains in-house code
BEARCLAW, Diapason UNCyml2tex
SLIDE 42
UNCvm Prologue Teaching Research Epilogue
UNCvm
1
Prologue
2
Teaching MATH566 - Undergraduate Introductory Numerical Analysis MATH761 - Graduate Numerical PDE
3
Research
4
Epilogue
SLIDE 43
UNCvm Prologue Teaching Research Epilogue MATH566 - Undergraduate Introductory Numerical Analysis
Challenges
Introduce scientific programming Provide friendly environment
SLIDE 44
UNCvm Prologue Teaching Research Epilogue MATH566 - Undergraduate Introductory Numerical Analysis
Challenges
Introduce scientific programming Provide friendly environment
SLIDE 45
UNCvm Prologue Teaching Research Epilogue MATH761 - Graduate Numerical PDE
Objectives
Instill good record-keeping Controlled numerical experiment Introduce advanced tools, techniques
SLIDE 46
UNCvm Prologue Teaching Research Epilogue MATH761 - Graduate Numerical PDE
Objectives
Instill good record-keeping Controlled numerical experiment Introduce advanced tools, techniques
SLIDE 47
UNCvm Prologue Teaching Research Epilogue MATH761 - Graduate Numerical PDE
Objectives
Instill good record-keeping Controlled numerical experiment Introduce advanced tools, techniques
SLIDE 48
UNCvm Prologue Teaching Research Epilogue
UNCvm
1
Prologue
2
Teaching
3
Research BEARCLAW
4
Epilogue
SLIDE 49
UNCvm Prologue Teaching Research Epilogue BEARCLAW
Software PDE solver package
AMR MPI multiphysics mixed-mode CPU/GPU
SLIDE 50
UNCvm Prologue Teaching Research Epilogue BEARCLAW
Patterned organic solar cells
Mimic nature
SLIDE 51
UNCvm Prologue Teaching Research Epilogue BEARCLAW
photonic Computational model
Solve Maxwell equations Time domain Dispersive media Auxiliary equation
SLIDE 52
UNCvm Prologue Teaching Research Epilogue
UNCvm
1
Prologue
2
Teaching
3
Research
4
Epilogue What to archive?
SLIDE 53
UNCvm Prologue Teaching Research Epilogue What to archive?
The whole thing
Reproduce and archive the entire environment Reproduce your workflow Comment and write as you code
SLIDE 54
UNCvm Prologue Teaching Research Epilogue What to archive?
The whole thing
Reproduce and archive the entire environment Reproduce your workflow Comment and write as you code
SLIDE 55
UNCvm Prologue Teaching Research Epilogue What to archive?