Amazon EC2, GPU computing, PyNX:Ptychography Vincent Favre-Nicolin - - PowerPoint PPT Presentation

▶

Sep 06, 2023 13 likes •77 views

Amazon EC2, GPU computing, PyNX:Ptychography Vincent Favre-Nicolin X-ray NanoProbe group, ESRF 1 l 66TH MEETING OF THE ESRF l 30-31 May 2014 l Author 26/07/2013 AMAZON GPU COMPUTING Gpu Accelerated computing instances: (old) G2: nVidia

SLIDE 1

26/07/2013 l 66TH MEETING OF THE ESRF l 30-31 May 2014 l Author 1

Amazon EC2, GPU computing, PyNX:Ptychography

Vincent Favre-Nicolin X-ray NanoProbe group, ESRF

SLIDE 2

AMAZON GPU COMPUTING

Gpu Accelerated computing instances:

(old) G2: nVidia GRID K520
P2: nVidia K80 (early 2015) (12GB, 4 Tflops theor.)
P2.xlarge: 1 K80, 61 GB memory
P2.8xlarge: 4 K80, 488 GB memory
P2.16xlarge: 8 K80, 732 GB memory
Performance ?
Usability for data analysis ?

26/07/2013 Page 2 l PANDAAS working group l 12 December 2016 l Vincent FAVRE-NICOLIN

SLIDE 3

PyNX: PTYCHOGRAPHY

On-going effort to provide tools for on/offline

analysis for Coherent Imaging techniques

Focused on using GPU/OpenCL for faster computing
Used at id01, id13@ESRF, running on dedicated GPU

machines (GPU: Titan X)

2D Ptychography:
Coherent images taken at different positions on a sample
100 to 1000 of images-moderatly fast data acquisition (1-100Hz)
Dataset can be exported in CXI format (http://cxidb.org/cxi.html)

26/07/2013 Page 3 l PANDAAS working group l 12 December 2016 l Vincent FAVRE-NICOLIN

SLIDE 4

IPYTHON NOTEBOOK: PyNX.PTYCHO

Quick test : launch ipython notebook
Machine:
debian 8 official
Nvidia drivers, OpenCL, clFFT
Scientific python packages + PyNX
Data (already transferred 87Mb)
Go to browser
Choose kernel for data analysis
Tweak parameters
Run analysis
Change parameters as needed and restart…

LIVE DEMO

26/07/2013 Page 4 l PANDAAS working group l 12 December 2016 l Vincent FAVRE-NICOLIN

ssh -CX -L:8888:localhost:8888 admin@ec2.****.compute.amazonaws.com

ipython3 notebook --pylab=inline

SLIDE 5

IPYTHON NOTEBOOK: PyNX.PTYCHO

GPU Analysis works (including using multi-GPU)
Some latency in initializing the GPUs ? (up to 20s)
No issues otherwise
Compared speed:

GPU K80 (Amazon) Titan X (ESRF) Read data (cxi) 58 Mpixel/s 86 Mpixel/s 2D FFT (400x400, 32 stack) 116 Gflop/s 282 Gflop/s dt/cycle (AP, 1025 frames) 0.544s 0.223s

Notebook can easily be configured to automatically

be available when starting the machine

26/07/2013 Page 5 l PANDAAS working group l 12 December 2016 l Vincent FAVRE-NICOLIN

SLIDE 6

AMAZON GPU COMPUTING: CONCLUSION

Pros:

On-demand GPU availability
Large computing power available
Best for offline/post-experiment data analysis
Extremely easy to provide AMIs with software for users when they

need it offline

Avoid conflicts between different software by providing several AMIs
Notebook analysis is very simple to use, flexible
Remote GUI processing possible (ssh –X,..)
Also great for tutorials (e.g. HERCULES)
On-demand cost ($0.2-1/hour/GPU)

Cons:

GPUs a bit outdated (no Maxwell, no Pascal) -> performance /2

compared to Maxwell Titan X

Notebook interface:
great only for linear data analysis process ?
No point-and-click interactivity
Not for ‘big’ data experiments (>>1Tb compressed) ?

TODO:

Simplified user auth / data access

26/07/2013 Page 6 l PANDAAS working group l 12 December 2016 l Vincent FAVRE-NICOLIN