Simulations of Misaligned Accretion GPU-accelerated onto Spinning - PowerPoint PPT Presentation

Simulations of Misaligned Accretion GPU-accelerated onto Spinning Black Holes Sasha Tchekhovskoy (PI, Northwestern) Matthew Liska, Casper Hesp (Amsterdam), Zack Andalman (Evanston Township High School) 
 Eric Coughlin, Nick Stone (Co-PI) (Columbia) Van Moer (NCSA) Younsi (UCL) Mark Ziri

How Do Black Holes Explode Galaxies/Clusters? Perseus Cluster (Fabian et al. 2003) (Forman et al. 2007) M87 MS0735.6 (McNamara et al. 2009) JETS! JETS! Alexander (Sasha) Tchekhovskoy Blue Waters Symposium 2018

We are Missing Something Important! MS0735.6 NASA NASA w ~ o a fl t u o ?? inflow BH ?? jet (McNamara et al. 2009) YES: typical disks are tilted No: we do not understand them (yet) Alexander (Sasha) Tchekhovskoy Blue Waters Symposium 2018

Tilted Disks are Hot • Thick disks precess due to general 
 relativistic frame dragging by BH spin 
 (Fragile et al. 2005, 2007; McKinney, AT+2013) ~ a • Thin disks can align due to 
 Bardeen-Petterson (1975) effect BH • Seen only in pseudo-Newtonian inflow simulations, not in GR 
 (Nixon et al. 2012; Nealon et al. 2015) • Do thin disks align in GR? ~ a • Challenge: enormous dynamical range . Need to resolve thin disk BH over long run times. • Cost ∝ (h/r) -5 -- prohibitive! • How could we possibly pull this off??! Alexander (Sasha) Tchekhovskoy Blue Waters Symposium 2018

H-AMR: What’s Your Nail? • Multi-GPU 3D H-AMR (“hammer”, Liska, AT, et al. 2018) : • Based on HARMPI • 85% parallel scaling to 4096 GPUs (MPI, OpenMP , OpenCL, CUDA, NVLINK, GPUDIRECT) • 100-1000x speedup on 1 GPU vs 1 BW CPU core 100M Matthew Liska 
 • Advanced features (extra few - 10x speedup): (U of Amsterdam) $10k • Adaptive Mesh Refinement (AMR) • Local adaptive time-stepping • Blue Waters is crucial for enabling 
 next-generation research: • 5M GPU-hours/yr = 5B CPU core-hours/yr 
 3M on NSF Blue Waters supercomputer • Science is no longer limited by 
 $30k computational resources ! Alexander (Sasha) Tchekhovskoy Blue Waters Symposium 2018

Resolution Matters 1x (low) resolution 2x resolution a = 0.93 h / r = 0.3 i = 30 ° (Liska, Hesp, AT+2018a) *thanks to Mark Van Moer 4x resolution is similar to 2x: 
 results are similar -> convergence

Credit: Z. Younsi, M. Liska, C. Hesp

Thick-ish Disks Precess and Align a = 0.93 Precession and i = 45 ° alignment h / r = 0.1 increase the Casper Hesp probability of (University of GRB detection Amsterdam) in BH-NS *thanks to Mark Van mergers Moer

Thin Weakly Misaligned Disks Align 2 2 80 40 a = 0.93 -3 0 -3 i = 10 ° h / r = 0.03 -40 Liska, Hesp, AT+2018b -80 -8 -8 -4 4 8 -80 -40 40 -8 0 0 80 • Thinnest disk simulations to date: h / r = 0.03 • First demonstration of (Bardeen-Petterson?) alignment in a general relativistic MHD simulation of a thin disk • E ffective resolution 2880 × 860 × 1200 , 3 AMR levels

Thin Strongly Misaligned Disks Align and Break 2 2 80 40 a = 0.93 -3 0 -3 i = 45 ° h / r = 0.03 -40 Liska, Hesp, AT+2018b -80 -8 -8 -4 4 8 -80 -40 40 -8 0 0 80 • First demonstration of (Bardeen-Petterson?) alignment and disk breaking in GRMHD! • E ffective resolution 2880 × 860 × 1200 , 3 AMR levels

Thin VERY Strongly Misaligned Disks Tear • Disks can tear up into individual segments • Extra dissipation and luminosity • Completely different luminosity profile • Can affect BH spin measurements • Larger observed disk size than 3 AMR levels expected? Effective resolution: 2880x860x1200 (Blackburn+2011)

Thinnest disk to date: h / r = 0.03 3 AMR levels Effective resolution: 2880x860x1200

Ridiculously thin disk: h / r = 0.015 4 AMR levels Effective resolution: 5760x1720x2400

Making the Disk from Scratch -5 -5 preliminary 400 a = 0.93 200 i = 0 ° parabolic orbit 0 R p = 7R g M BH = 10 6 M ∗ -200 -400 Liska, Hesp, AT+2018c -12 -12 -100 -50 0 50 100 -400 -200 0 200 400 • Initial conditions computed by the phantom code by Eric Coughlin • First simulation in GR of a star on a parabolic orbit tidally disrupted by supermassive BH, M BH = 10 6 M ∗ ( ≫ 500M ∗ , Shiokawa+15, see also Sadowski+2016 ) • effective resolution 1792 × 860 × 1200 , 3 AMR levels Andalman et al 2018, in prep

D i s k A p p e a r s t o F w o i r t h m a T i l t , T o o *thanks to Casper Hesp, Mark Van Moer

Advances Enabled by Blue Waters • We are beginning to understand typical, tilted accretion at unprecedented resolutions • Jets can precess together with tilted Matthew Liska disks over multiple precession cycles Zack Andalman (University of (Evanston Amsterdam) • Bardeen-Petterson-like alignment, Township High School) breaking, and tearing of thin disks first seen in GRMHD ➛ essentially unexplored observational manifestations • Advances in making disks from Casper Hesp scratch by pulling stars apart (University of Amsterdam)