Progress on BF Kernel Correction at UC Davis Craig Lage August 14, - PowerPoint PPT Presentation

Progress on BF Kernel Correction at UC Davis Craig Lage August 14, 2018 Acknowledgements:Andrew Bradshaw 1 / 15

Sum of Covariances Overestimates Variance - ?? Sum from -1 to 1 Sum from -4 to 4 Photon Transfer Curve and 'Lost Variance' SEGMENT13 Photon Transfer Curve and 'Lost Variance' SEGMENT13 10000 10000 Variance Only Variance + Covariance: 1 Pixels Away Variance Only Variance + Covariance: 4 Pixels Away Gain = 4.3968 Gain = 4.3141 Gain = 4.3968 Gain = 4.3717 Intercept = 28.3363 Intercept = -6.3875 Intercept = 28.3363 Intercept = -18.2966 8000 8000 Noise = 23.4 e- Quad Term = -2e-07 Noise = 23.4 e- Quad Term = 3e-07 C00 (@80K e-) = -0.062318 C00 (@80K e-) = -0.062318 Variance(ADU^2) Red Crosses - Linear part of Variance Variance(ADU^2) Red Crosses - Linear part of Variance 6000 6000 4000 4000 2000 2000 0 0 0 5000 10000 15000 20000 25000 30000 35000 40000 0 5000 10000 15000 20000 25000 30000 35000 40000 Flux(ADU) Flux(ADU) Sum from -2 to 2 Sum from -8 to 8 Photon Transfer Curve and 'Lost Variance' SEGMENT13 Photon Transfer Curve and 'Lost Variance' SEGMENT13 10000 10000 Variance Only Variance + Covariance: 2 Pixels Away Variance Only Variance + Covariance: 8 Pixels Away Gain = 4.3968 Gain = 4.3089 Gain = 4.3968 Gain = 4.3900 Intercept = 28.3363 Intercept = -5.8782 Intercept = 28.3363 Intercept = 2.7966 8000 8000 Noise = 23.4 e- Quad Term = 1e-08 Noise = 23.4 e- Quad Term = 5e-07 C00 (@80K e-) = -0.062318 C00 (@80K e-) = -0.062318 Variance(ADU^2) Red Crosses - Linear part of Variance Variance(ADU^2) Red Crosses - Linear part of Variance 6000 6000 4000 4000 2000 2000 0 0 0 5000 10000 15000 20000 25000 30000 35000 40000 0 5000 10000 15000 20000 25000 30000 35000 40000 Flux(ADU) Flux(ADU) . As we sum to larger and larger distances, the summed covariance overestimates the variance. The reason is not understood. 2 / 15

BF Kernel Correction - As Measured Covariances Covariance vs Distance Variance-1 As_Measured 10 -1 As_Measured C00; Data1=-0.0623; Data2=-0.0623 As_Measured_Neg As_Measured_Neg C10; Data=0.0040; Data2=0.0040 10 -2 C01; Data=0.0131; Data2=0.0131 Covariance B-F Kernel extracted from 3800 Flats 10 -3 Covariances(*1E7) - X-Slice Covariances(*1E7) - Y-Slice Covariances(*1E7) 0.0 0 0 0 2.5 10 -4 2 10 4 5.0 5 6 20 7.5 0 20 10 -5 0 20 0 20 10 0 10 1 10 2 i 2 + j 2 Kernel(*1E7) - X-Slice Kernel(*1E7) - Y-Slice Kernel(*1E7) 0 0.5 Brighter-Fatter - 30 micron Spots - SEGMENT13 0 0 0.0 1.10 10 0.5 X Slope = 0.80 % per 50K e-, Intercept = 1.027 1.0 1 1 Y Slope = 0.90 % per 50K e-, Intercept = 1.029 1.5 20 1.08 0 20 Corrected X Slope = 0.48 % per 50K e-, Intercept = 1.027 0 20 0 20 Corrected Y Slope = 0.49 % per 50K e-, Intercept = 1.029 Sigma (Pixels) 1.06 1.04 Sigma-x 1.02 Sigma-y Corrected Sigma-x Corrected Sigma-y 1.00 0 50000 100000 Central Peak(electrons) 3 / 15

Reduce Variance to force Zero Sum Covariance vs Distance Variance-1 As_Measured 10 -1 Zero_Sum C00; Data1=-0.0623; Data2=-0.0953 As_Measured_Neg Zero_Sum_Neg C10; Data=0.0040; Data2=0.0040 10 -2 C01; Data=0.0131; Data2=0.0131 Covariance B-F Kernel extracted from 3800 Flats 10 -3 Covariances(*1E7) - X-Slice Covariances(*1E7) - Y-Slice Covariances(*1E7) 0 0 0 0 2 10 -4 4 5 10 5 6 8 10 10 10 20 0 20 10 -5 0 20 0 20 10 0 10 1 10 2 i 2 + j 2 Kernel(*1E7) - X-Slice Kernel(*1E7) - Y-Slice Kernel(*1E7) 0 0 0 0.0 0.5 Brighter-Fatter - 30 micron Spots - SEGMENT13 1.0 1.10 1.5 2.0 2 2 10 2.5 X Slope = 0.80 % per 50K e-, Intercept = 1.027 3.0 3.5 Y Slope = 0.90 % per 50K e-, Intercept = 1.029 4.0 20 4 4 1.08 0 20 Corrected X Slope = 0.02 % per 50K e-, Intercept = 1.027 0 20 0 20 Corrected Y Slope = 0.02 % per 50K e-, Intercept = 1.029 Sigma (Pixels) 1.06 1.04 Sigma-x 1.02 Sigma-y Corrected Sigma-x Corrected Sigma-y 1.00 0 50000 100000 Central Peak(electrons) 4 / 15

Summary These slides are intended to summarize the progress we have made at UC Davis in implementing the Coulton,et.al. for correction of the Brighter-Fatter Effect. The work is far from done, but I wanted to capture where we stand. In particular, there are the following issues: Slide 2 shows that when we add up the covariances at larger and larger distances, we begin to over-compensate the lost variance from the center. Why is this?? Using the measured correlations directly in the Coulton,et.al. algorithm significantly undercorrects for the BF effect. In order to fully correct, I need to increase the magnitude of the central lost covariance by a large amount - about 50%. At this point, I’m not sure if these issues are a measurement issue, a code bug, or a deficiency in the method. On the positive side, it is clear that the method can correct well with the right kernel. See Slide 4. Next Steps: Get the algorithms used by the DM team on HSC images and compare the code. This may identify a code issue. Andrew plans to try the “best” kernel on more complex images from the star/galaxy mask. 5 / 15

BF Kernel applied to Optical Simulator galaxy images 6 / 15

Back-up slides and other attempts to improve correction 7 / 15

Measured Covariances - 3800 Flats Total Covariance vs Distance: 80000 e- 10 0 Correlation Coefficient Matrix Sim 10 0 Variance-1 Sim-Neg Model Variance-1 Positive Data 10 -1 Based on 3800 total flats Negative 10 1 Data-Neg Slope = -1.124 Slope = -1.047 Correlation Coefficient C10 = 0.003958 C00; Data=-0.0623; Sim=-0.0708 C01 = 0.013147 2 10 -2 10 C10; Data=0.0131; Sim=0.0057 δ Area / Area C00 = -0.062278 C01; Data=0.0040; Sim=0.0122 χ 2 = 1071.10 10 3 10 -3 10 4 10 -4 10 5 10 0 10 1 10 2 i 2 + j 2 10 -5 10 0 10 1 10 2 i 2 + j 2 . Measured correlations agree reasonably well with simulated area distortions(right) - we can’t be too far off. Two “close in” negative correlations are C20 and C30 - these are believed to be impacted by serial deferred charge issue. C10 is probably also impacted and this will need to be compensated for. 8 / 15

We estimate covariance by keeping only the part that grows quadratically with flux. Covariance of Pixel (0,1) Covariance of Pixel (1,0) Covariance of Pixel (1,1) 400 400 400 Correlations(ADU^2) Correlations(ADU^2) Correlations(ADU^2) 350 350 350 Intercept = -2.319531 Intercept = 1.956354 Intercept = -1.149540 300 300 300 Slope = 0.000763 Slope = 0.000754 Slope = 0.000319 250 250 250 Quad Term = 1.64157e-07 Quad Term = 4.34528e-08 Quad Term = 3.74959e-08 200 200 200 Cov_Value = 0.013133 Cov_Value = 0.003476 Cov_Value = 0.003000 150 150 150 100 100 100 50 50 50 0 0 0 50 50 50 0 5000 10000150002000025000300003500040000 0 5000 10000150002000025000300003500040000 0 5000 10000150002000025000300003500040000 Flux(ADU) Flux(ADU) Flux(ADU) Covariance of Pixel (2,0) Covariance of Pixel (0,2) Covariance of Pixel (2,2) 400 400 400 Correlations(ADU^2) 350 Correlations(ADU^2) 350 Correlations(ADU^2) 350 Intercept = 3.310491 Intercept = -0.116212 Intercept = -0.158036 300 300 300 Slope = 0.000432 Slope = -0.000145 Slope = -0.000129 250 250 250 Quad Term = 7.7489e-10 Quad Term = 2.88966e-08 Quad Term = 1.10857e-08 200 200 200 Cov_Value = 0.000062 Cov_Value = 0.002312 Cov_Value = 0.000887 150 150 150 100 100 100 50 50 50 0 0 0 50 50 50 0 5000 10000150002000025000300003500040000 0 5000 10000150002000025000300003500040000 0 5000 10000150002000025000300003500040000 Flux(ADU) Flux(ADU) Flux(ADU) . The intent here is to remove correlations not due to the BF effect. 9 / 15

BF Kernel Correction - Best Results B-F Kernel extracted from 3800 Flats Brighter-Fatter - 30 micron Spots - SEGMENT13 Covariances(*1E7) - X-Slice Covariances(*1E7) - Y-Slice 1.10 Covariances(*1E7) 0 0 0 0 X Slope = 0.80 % per 50K e-, Intercept = 1.031 2 Y Slope = 0.91 % per 50K e-, Intercept = 1.033 4 5 10 5 1.08 6 Corrected X Slope = 0.01 % per 50K e-, Intercept = 1.032 8 10 Corrected Y Slope = 0.05 % per 50K e-, Intercept = 1.034 10 10 20 Sigma (Pixels) 0 20 1.06 0 20 0 20 Kernel(*1E7) - X-Slice Kernel(*1E7) - Y-Slice Kernel(*1E7) 1.04 0 0 0 0.0 0.5 1.0 1.5 Sigma-x 2.0 2 2 10 1.02 Sigma-y 2.5 3.0 Corrected Sigma-x 3.5 4.0 Corrected Sigma-y 20 4 4 0 20 1.00 0 20 0 20 0 50000 100000 Central Peak(electrons) . Measured covariances are used for nearest neighbors. For R=2 and beyond, use modeled covariances (the green crosses in the left hand plot of Figure 5). Make C00 more negative to force zero sum. Assumes some effect is causing the magnitude of the lost central variance to be reduced. 10 / 15