Machine Learning and Artificial Intelligence in Recent Trends, - - PowerPoint PPT Presentation

SLIDE 1

Sebastian Raschka, Ph.D.

• Assist. Prof.

• Dep. of Statistics

1

Machine Learning and Artificial Intelligence in

sebastianraschka.com @rasbt

Recent Trends, Technologies, and Challenges

SLIDE 2

2

information

Article

Machine Learning in Python: Main Developments and Technology Trends in Data Science, Machine Learning, and Artificial Intelligence

Sebastian Raschka 1,*,†, Joshua Patterson 2 and Corey Nolet 2,3

1

Department of Statistics, University of Wisconsin-Madison, Madison, WI 53575, USA

2

NVIDIA, Santa Clara, CA 95051, USA; joshuap@nvidia.com (J.P.); cnolet@nvidia.com (C.N.)

3

Department of Comp Sci & Electrical Engineering, University of Maryland, Baltimore County, Baltimore, MD 21250, USA * Correspondence: sraschka@wisc.edu † Current address: 1300 University Ave, Medical Sciences Building, Madison, WI 53706, USA. Received: 6 February 2020; Accepted: 31 March 2020; Published: 4 April 2020

• See “Machine Learning with Python,” a special issue of Information (ISSN 2078-2489)

https://www.mdpi.com/journal/information/special_issues/ML_Python

SLIDE 3

3

Part 1: Technologies and Tools

SLIDE 4

4

https://cse.engin.umich.edu/about/history/

https://careers.google.com/locations/pittsburgh/

SLIDE 5

Python for-loops are bad

5

z = ∑

i

xiwi + b

SLIDE 6

z = ∑

i

xiwi + b = x⊤w

Python for-loops are bad: Use SIMD & vectorized code whenever you can

6

SLIDE 7

For-loops vs. Vectorized Code

7

Dot product is approx.1500x faster

SLIDE 8

Can we speed this up further using GPUs?

8

GPU is approx. 3x slower

SLIDE 9

9

Can we speed this up further using GPUs? (Yes, if the data / computation is large)

GPU is approx. 230x faster

SLIDE 10

10

CPU Memory Data Preparation Visualization Model Training Dask Pandas Analytics Scikit-Learn Machine Learning Network-X Graph Analytics PyTorch Chainer MxNet Deep Learning Matplotlib Seaborn Visualization GPU Memory

Figure 1. The standard Python ecosystem for machine learning, data science, and scientific computing.

Sebastian Raschka, Joshua Patterson, and Corey Nolet (2020)
 Machine Learning in Python: Main developments and technology trends in data science, machine learning, and artificial intelligence
 Information 2020, 11, 193

SLIDE 11

11

GPU Memory Data Preparation Visualization Model Training Dask cuDF cuIO Analytics cuML Machine Learning cuGraph Graph Analytics PyTorch Chainer MxNet Deep Learning cuXfilter <> pyViz Visualization

Figure 4. RAPIDS is an open source effort to support and grow the ecosystem of GPU-accelerated Python tools for data science, machine learning, and scientific computing. RAPIDS supports existing libraries, fills gaps by providing open source libraries with crucial components that are missing from the Python community, and promotes cohesion across the ecosystem by supporting interoperability across the libraries.

Sebastian Raschka, Joshua Patterson, and Corey Nolet (2020)
 Machine Learning in Python: Main developments and technology trends in data science, machine learning, and artificial intelligence
 Information 2020, 11, 193

SLIDE 12

GPUs

12

• https://www.theverge.com/2015/5/31/8695075/nvidia-geforce-gtx-980-ti-announced

Image: https://www.amazon.com/Nvidia-GEFORCE-GTX-1080-Ti/dp/B06XH5ZCLP Image: https://www.nvidia.com/en-in/geforce/graphics-cards/rtx-2080-ti/

Sebastian Raschka. Python Machine Learning. Birmingham, UK: Packt Publishing, 2015 Sebastian Raschka and Vahid

• Mirjalili. Python Machine

Learning 2nd Ed. Birmingham, UK: Packt Publishing, 2017 Sebastian Raschka and Vahid

• Mirjalili. Python Machine

Learning 3rd Ed. Birmingham, UK: Packt Publishing, 2019

SLIDE 13

13

Source: https://lambdalabs.com/blog/2080-ti-deep-learning-benchmarks/

~$8,000; 16 Gb Memory ~$1,100; 11 Gb Memory

SLIDE 14

14

https://arstechnica.com/gadgets/2018/07/the-ai-revolution-has-spawned-a-new-chips-arms-race/

Developing Specialized Hardware

https://developer.arm.com/products/processors/machine-learning/arm-ml-processor https://www.reuters.com/article/us-amazon-com-nvidia/amazon-launches-machine-learning-chip- taking-on-nvidia-intel-idUSKCN1NX2PY https://www.marketwatch.com/story/new-nvidia-chip-extends-the-companys-lead-in- graphics-artificial-intelligence-2018-08-14

SLIDE 15

15

Which DL framework 
 is most popular?

Deep Learning frameworks

The new "Emacs vs VIM"

SLIDE 16

16

https://thegradient.pub/state-of-ml-frameworks-2019-pytorch-dominates-research- tensorflow-dominates-industry/

SLIDE 17

17

https://thegradient.pub/state-of-ml-frameworks-2019-pytorch-dominates-research- tensorflow-dominates-industry/

This graph was generated by scraping every paper published in a major ML conference over the last few years. Papers were categorized based on whether they mention PyTorch or TensorFlow, excluding papers with authors affiliated with either Google or Facebook, as well as those that mention both TensorFlow and PyTorch

SLIDE 18

"Most I've spoken to (and I'm from a background in ML academia); PyTorch is by a very slim margin faster than TensorFlow 2.0 in our experiences when you run TensorFlow in non-Eager mode. However, since Eager mode is now enabled by default in TensorFlow 2.0; PyTorch is significantly faster."

18

https://www.reddit.com/r/MachineLearning/comments/f19dj4/d_tensorflow_20_v_pytorch_performance_question/

SLIDE 19

19

Source: https://medium.com/huggingface/benchmarking-transformers-pytorch-and-tensorflow-e2917fb891c2

SLIDE 20

20

b

In In: Out: Out:

Defining g the gr graph Initializing g and evaluating g the gr graph

a

In In: Out: Out:

Figure 7. Comparison between (a) a static computation graph in TensorFlow 1.15 and (b) an imperative programming paradigm enabled by dynamic graphs in PyTorch 1.4.

Sebastian Raschka, Joshua Patterson, and Corey Nolet (2020)
 Machine Learning in Python: Main developments and technology trends in data science, machine learning, and artificial intelligence
 Information 2020, 11, 193

SLIDE 21

21

"intermediate" representation (IR) 
 <=> "static graph"

Image Source: https://thegradient.pub/state-of-ml- frameworks-2019-pytorch-dominates-research- tensorflow-dominates-industry/

Two ways for turning a PyTorch model into a static graph for

• ptimization and

deployment:


a) Tracing b) Scripting

DL Frameworks are converging

SLIDE 22

22

DL Frameworks are converging

SLIDE 23

23

Technologies

Challenges

"Hot" research areas

SLIDE 24

Challenges:

24

Adding forward-mode autodiff for efficient higher-order derivatives (e.g., Hessians)

SLIDE 25

25

JAX

Forward mode AD for PyTorch: https:// github.com/pytorch/pytorch/issues/10223 Composable transformations of Python+NumPy programs: differentiate, vectorize, JIT to GPU/TPU, and more

Swift

SLIDE 26

Challenges:

26

Adversarial attacks

SLIDE 27

27

Phantom of the ADAS: Phantom Attacks on Driver-Assistance Systems

Ben Nassi1, Dudi Nassi1, Raz Ben-Netanel1, Yisroel Mirsky1,2, Oleg Drokin3, Yuval Elovici1 Video Demonstration - https://youtu.be/1cSw4fXYqWI {nassib,nassid,razx,yisroel,elovici}@post.bgu.ac.il, green@linuxhacker.ru

1 Ben-Gurion University of the Negev, 2 Georgia Tech,3 Independent Tesla Researcher

ABSTRACT The absence of deployed vehicular communication sys- tems, which prevents the advanced driving assistance systems (ADASs) and autopilots of semi/fully autonomous cars to validate their virtual perception regarding the physical en- vironment surrounding the car with a third party, has been exploited in various attacks suggested by researchers. Since the application of these attacks comes with a cost (exposure

• f the attacker’s identity), the delicate exposure vs. application

balance has held, and attacks of this kind have not yet been encountered in the wild. In this paper, we investigate a new perceptual challenge that causes the ADASs and autopi- lots of semi/fully autonomous to consider depthless objects (phantoms) as real. We show how attackers can exploit this perceptual challenge to apply phantom attacks and change the abovementioned balance, without the need to physically approach the attack scene, by projecting a phantom via a

a b

• Fig. 1: Perceptual Challenge: Would you consider the projec-

tion of the person (a) and road sign (b) real? Telsa considers (a) a real person and Mobileye 630 PRO considers (b) a real road sign.

SLIDE 28

28

a b

• Fig. 1: Perceptual Challenge: Would you consider the projec-

tion of the person (a) and road sign (b) real? Telsa considers (a) a real person and Mobileye 630 PRO considers (b) a real road sign.

SLIDE 29

29

GenAttack [199] no yes no no no Defense mechanisms Feature Squeezing [200] no no yes no yes Spatial Smoothing [200] no no yes no yes Label Smoothing [200] no no yes no yes Gaussian Augmentation [201] no no yes no yes Adversarial Training [185] no no yes yes yes Thermometer Encoding [202] no no yes yes yes NAT [203] no no no yes no EAT [177] no no no yes no DD [204] no no no yes no IGR [205] no no no yes no EIT [206] no no yes yes no RT [207] no no no yes no PixelDefend [208] no no yes yes no Regr.-based classfication [209] no no no yes no JPEG compression [210] no no yes no no

shown).

Cleverhans v3.0.1 FoolBox v2.3.0 ART v1.1.0 DEEPSEC (2019) AdvBox v0.4.1 Supported frameworks TensorFlow yes yes yes no yes MXNet yes yes yes no yes PyTorch no yes yes yes yes PaddlePaddle no no no no yes (Evasion) attack mechanisms BLB [163] yes no no yes no AMD [170] yes no no no no ZOO [171] no no yes no no VA [172] yes yes yes no no AP [173] no no yes no no STA [174] no yes yes no no DTA [175] no no yes no no FGSM [176] yes yes yes yes yes R+FGSM [177] no no no yes no R+LLC [177] no no no yes no U-MI-FGSM [178] yes yes no yes no T-MI-FGSM [178] yes yes no yes no BIM [179] no yes yes yes yes LLC / ILLC [179] no yes no yes no UAP [180] no no yes yes no DeepFool [181] yes yes yes yes yes NewtonFool [182] no yes yes no no JSMA [183] yes yes yes yes yes CW/CW2 [184] yes yes yes yes yes PGD [185] yes no yes yes yes OM [186] no no no yes no EAD [187] yes yes yes yes no Boundary Attack [188] no yes yes no no HopSkipJumpAttack [189] yes yes yes no no MaxConf [190] yes no no no no Inversion attack [191] yes yes no no no SparseL1 [192] yes yes no no no SPSA [193] yes no no no no HCLU [194] no no yes no no ADef [195] no yes no no no DDNL2 [196] no yes no no no Local Search [197] no yes no no no Pointwise attack [198] no yes no no no GenAttack [199] no yes no no no Defense mechanisms

Machine Learning in Python: Main developments and technology trends in data science, machine learning, and artificial intelligence (2020). Sebastian Raschka, Joshua Patterson, and Corey Nolet

SLIDE 30

Neglected Research Areas

30

SLIDE 31

31

Wenzhi Cao, Vahid Mirjalili, and Sebastian Raschka. "Rank-consistent ordinal regression for neural networks." arXiv:1901.07884 (2019).

Input image 7x7 conv @64 stride=2

…

3x3 conv @512 stride=1

. . .

Weight sharing across k-1 tasks

b1

<latexit sha1_base64="oxRwFKiWcKO5u9w5ICndUHjxWI=">AB6nicbZBNS8NAEIYn9avWr6hHL4tF8FQSEfRY9OKxov2ANpTNdtMu3WzC7kQoT/BiwdFvPqLvPlv3LY5aOsLCw/vzLAzb5hKYdDzvp3S2vrG5lZ5u7Kzu7d/4B4etUySacabLJGJ7oTUcCkUb6JAyTup5jQOJW+H49tZvf3EtRGJesRJyoOYDpWIBKNorYew7/fdqlfz5iKr4BdQhUKNvVGyQsi7lCJqkxXd9LMcipRsEkn1Z6meEpZWM65F2LisbcBPl81Sk5s86ARIm2TyGZu78nchobM4lD2xlTHJnl2sz8r9bNMLoOcqHSDLli4+iTBJMyOxuMhCaM5QTC5RpYXclbEQ1ZWjTqdgQ/OWTV6F1UfMt319W6zdFHGU4gVM4Bx+uoA530IAmMBjCM7zCmyOdF+fd+Vi0lpxi5hj+yPn8AeofjYo=</latexit><latexit sha1_base64="oxRwFKiWcKO5u9w5ICndUHjxWI=">AB6nicbZBNS8NAEIYn9avWr6hHL4tF8FQSEfRY9OKxov2ANpTNdtMu3WzC7kQoT/BiwdFvPqLvPlv3LY5aOsLCw/vzLAzb5hKYdDzvp3S2vrG5lZ5u7Kzu7d/4B4etUySacabLJGJ7oTUcCkUb6JAyTup5jQOJW+H49tZvf3EtRGJesRJyoOYDpWIBKNorYew7/fdqlfz5iKr4BdQhUKNvVGyQsi7lCJqkxXd9LMcipRsEkn1Z6meEpZWM65F2LisbcBPl81Sk5s86ARIm2TyGZu78nchobM4lD2xlTHJnl2sz8r9bNMLoOcqHSDLli4+iTBJMyOxuMhCaM5QTC5RpYXclbEQ1ZWjTqdgQ/OWTV6F1UfMt319W6zdFHGU4gVM4Bx+uoA530IAmMBjCM7zCmyOdF+fd+Vi0lpxi5hj+yPn8AeofjYo=</latexit><latexit sha1_base64="oxRwFKiWcKO5u9w5ICndUHjxWI=">AB6nicbZBNS8NAEIYn9avWr6hHL4tF8FQSEfRY9OKxov2ANpTNdtMu3WzC7kQoT/BiwdFvPqLvPlv3LY5aOsLCw/vzLAzb5hKYdDzvp3S2vrG5lZ5u7Kzu7d/4B4etUySacabLJGJ7oTUcCkUb6JAyTup5jQOJW+H49tZvf3EtRGJesRJyoOYDpWIBKNorYew7/fdqlfz5iKr4BdQhUKNvVGyQsi7lCJqkxXd9LMcipRsEkn1Z6meEpZWM65F2LisbcBPl81Sk5s86ARIm2TyGZu78nchobM4lD2xlTHJnl2sz8r9bNMLoOcqHSDLli4+iTBJMyOxuMhCaM5QTC5RpYXclbEQ1ZWjTqdgQ/OWTV6F1UfMt319W6zdFHGU4gVM4Bx+uoA530IAmMBjCM7zCmyOdF+fd+Vi0lpxi5hj+yPn8AeofjYo=</latexit><latexit sha1_base64="oxRwFKiWcKO5u9w5ICndUHjxWI=">AB6nicbZBNS8NAEIYn9avWr6hHL4tF8FQSEfRY9OKxov2ANpTNdtMu3WzC7kQoT/BiwdFvPqLvPlv3LY5aOsLCw/vzLAzb5hKYdDzvp3S2vrG5lZ5u7Kzu7d/4B4etUySacabLJGJ7oTUcCkUb6JAyTup5jQOJW+H49tZvf3EtRGJesRJyoOYDpWIBKNorYew7/fdqlfz5iKr4BdQhUKNvVGyQsi7lCJqkxXd9LMcipRsEkn1Z6meEpZWM65F2LisbcBPl81Sk5s86ARIm2TyGZu78nchobM4lD2xlTHJnl2sz8r9bNMLoOcqHSDLli4+iTBJMyOxuMhCaM5QTC5RpYXclbEQ1ZWjTqdgQ/OWTV6F1UfMt319W6zdFHGU4gVM4Bx+uoA530IAmMBjCM7zCmyOdF+fd+Vi0lpxi5hj+yPn8AeofjYo=</latexit>

b2

<latexit sha1_base64="KmwIREzjRQ4yYecphZP2rZMfohI=">AB6nicbZBNS8NAEIYn9avWr6pHL4tF8FSItRj0YvHivYD2lA20m7dLMJuxuhP4ELx4U8eov8ua/cdvmoK0vLDy8M8POvEiuDau+0UNja3tneKu6W9/YPDo/LxSVvHqWLYrGIVTegGgWX2DLcCOwmCmkUCOwEk9t5vfOESvNYPpgn5ER5KHnFjrYdgUBuUK27VXYisg5dDBXI1B+Wv/jBmaYTSMEG17nluYvyMKsOZwFmpn2pMKJvQEfYsShqh9rPFqjNyYZ0hCWNlnzRk4f6eyGik9TQKbGdEzViv1ubmf7VeasJrP+MySQ1KtvwoTAUxMZnfTYZcITNiaoEyxe2uhI2poszYdEo2BG/15HVo16qe5furSuMmj6MIZ3AOl+BHRpwB01oAYMRPMrvDnCeXHenY9la8HJZ07hj5zPH+ujYs=</latexit><latexit sha1_base64="KmwIREzjRQ4yYecphZP2rZMfohI=">AB6nicbZBNS8NAEIYn9avWr6pHL4tF8FSItRj0YvHivYD2lA20m7dLMJuxuhP4ELx4U8eov8ua/cdvmoK0vLDy8M8POvEiuDau+0UNja3tneKu6W9/YPDo/LxSVvHqWLYrGIVTegGgWX2DLcCOwmCmkUCOwEk9t5vfOESvNYPpgn5ER5KHnFjrYdgUBuUK27VXYisg5dDBXI1B+Wv/jBmaYTSMEG17nluYvyMKsOZwFmpn2pMKJvQEfYsShqh9rPFqjNyYZ0hCWNlnzRk4f6eyGik9TQKbGdEzViv1ubmf7VeasJrP+MySQ1KtvwoTAUxMZnfTYZcITNiaoEyxe2uhI2poszYdEo2BG/15HVo16qe5furSuMmj6MIZ3AOl+BHRpwB01oAYMRPMrvDnCeXHenY9la8HJZ07hj5zPH+ujYs=</latexit><latexit sha1_base64="KmwIREzjRQ4yYecphZP2rZMfohI=">AB6nicbZBNS8NAEIYn9avWr6pHL4tF8FSItRj0YvHivYD2lA20m7dLMJuxuhP4ELx4U8eov8ua/cdvmoK0vLDy8M8POvEiuDau+0UNja3tneKu6W9/YPDo/LxSVvHqWLYrGIVTegGgWX2DLcCOwmCmkUCOwEk9t5vfOESvNYPpgn5ER5KHnFjrYdgUBuUK27VXYisg5dDBXI1B+Wv/jBmaYTSMEG17nluYvyMKsOZwFmpn2pMKJvQEfYsShqh9rPFqjNyYZ0hCWNlnzRk4f6eyGik9TQKbGdEzViv1ubmf7VeasJrP+MySQ1KtvwoTAUxMZnfTYZcITNiaoEyxe2uhI2poszYdEo2BG/15HVo16qe5furSuMmj6MIZ3AOl+BHRpwB01oAYMRPMrvDnCeXHenY9la8HJZ07hj5zPH+ujYs=</latexit><latexit sha1_base64="KmwIREzjRQ4yYecphZP2rZMfohI=">AB6nicbZBNS8NAEIYn9avWr6pHL4tF8FSItRj0YvHivYD2lA20m7dLMJuxuhP4ELx4U8eov8ua/cdvmoK0vLDy8M8POvEiuDau+0UNja3tneKu6W9/YPDo/LxSVvHqWLYrGIVTegGgWX2DLcCOwmCmkUCOwEk9t5vfOESvNYPpgn5ER5KHnFjrYdgUBuUK27VXYisg5dDBXI1B+Wv/jBmaYTSMEG17nluYvyMKsOZwFmpn2pMKJvQEfYsShqh9rPFqjNyYZ0hCWNlnzRk4f6eyGik9TQKbGdEzViv1ubmf7VeasJrP+MySQ1KtvwoTAUxMZnfTYZcITNiaoEyxe2uhI2poszYdEo2BG/15HVo16qe5furSuMmj6MIZ3AOl+BHRpwB01oAYMRPMrvDnCeXHenY9la8HJZ07hj5zPH+ujYs=</latexit>

w1

<latexit sha1_base64="ozSIzVA/SGXegmac4XRXthOpvw0=">AB6nicbZBNS8NAEIYn9avWr6pHL4tF8FQSEeqx6MVjRfsBbSib7aZdutmE3YlSQn+CFw+KePUXefPfuG1z0NYXFh7emWFn3iCRwqDrfjuFtfWNza3idmlnd2/oHx41DJxqhlvsljGuhNQw6VQvIkCJe8kmtMokLwdjG9m9fYj10bE6gEnCfcjOlQiFIyite6f+l6/XHGr7lxkFbwcKpCr0S9/9QYxSyOukElqTNdzE/QzqlEwyaelXmp4QtmYDnXoqIRN342X3VKzqwzIGs7VNI5u7viYxGxkyiwHZGFEdmuTYz/6t1Uwyv/EyoJEWu2OKjMJUEYzK7mwyE5gzlxAJlWthdCRtRTRnadEo2BG/5FVoXVQ9y3eXlfp1HkcRTuAUzsGDGtThFhrQBAZDeIZXeHOk8+K8Ox+L1oKTzxzDHzmfPwosjZ8=</latexit><latexit sha1_base64="ozSIzVA/SGXegmac4XRXthOpvw0=">AB6nicbZBNS8NAEIYn9avWr6pHL4tF8FQSEeqx6MVjRfsBbSib7aZdutmE3YlSQn+CFw+KePUXefPfuG1z0NYXFh7emWFn3iCRwqDrfjuFtfWNza3idmlnd2/oHx41DJxqhlvsljGuhNQw6VQvIkCJe8kmtMokLwdjG9m9fYj10bE6gEnCfcjOlQiFIyite6f+l6/XHGr7lxkFbwcKpCr0S9/9QYxSyOukElqTNdzE/QzqlEwyaelXmp4QtmYDnXoqIRN342X3VKzqwzIGs7VNI5u7viYxGxkyiwHZGFEdmuTYz/6t1Uwyv/EyoJEWu2OKjMJUEYzK7mwyE5gzlxAJlWthdCRtRTRnadEo2BG/5FVoXVQ9y3eXlfp1HkcRTuAUzsGDGtThFhrQBAZDeIZXeHOk8+K8Ox+L1oKTzxzDHzmfPwosjZ8=</latexit><latexit sha1_base64="ozSIzVA/SGXegmac4XRXthOpvw0=">AB6nicbZBNS8NAEIYn9avWr6pHL4tF8FQSEeqx6MVjRfsBbSib7aZdutmE3YlSQn+CFw+KePUXefPfuG1z0NYXFh7emWFn3iCRwqDrfjuFtfWNza3idmlnd2/oHx41DJxqhlvsljGuhNQw6VQvIkCJe8kmtMokLwdjG9m9fYj10bE6gEnCfcjOlQiFIyite6f+l6/XHGr7lxkFbwcKpCr0S9/9QYxSyOukElqTNdzE/QzqlEwyaelXmp4QtmYDnXoqIRN342X3VKzqwzIGs7VNI5u7viYxGxkyiwHZGFEdmuTYz/6t1Uwyv/EyoJEWu2OKjMJUEYzK7mwyE5gzlxAJlWthdCRtRTRnadEo2BG/5FVoXVQ9y3eXlfp1HkcRTuAUzsGDGtThFhrQBAZDeIZXeHOk8+K8Ox+L1oKTzxzDHzmfPwosjZ8=</latexit><latexit sha1_base64="ozSIzVA/SGXegmac4XRXthOpvw0=">AB6nicbZBNS8NAEIYn9avWr6pHL4tF8FQSEeqx6MVjRfsBbSib7aZdutmE3YlSQn+CFw+KePUXefPfuG1z0NYXFh7emWFn3iCRwqDrfjuFtfWNza3idmlnd2/oHx41DJxqhlvsljGuhNQw6VQvIkCJe8kmtMokLwdjG9m9fYj10bE6gEnCfcjOlQiFIyite6f+l6/XHGr7lxkFbwcKpCr0S9/9QYxSyOukElqTNdzE/QzqlEwyaelXmp4QtmYDnXoqIRN342X3VKzqwzIGs7VNI5u7viYxGxkyiwHZGFEdmuTYz/6t1Uwyv/EyoJEWu2OKjMJUEYzK7mwyE5gzlxAJlWthdCRtRTRnadEo2BG/5FVoXVQ9y3eXlfp1HkcRTuAUzsGDGtThFhrQBAZDeIZXeHOk8+K8Ox+L1oKTzxzDHzmfPwosjZ8=</latexit>

w2

<latexit sha1_base64="sAe26MpFncoK5AcSpzUnkA9I=">AB6nicbZBNS8NAEIYn9avWr6pHL4tF8FSIuix6MVjRfsBbSib7aZdutmE3YlSQn+CFw+KePUXefPfuG1z0NYXFh7emWFn3iCRwqDrfjuFtfWNza3idmlnd2/oHx41DJxqhlvsljGuhNQw6VQvIkCJe8kmtMokLwdjG9m9fYj10bE6gEnCfcjOlQiFIyite6f+rV+ueJW3bnIKng5VCBXo1/+6g1ilkZcIZPUmK7nJuhnVKNgk9LvdTwhLIxHfKuRUjbvxsvuqUnFlnQMJY26eQzN3fExmNjJlEge2MKI7Mcm1m/lfrphe+ZlQSYpcscVHYSoJxmR2NxkIzRnKiQXKtLC7EjaimjK06ZRsCN7yavQqlU9y3cXlfp1HkcRTuAUzsGDS6jDLTSgCQyG8Ayv8OZI58V5dz4WrQUnzmGP3I+fwALsI2g</latexit><latexit sha1_base64="sAe26MpFncoK5AcSpzUnkA9I=">AB6nicbZBNS8NAEIYn9avWr6pHL4tF8FSIuix6MVjRfsBbSib7aZdutmE3YlSQn+CFw+KePUXefPfuG1z0NYXFh7emWFn3iCRwqDrfjuFtfWNza3idmlnd2/oHx41DJxqhlvsljGuhNQw6VQvIkCJe8kmtMokLwdjG9m9fYj10bE6gEnCfcjOlQiFIyite6f+rV+ueJW3bnIKng5VCBXo1/+6g1ilkZcIZPUmK7nJuhnVKNgk9LvdTwhLIxHfKuRUjbvxsvuqUnFlnQMJY26eQzN3fExmNjJlEge2MKI7Mcm1m/lfrphe+ZlQSYpcscVHYSoJxmR2NxkIzRnKiQXKtLC7EjaimjK06ZRsCN7yavQqlU9y3cXlfp1HkcRTuAUzsGDS6jDLTSgCQyG8Ayv8OZI58V5dz4WrQUnzmGP3I+fwALsI2g</latexit><latexit sha1_base64="sAe26MpFncoK5AcSpzUnkA9I=">AB6nicbZBNS8NAEIYn9avWr6pHL4tF8FSIuix6MVjRfsBbSib7aZdutmE3YlSQn+CFw+KePUXefPfuG1z0NYXFh7emWFn3iCRwqDrfjuFtfWNza3idmlnd2/oHx41DJxqhlvsljGuhNQw6VQvIkCJe8kmtMokLwdjG9m9fYj10bE6gEnCfcjOlQiFIyite6f+rV+ueJW3bnIKng5VCBXo1/+6g1ilkZcIZPUmK7nJuhnVKNgk9LvdTwhLIxHfKuRUjbvxsvuqUnFlnQMJY26eQzN3fExmNjJlEge2MKI7Mcm1m/lfrphe+ZlQSYpcscVHYSoJxmR2NxkIzRnKiQXKtLC7EjaimjK06ZRsCN7yavQqlU9y3cXlfp1HkcRTuAUzsGDS6jDLTSgCQyG8Ayv8OZI58V5dz4WrQUnzmGP3I+fwALsI2g</latexit><latexit sha1_base64="sAe26MpFncoK5AcSpzUnkA9I=">AB6nicbZBNS8NAEIYn9avWr6pHL4tF8FSIuix6MVjRfsBbSib7aZdutmE3YlSQn+CFw+KePUXefPfuG1z0NYXFh7emWFn3iCRwqDrfjuFtfWNza3idmlnd2/oHx41DJxqhlvsljGuhNQw6VQvIkCJe8kmtMokLwdjG9m9fYj10bE6gEnCfcjOlQiFIyite6f+rV+ueJW3bnIKng5VCBXo1/+6g1ilkZcIZPUmK7nJuhnVKNgk9LvdTwhLIxHfKuRUjbvxsvuqUnFlnQMJY26eQzN3fExmNjJlEge2MKI7Mcm1m/lfrphe+ZlQSYpcscVHYSoJxmR2NxkIzRnKiQXKtLC7EjaimjK06ZRsCN7yavQqlU9y3cXlfp1HkcRTuAUzsGDS6jDLTSgCQyG8Ayv8OZI58V5dz4WrQUnzmGP3I+fwALsI2g</latexit>

. . .

ResNet-34

7x7 AvgPool stride=1

wm

<latexit sha1_base64="3SltFZgdSbEcduFdJMJ4sVJM+s=">AB6nicbZBNSwMxEIZn61etX1WPXoJF8FR2RdBj0YvHirYW2qVk07QNTbJLMquUpT/BiwdFvPqLvPlvTNs9aOsLgYd3ZsjMGyVSWPT9b6+wsrq2vlHcLG1t7+zulfcPmjZODeMNFsvYtCJquRSaN1Cg5K3EcKoiyR+i0fW0/vDIjRWxvsdxwkNFB1r0BaPorLunruqWK37Vn4ksQ5BDBXLVu+WvTi9mqeIamaTWtgM/wTCjBgWTfFLqpJYnlI3ogLcdaq4DbPZqhNy4pwe6cfGPY1k5v6eyKiydqwi16koDu1ibWr+V2un2L8M6GTFLlm84/6qSQYk+ndpCcMZyjHDigzwu1K2JAaytClU3IhBIsnL0PzrBo4vj2v1K7yOIpwBMdwCgFcQA1uoA4NYDCAZ3iFN096L9679zFvLXj5zCH8kf5A2Ucjds=</latexit><latexit sha1_base64="3SltFZgdSbEcduFdJMJ4sVJM+s=">AB6nicbZBNSwMxEIZn61etX1WPXoJF8FR2RdBj0YvHirYW2qVk07QNTbJLMquUpT/BiwdFvPqLvPlvTNs9aOsLgYd3ZsjMGyVSWPT9b6+wsrq2vlHcLG1t7+zulfcPmjZODeMNFsvYtCJquRSaN1Cg5K3EcKoiyR+i0fW0/vDIjRWxvsdxwkNFB1r0BaPorLunruqWK37Vn4ksQ5BDBXLVu+WvTi9mqeIamaTWtgM/wTCjBgWTfFLqpJYnlI3ogLcdaq4DbPZqhNy4pwe6cfGPY1k5v6eyKiydqwi16koDu1ibWr+V2un2L8M6GTFLlm84/6qSQYk+ndpCcMZyjHDigzwu1K2JAaytClU3IhBIsnL0PzrBo4vj2v1K7yOIpwBMdwCgFcQA1uoA4NYDCAZ3iFN096L9679zFvLXj5zCH8kf5A2Ucjds=</latexit><latexit sha1_base64="3SltFZgdSbEcduFdJMJ4sVJM+s=">AB6nicbZBNSwMxEIZn61etX1WPXoJF8FR2RdBj0YvHirYW2qVk07QNTbJLMquUpT/BiwdFvPqLvPlvTNs9aOsLgYd3ZsjMGyVSWPT9b6+wsrq2vlHcLG1t7+zulfcPmjZODeMNFsvYtCJquRSaN1Cg5K3EcKoiyR+i0fW0/vDIjRWxvsdxwkNFB1r0BaPorLunruqWK37Vn4ksQ5BDBXLVu+WvTi9mqeIamaTWtgM/wTCjBgWTfFLqpJYnlI3ogLcdaq4DbPZqhNy4pwe6cfGPY1k5v6eyKiydqwi16koDu1ibWr+V2un2L8M6GTFLlm84/6qSQYk+ndpCcMZyjHDigzwu1K2JAaytClU3IhBIsnL0PzrBo4vj2v1K7yOIpwBMdwCgFcQA1uoA4NYDCAZ3iFN096L9679zFvLXj5zCH8kf5A2Ucjds=</latexit><latexit sha1_base64="3SltFZgdSbEcduFdJMJ4sVJM+s=">AB6nicbZBNSwMxEIZn61etX1WPXoJF8FR2RdBj0YvHirYW2qVk07QNTbJLMquUpT/BiwdFvPqLvPlvTNs9aOsLgYd3ZsjMGyVSWPT9b6+wsrq2vlHcLG1t7+zulfcPmjZODeMNFsvYtCJquRSaN1Cg5K3EcKoiyR+i0fW0/vDIjRWxvsdxwkNFB1r0BaPorLunruqWK37Vn4ksQ5BDBXLVu+WvTi9mqeIamaTWtgM/wTCjBgWTfFLqpJYnlI3ogLcdaq4DbPZqhNy4pwe6cfGPY1k5v6eyKiydqwi16koDu1ibWr+V2un2L8M6GTFLlm84/6qSQYk+ndpCcMZyjHDigzwu1K2JAaytClU3IhBIsnL0PzrBo4vj2v1K7yOIpwBMdwCgFcQA1uoA4NYDCAZ3iFN096L9679zFvLXj5zCH8kf5A2Ucjds=</latexit>

{

<latexit sha1_base64="tFROaEGhmnMpUlcEPFjR2z/BNI=">AB6XicbZBNS8NAEIYn9avWr6hHL4tF8FQSEfRY9OKxiv2ANpTNdtIu3WzC7kYof/AiwdFvPqPvPlv3LY5aOsLCw/vzLAzb5gKro3nfTultfWNza3ydmVnd2/wD08aukUwybLBGJ6oRUo+ASm4YbgZ1UIY1Dge1wfDurt59QaZ7IRzNJMYjpUPKIM2qs9dDL+27Vq3lzkVXwC6hCoUbf/eoNEpbFKA0TVOu76UmyKkynAmcVnqZxpSyMR1i16KkMeogn286JWfWGZAoUfZJQ+bu74mcxlpP4tB2xtSM9HJtZv5X62Ymug5yLtPMoGSLj6JMEJOQ2dlkwBUyIyYWKFPc7krYiCrKjA2nYkPwl09ehdZFzbd8f1mt3xRxlOETuEcfLiCOtxBA5rAIJneIU3Z+y8O/Ox6K15BQzx/BHzucPm4aNZQ=</latexit><latexit sha1_base64="tFROaEGhmnMpUlcEPFjR2z/BNI=">AB6XicbZBNS8NAEIYn9avWr6hHL4tF8FQSEfRY9OKxiv2ANpTNdtIu3WzC7kYof/AiwdFvPqPvPlv3LY5aOsLCw/vzLAzb5gKro3nfTultfWNza3ydmVnd2/wD08aukUwybLBGJ6oRUo+ASm4YbgZ1UIY1Dge1wfDurt59QaZ7IRzNJMYjpUPKIM2qs9dDL+27Vq3lzkVXwC6hCoUbf/eoNEpbFKA0TVOu76UmyKkynAmcVnqZxpSyMR1i16KkMeogn286JWfWGZAoUfZJQ+bu74mcxlpP4tB2xtSM9HJtZv5X62Ymug5yLtPMoGSLj6JMEJOQ2dlkwBUyIyYWKFPc7krYiCrKjA2nYkPwl09ehdZFzbd8f1mt3xRxlOETuEcfLiCOtxBA5rAIJneIU3Z+y8O/Ox6K15BQzx/BHzucPm4aNZQ=</latexit><latexit sha1_base64="tFROaEGhmnMpUlcEPFjR2z/BNI=">AB6XicbZBNS8NAEIYn9avWr6hHL4tF8FQSEfRY9OKxiv2ANpTNdtIu3WzC7kYof/AiwdFvPqPvPlv3LY5aOsLCw/vzLAzb5gKro3nfTultfWNza3ydmVnd2/wD08aukUwybLBGJ6oRUo+ASm4YbgZ1UIY1Dge1wfDurt59QaZ7IRzNJMYjpUPKIM2qs9dDL+27Vq3lzkVXwC6hCoUbf/eoNEpbFKA0TVOu76UmyKkynAmcVnqZxpSyMR1i16KkMeogn286JWfWGZAoUfZJQ+bu74mcxlpP4tB2xtSM9HJtZv5X62Ymug5yLtPMoGSLj6JMEJOQ2dlkwBUyIyYWKFPc7krYiCrKjA2nYkPwl09ehdZFzbd8f1mt3xRxlOETuEcfLiCOtxBA5rAIJneIU3Z+y8O/Ox6K15BQzx/BHzucPm4aNZQ=</latexit><latexit sha1_base64="tFROaEGhmnMpUlcEPFjR2z/BNI=">AB6XicbZBNS8NAEIYn9avWr6hHL4tF8FQSEfRY9OKxiv2ANpTNdtIu3WzC7kYof/AiwdFvPqPvPlv3LY5aOsLCw/vzLAzb5gKro3nfTultfWNza3ydmVnd2/wD08aukUwybLBGJ6oRUo+ASm4YbgZ1UIY1Dge1wfDurt59QaZ7IRzNJMYjpUPKIM2qs9dDL+27Vq3lzkVXwC6hCoUbf/eoNEpbFKA0TVOu76UmyKkynAmcVnqZxpSyMR1i16KkMeogn286JWfWGZAoUfZJQ+bu74mcxlpP4tB2xtSM9HJtZv5X62Ymug5yLtPMoGSLj6JMEJOQ2dlkwBUyIyYWKFPc7krYiCrKjA2nYkPwl09ehdZFzbd8f1mt3xRxlOETuEcfLiCOtxBA5rAIJneIU3Z+y8O/Ox6K15BQzx/BHzucPm4aNZQ=</latexit>

K − 1

<latexit sha1_base64="r9EyNDc4ZkK0RbMq+OXc0Let6lU=">AB6nicbZBNS8NAEIYn9avWr6pHL4tF8GJRNBj0YvgpaL9gDaUzXbSLt1swu5GKE/wYsHRbz6i7z5b9y2OWjrCwsP78ywM2+QCK6N6347hZXVtfWN4mZpa3tnd6+8f9DUcaoYNlgsYtUOqEbBJTYMNwLbiUIaBQJbwehmWm89odI8lo9mnKAf0YHkIWfUWOvh7szrlStu1Z2JLIOXQwVy1Xvlr24/ZmE0jBte54bmL8jCrDmcBJqZtqTCgb0QF2LEoaofaz2aoTcmKdPgljZ80ZOb+nshopPU4CmxnRM1QL9am5n+1TmrCKz/jMkNSjb/KEwFMTGZ3k36XCEzYmyBMsXtroQNqaLM2HRKNgRv8eRlaJ5XPcv3F5XadR5HEY7gGE7Bg0uowS3UoQEMBvAMr/DmCOfFeXc+5q0FJ585hD9yPn8AexuNQ=</latexit><latexit sha1_base64="r9EyNDc4ZkK0RbMq+OXc0Let6lU=">AB6nicbZBNS8NAEIYn9avWr6pHL4tF8GJRNBj0YvgpaL9gDaUzXbSLt1swu5GKE/wYsHRbz6i7z5b9y2OWjrCwsP78ywM2+QCK6N6347hZXVtfWN4mZpa3tnd6+8f9DUcaoYNlgsYtUOqEbBJTYMNwLbiUIaBQJbwehmWm89odI8lo9mnKAf0YHkIWfUWOvh7szrlStu1Z2JLIOXQwVy1Xvlr24/ZmE0jBte54bmL8jCrDmcBJqZtqTCgb0QF2LEoaofaz2aoTcmKdPgljZ80ZOb+nshopPU4CmxnRM1QL9am5n+1TmrCKz/jMkNSjb/KEwFMTGZ3k36XCEzYmyBMsXtroQNqaLM2HRKNgRv8eRlaJ5XPcv3F5XadR5HEY7gGE7Bg0uowS3UoQEMBvAMr/DmCOfFeXc+5q0FJ585hD9yPn8AexuNQ=</latexit><latexit sha1_base64="r9EyNDc4ZkK0RbMq+OXc0Let6lU=">AB6nicbZBNS8NAEIYn9avWr6pHL4tF8GJRNBj0YvgpaL9gDaUzXbSLt1swu5GKE/wYsHRbz6i7z5b9y2OWjrCwsP78ywM2+QCK6N6347hZXVtfWN4mZpa3tnd6+8f9DUcaoYNlgsYtUOqEbBJTYMNwLbiUIaBQJbwehmWm89odI8lo9mnKAf0YHkIWfUWOvh7szrlStu1Z2JLIOXQwVy1Xvlr24/ZmE0jBte54bmL8jCrDmcBJqZtqTCgb0QF2LEoaofaz2aoTcmKdPgljZ80ZOb+nshopPU4CmxnRM1QL9am5n+1TmrCKz/jMkNSjb/KEwFMTGZ3k36XCEzYmyBMsXtroQNqaLM2HRKNgRv8eRlaJ5XPcv3F5XadR5HEY7gGE7Bg0uowS3UoQEMBvAMr/DmCOfFeXc+5q0FJ585hD9yPn8AexuNQ=</latexit><latexit sha1_base64="r9EyNDc4ZkK0RbMq+OXc0Let6lU=">AB6nicbZBNS8NAEIYn9avWr6pHL4tF8GJRNBj0YvgpaL9gDaUzXbSLt1swu5GKE/wYsHRbz6i7z5b9y2OWjrCwsP78ywM2+QCK6N6347hZXVtfWN4mZpa3tnd6+8f9DUcaoYNlgsYtUOqEbBJTYMNwLbiUIaBQJbwehmWm89odI8lo9mnKAf0YHkIWfUWOvh7szrlStu1Z2JLIOXQwVy1Xvlr24/ZmE0jBte54bmL8jCrDmcBJqZtqTCgb0QF2LEoaofaz2aoTcmKdPgljZ80ZOb+nshopPU4CmxnRM1QL9am5n+1TmrCKz/jMkNSjb/KEwFMTGZ3k36XCEzYmyBMsXtroQNqaLM2HRKNgRv8eRlaJ5XPcv3F5XadR5HEY7gGE7Bg0uowS3UoQEMBvAMr/DmCOfFeXc+5q0FJ585hD9yPn8AexuNQ=</latexit>

bK−1

<latexit sha1_base64="LwOEvZG6tsrLM9p/zFnxW8ucU=">AB7nicbZBNS8NAEIYn9avWr6pHL4tF8GJpKDHohfBSwX7AW0om+2mXbrZhN2JUEJ/hBcPinj193jz37htc9DWFxYe3plhZ94gkcKg6347hbX1jc2t4nZpZ3dv/6B8eNQycaoZb7JYxroTUMOlULyJAiXvJrTKJC8HYxvZ/X2E9dGxOoRJwn3IzpUIhSMorXaQT+7v/Cm/XLFrbpzkVXwcqhArka/NUbxCyNuEImqTFdz03Qz6hGwSflnqp4QlYzrkXYuKRtz42XzdKTmzoCEsbZPIZm7vycyGhkziQLbGVEcmeXazPyv1k0xvPYzoZIUuWKLj8JUEozJ7HYyEJozlBMLlGlhdyVsRDVlaBMq2RC85ZNXoXVZ9Sw/1Cr1mzyOIpzAKZyDB1dQhztoQBMYjOEZXuHNSZwX5935WLQWnHzmGP7I+fwBshSPIg=</latexit><latexit sha1_base64="LwOEvZG6tsrLM9p/zFnxW8ucU=">AB7nicbZBNS8NAEIYn9avWr6pHL4tF8GJpKDHohfBSwX7AW0om+2mXbrZhN2JUEJ/hBcPinj193jz37htc9DWFxYe3plhZ94gkcKg6347hbX1jc2t4nZpZ3dv/6B8eNQycaoZb7JYxroTUMOlULyJAiXvJrTKJC8HYxvZ/X2E9dGxOoRJwn3IzpUIhSMorXaQT+7v/Cm/XLFrbpzkVXwcqhArka/NUbxCyNuEImqTFdz03Qz6hGwSflnqp4QlYzrkXYuKRtz42XzdKTmzoCEsbZPIZm7vycyGhkziQLbGVEcmeXazPyv1k0xvPYzoZIUuWKLj8JUEozJ7HYyEJozlBMLlGlhdyVsRDVlaBMq2RC85ZNXoXVZ9Sw/1Cr1mzyOIpzAKZyDB1dQhztoQBMYjOEZXuHNSZwX5935WLQWnHzmGP7I+fwBshSPIg=</latexit><latexit sha1_base64="LwOEvZG6tsrLM9p/zFnxW8ucU=">AB7nicbZBNS8NAEIYn9avWr6pHL4tF8GJpKDHohfBSwX7AW0om+2mXbrZhN2JUEJ/hBcPinj193jz37htc9DWFxYe3plhZ94gkcKg6347hbX1jc2t4nZpZ3dv/6B8eNQycaoZb7JYxroTUMOlULyJAiXvJrTKJC8HYxvZ/X2E9dGxOoRJwn3IzpUIhSMorXaQT+7v/Cm/XLFrbpzkVXwcqhArka/NUbxCyNuEImqTFdz03Qz6hGwSflnqp4QlYzrkXYuKRtz42XzdKTmzoCEsbZPIZm7vycyGhkziQLbGVEcmeXazPyv1k0xvPYzoZIUuWKLj8JUEozJ7HYyEJozlBMLlGlhdyVsRDVlaBMq2RC85ZNXoXVZ9Sw/1Cr1mzyOIpzAKZyDB1dQhztoQBMYjOEZXuHNSZwX5935WLQWnHzmGP7I+fwBshSPIg=</latexit><latexit sha1_base64="LwOEvZG6tsrLM9p/zFnxW8ucU=">AB7nicbZBNS8NAEIYn9avWr6pHL4tF8GJpKDHohfBSwX7AW0om+2mXbrZhN2JUEJ/hBcPinj193jz37htc9DWFxYe3plhZ94gkcKg6347hbX1jc2t4nZpZ3dv/6B8eNQycaoZb7JYxroTUMOlULyJAiXvJrTKJC8HYxvZ/X2E9dGxOoRJwn3IzpUIhSMorXaQT+7v/Cm/XLFrbpzkVXwcqhArka/NUbxCyNuEImqTFdz03Qz6hGwSflnqp4QlYzrkXYuKRtz42XzdKTmzoCEsbZPIZm7vycyGhkziQLbGVEcmeXazPyv1k0xvPYzoZIUuWKLj8JUEozJ7HYyEJozlBMLlGlhdyVsRDVlaBMq2RC85ZNXoXVZ9Sw/1Cr1mzyOIpzAKZyDB1dQhztoQBMYjOEZXuHNSZwX5935WLQWnHzmGP7I+fwBshSPIg=</latexit>

b P(yi > r1)

<latexit sha1_base64="JEWZgW/c9FbzIeNeK3w5m6yuKL4=">AB/XicbZDLSsNAFIYn9VbrLV52boJFqJuSiKArKbpxWcFeoA1hMjlph04mYWaixFB8FTcuFHre7jzbZy2WjrDwMf/zmHc+b3E0alsu1vo7S0vLK6Vl6vbGxube+Yu3tGaeCQIvELBZdH0tglENLUcWgmwjAkc+g4+uJ/XOPQhJY36nsgTcCA84DSnBSluedB/oAEMscqb41rm0UvhOSeWbXr9lTWIjgFVFGhpmd+9YOYpBFwRiWsufYiXJzLBQlDMaVfiohwWSEB9DTyHE0s2n14+tY+0EVhgL/biypu7viRxHUmaRrzsjrIZyvjYx/6v1UhVeuDnlSaqAk9miMGWiq1JFZABRDFMg2YCKpvtcgQC0yUDqyiQ3Dmv7wI7dO6o/n2rNq4KuIo0N0hGrIQeogW5QE7UQY/oGb2iN+PJeDHejY9Za8koZvbRHxmfP9pzlM8=</latexit><latexit sha1_base64="JEWZgW/c9FbzIeNeK3w5m6yuKL4=">AB/XicbZDLSsNAFIYn9VbrLV52boJFqJuSiKArKbpxWcFeoA1hMjlph04mYWaixFB8FTcuFHre7jzbZy2WjrDwMf/zmHc+b3E0alsu1vo7S0vLK6Vl6vbGxube+Yu3tGaeCQIvELBZdH0tglENLUcWgmwjAkc+g4+uJ/XOPQhJY36nsgTcCA84DSnBSluedB/oAEMscqb41rm0UvhOSeWbXr9lTWIjgFVFGhpmd+9YOYpBFwRiWsufYiXJzLBQlDMaVfiohwWSEB9DTyHE0s2n14+tY+0EVhgL/biypu7viRxHUmaRrzsjrIZyvjYx/6v1UhVeuDnlSaqAk9miMGWiq1JFZABRDFMg2YCKpvtcgQC0yUDqyiQ3Dmv7wI7dO6o/n2rNq4KuIo0N0hGrIQeogW5QE7UQY/oGb2iN+PJeDHejY9Za8koZvbRHxmfP9pzlM8=</latexit><latexit sha1_base64="JEWZgW/c9FbzIeNeK3w5m6yuKL4=">AB/XicbZDLSsNAFIYn9VbrLV52boJFqJuSiKArKbpxWcFeoA1hMjlph04mYWaixFB8FTcuFHre7jzbZy2WjrDwMf/zmHc+b3E0alsu1vo7S0vLK6Vl6vbGxube+Yu3tGaeCQIvELBZdH0tglENLUcWgmwjAkc+g4+uJ/XOPQhJY36nsgTcCA84DSnBSluedB/oAEMscqb41rm0UvhOSeWbXr9lTWIjgFVFGhpmd+9YOYpBFwRiWsufYiXJzLBQlDMaVfiohwWSEB9DTyHE0s2n14+tY+0EVhgL/biypu7viRxHUmaRrzsjrIZyvjYx/6v1UhVeuDnlSaqAk9miMGWiq1JFZABRDFMg2YCKpvtcgQC0yUDqyiQ3Dmv7wI7dO6o/n2rNq4KuIo0N0hGrIQeogW5QE7UQY/oGb2iN+PJeDHejY9Za8koZvbRHxmfP9pzlM8=</latexit><latexit sha1_base64="JEWZgW/c9FbzIeNeK3w5m6yuKL4=">AB/XicbZDLSsNAFIYn9VbrLV52boJFqJuSiKArKbpxWcFeoA1hMjlph04mYWaixFB8FTcuFHre7jzbZy2WjrDwMf/zmHc+b3E0alsu1vo7S0vLK6Vl6vbGxube+Yu3tGaeCQIvELBZdH0tglENLUcWgmwjAkc+g4+uJ/XOPQhJY36nsgTcCA84DSnBSluedB/oAEMscqb41rm0UvhOSeWbXr9lTWIjgFVFGhpmd+9YOYpBFwRiWsufYiXJzLBQlDMaVfiohwWSEB9DTyHE0s2n14+tY+0EVhgL/biypu7viRxHUmaRrzsjrIZyvjYx/6v1UhVeuDnlSaqAk9miMGWiq1JFZABRDFMg2YCKpvtcgQC0yUDqyiQ3Dmv7wI7dO6o/n2rNq4KuIo0N0hGrIQeogW5QE7UQY/oGb2iN+PJeDHejY9Za8koZvbRHxmfP9pzlM8=</latexit>

b P(yi > r2)

<latexit sha1_base64="TPnmRaLx/eL+eApEw1jpI9oVN4=">AB/XicbZDLSsNAFIZPvNZ6i5edm2AR6qYkRdCVFN24rGAv0IYwmUzaoZNJmJkoMRfxY0LRdz6Hu58G6dtFtr6w8DHf87hnPn9hFGpbPvbWFpeWV1bL2UN7e2d3bNvf2jFOBSQvHLBZdH0nCKCctRUj3UQFPmMdPzR9aTeuSdC0pjfqSwhboQGnIYUI6UtzsP9CADJHKm+Nq5tFL4dVPbNi1+yprEVwCqhAoaZnfvWDGKcR4QozJGXPsRPl5kgoihkZl/upJAnCIzQgPY0cRUS6+fT6sXWincAKY6EfV9bU/T2Ro0jKLPJ1Z4TUM7XJuZ/tV6qwgs3pzxJFeF4tihMmaViaxKFVBsGKZBoQF1bdaeIgEwkoHVtYhOPNfXoR2veZovj2rNK6KOEpwBMdQBQfOoQE30IQWYHiEZ3iFN+PJeDHejY9Z65JRzBzAHxmfP9v4lNA=</latexit><latexit sha1_base64="TPnmRaLx/eL+eApEw1jpI9oVN4=">AB/XicbZDLSsNAFIZPvNZ6i5edm2AR6qYkRdCVFN24rGAv0IYwmUzaoZNJmJkoMRfxY0LRdz6Hu58G6dtFtr6w8DHf87hnPn9hFGpbPvbWFpeWV1bL2UN7e2d3bNvf2jFOBSQvHLBZdH0nCKCctRUj3UQFPmMdPzR9aTeuSdC0pjfqSwhboQGnIYUI6UtzsP9CADJHKm+Nq5tFL4dVPbNi1+yprEVwCqhAoaZnfvWDGKcR4QozJGXPsRPl5kgoihkZl/upJAnCIzQgPY0cRUS6+fT6sXWincAKY6EfV9bU/T2Ro0jKLPJ1Z4TUM7XJuZ/tV6qwgs3pzxJFeF4tihMmaViaxKFVBsGKZBoQF1bdaeIgEwkoHVtYhOPNfXoR2veZovj2rNK6KOEpwBMdQBQfOoQE30IQWYHiEZ3iFN+PJeDHejY9Z65JRzBzAHxmfP9v4lNA=</latexit><latexit sha1_base64="TPnmRaLx/eL+eApEw1jpI9oVN4=">AB/XicbZDLSsNAFIZPvNZ6i5edm2AR6qYkRdCVFN24rGAv0IYwmUzaoZNJmJkoMRfxY0LRdz6Hu58G6dtFtr6w8DHf87hnPn9hFGpbPvbWFpeWV1bL2UN7e2d3bNvf2jFOBSQvHLBZdH0nCKCctRUj3UQFPmMdPzR9aTeuSdC0pjfqSwhboQGnIYUI6UtzsP9CADJHKm+Nq5tFL4dVPbNi1+yprEVwCqhAoaZnfvWDGKcR4QozJGXPsRPl5kgoihkZl/upJAnCIzQgPY0cRUS6+fT6sXWincAKY6EfV9bU/T2Ro0jKLPJ1Z4TUM7XJuZ/tV6qwgs3pzxJFeF4tihMmaViaxKFVBsGKZBoQF1bdaeIgEwkoHVtYhOPNfXoR2veZovj2rNK6KOEpwBMdQBQfOoQE30IQWYHiEZ3iFN+PJeDHejY9Z65JRzBzAHxmfP9v4lNA=</latexit><latexit sha1_base64="TPnmRaLx/eL+eApEw1jpI9oVN4=">AB/XicbZDLSsNAFIZPvNZ6i5edm2AR6qYkRdCVFN24rGAv0IYwmUzaoZNJmJkoMRfxY0LRdz6Hu58G6dtFtr6w8DHf87hnPn9hFGpbPvbWFpeWV1bL2UN7e2d3bNvf2jFOBSQvHLBZdH0nCKCctRUj3UQFPmMdPzR9aTeuSdC0pjfqSwhboQGnIYUI6UtzsP9CADJHKm+Nq5tFL4dVPbNi1+yprEVwCqhAoaZnfvWDGKcR4QozJGXPsRPl5kgoihkZl/upJAnCIzQgPY0cRUS6+fT6sXWincAKY6EfV9bU/T2Ro0jKLPJ1Z4TUM7XJuZ/tV6qwgs3pzxJFeF4tihMmaViaxKFVBsGKZBoQF1bdaeIgEwkoHVtYhOPNfXoR2veZovj2rNK6KOEpwBMdQBQfOoQE30IQWYHiEZ3iFN+PJeDHejY9Z65JRzBzAHxmfP9v4lNA=</latexit>

b P(yi > rK−1)

<latexit sha1_base64="EsrSGPlu5mWGfLIJjs5E47reN0Y=">ACAXicbZDLSsNAFIYn9VbrLepGcDNYhLqwJCLoSopuBDcV7AXaECaTSTt0MgkzEyWEuPFV3LhQxK1v4c63cdpmoa0/DHz85xzOnN+LGZXKsr6N0sLi0vJKebWytr6xuWVu7RlAhMWjhikeh6SBJGOWkpqhjpxoKg0GOk42uxvXOPRGSRvxOpTFxQjTgNKAYKW25l7/gfpkiFTWzGupSy+Em90c2/mRa1atujURnAe7gCo1HTNr74f4SQkXGpOzZVqycDAlFMSN5pZ9IEiM8QgPS08hRSKSTS7I4aF2fBhEQj+u4MT9PZGhUMo09HRniNRQztbG5n+1XqKCcyejPE4U4Xi6KEgYVBEcxwF9KghWLNWAsKD6rxAPkUBY6dAqOgR79uR5aJ/Ubc23p9XGZRFHGeyDA1ADNjgDXANmqAFMHgEz+AVvBlPxovxbnxMW0tGMbML/sj4/AG94JZn</latexit><latexit sha1_base64="EsrSGPlu5mWGfLIJjs5E47reN0Y=">ACAXicbZDLSsNAFIYn9VbrLepGcDNYhLqwJCLoSopuBDcV7AXaECaTSTt0MgkzEyWEuPFV3LhQxK1v4c63cdpmoa0/DHz85xzOnN+LGZXKsr6N0sLi0vJKebWytr6xuWVu7RlAhMWjhikeh6SBJGOWkpqhjpxoKg0GOk42uxvXOPRGSRvxOpTFxQjTgNKAYKW25l7/gfpkiFTWzGupSy+Em90c2/mRa1atujURnAe7gCo1HTNr74f4SQkXGpOzZVqycDAlFMSN5pZ9IEiM8QgPS08hRSKSTS7I4aF2fBhEQj+u4MT9PZGhUMo09HRniNRQztbG5n+1XqKCcyejPE4U4Xi6KEgYVBEcxwF9KghWLNWAsKD6rxAPkUBY6dAqOgR79uR5aJ/Ubc23p9XGZRFHGeyDA1ADNjgDXANmqAFMHgEz+AVvBlPxovxbnxMW0tGMbML/sj4/AG94JZn</latexit><latexit sha1_base64="EsrSGPlu5mWGfLIJjs5E47reN0Y=">ACAXicbZDLSsNAFIYn9VbrLepGcDNYhLqwJCLoSopuBDcV7AXaECaTSTt0MgkzEyWEuPFV3LhQxK1v4c63cdpmoa0/DHz85xzOnN+LGZXKsr6N0sLi0vJKebWytr6xuWVu7RlAhMWjhikeh6SBJGOWkpqhjpxoKg0GOk42uxvXOPRGSRvxOpTFxQjTgNKAYKW25l7/gfpkiFTWzGupSy+Em90c2/mRa1atujURnAe7gCo1HTNr74f4SQkXGpOzZVqycDAlFMSN5pZ9IEiM8QgPS08hRSKSTS7I4aF2fBhEQj+u4MT9PZGhUMo09HRniNRQztbG5n+1XqKCcyejPE4U4Xi6KEgYVBEcxwF9KghWLNWAsKD6rxAPkUBY6dAqOgR79uR5aJ/Ubc23p9XGZRFHGeyDA1ADNjgDXANmqAFMHgEz+AVvBlPxovxbnxMW0tGMbML/sj4/AG94JZn</latexit><latexit sha1_base64="EsrSGPlu5mWGfLIJjs5E47reN0Y=">ACAXicbZDLSsNAFIYn9VbrLepGcDNYhLqwJCLoSopuBDcV7AXaECaTSTt0MgkzEyWEuPFV3LhQxK1v4c63cdpmoa0/DHz85xzOnN+LGZXKsr6N0sLi0vJKebWytr6xuWVu7RlAhMWjhikeh6SBJGOWkpqhjpxoKg0GOk42uxvXOPRGSRvxOpTFxQjTgNKAYKW25l7/gfpkiFTWzGupSy+Em90c2/mRa1atujURnAe7gCo1HTNr74f4SQkXGpOzZVqycDAlFMSN5pZ9IEiM8QgPS08hRSKSTS7I4aF2fBhEQj+u4MT9PZGhUMo09HRniNRQztbG5n+1XqKCcyejPE4U4Xi6KEgYVBEcxwF9KghWLNWAsKD6rxAPkUBY6dAqOgR79uR5aJ/Ubc23p9XGZRFHGeyDA1ADNjgDXANmqAFMHgEz+AVvBlPxovxbnxMW0tGMbML/sj4/AG94JZn</latexit>

{

<latexit sha1_base64="tFROaEGhmnMpUlcEPFjR2z/BNI=">AB6XicbZBNS8NAEIYn9avWr6hHL4tF8FQSEfRY9OKxiv2ANpTNdtIu3WzC7kYof/AiwdFvPqPvPlv3LY5aOsLCw/vzLAzb5gKro3nfTultfWNza3ydmVnd2/wD08aukUwybLBGJ6oRUo+ASm4YbgZ1UIY1Dge1wfDurt59QaZ7IRzNJMYjpUPKIM2qs9dDL+27Vq3lzkVXwC6hCoUbf/eoNEpbFKA0TVOu76UmyKkynAmcVnqZxpSyMR1i16KkMeogn286JWfWGZAoUfZJQ+bu74mcxlpP4tB2xtSM9HJtZv5X62Ymug5yLtPMoGSLj6JMEJOQ2dlkwBUyIyYWKFPc7krYiCrKjA2nYkPwl09ehdZFzbd8f1mt3xRxlOETuEcfLiCOtxBA5rAIJneIU3Z+y8O/Ox6K15BQzx/BHzucPm4aNZQ=</latexit><latexit sha1_base64="tFROaEGhmnMpUlcEPFjR2z/BNI=">AB6XicbZBNS8NAEIYn9avWr6hHL4tF8FQSEfRY9OKxiv2ANpTNdtIu3WzC7kYof/AiwdFvPqPvPlv3LY5aOsLCw/vzLAzb5gKro3nfTultfWNza3ydmVnd2/wD08aukUwybLBGJ6oRUo+ASm4YbgZ1UIY1Dge1wfDurt59QaZ7IRzNJMYjpUPKIM2qs9dDL+27Vq3lzkVXwC6hCoUbf/eoNEpbFKA0TVOu76UmyKkynAmcVnqZxpSyMR1i16KkMeogn286JWfWGZAoUfZJQ+bu74mcxlpP4tB2xtSM9HJtZv5X62Ymug5yLtPMoGSLj6JMEJOQ2dlkwBUyIyYWKFPc7krYiCrKjA2nYkPwl09ehdZFzbd8f1mt3xRxlOETuEcfLiCOtxBA5rAIJneIU3Z+y8O/Ox6K15BQzx/BHzucPm4aNZQ=</latexit><latexit sha1_base64="tFROaEGhmnMpUlcEPFjR2z/BNI=">AB6XicbZBNS8NAEIYn9avWr6hHL4tF8FQSEfRY9OKxiv2ANpTNdtIu3WzC7kYof/AiwdFvPqPvPlv3LY5aOsLCw/vzLAzb5gKro3nfTultfWNza3ydmVnd2/wD08aukUwybLBGJ6oRUo+ASm4YbgZ1UIY1Dge1wfDurt59QaZ7IRzNJMYjpUPKIM2qs9dDL+27Vq3lzkVXwC6hCoUbf/eoNEpbFKA0TVOu76UmyKkynAmcVnqZxpSyMR1i16KkMeogn286JWfWGZAoUfZJQ+bu74mcxlpP4tB2xtSM9HJtZv5X62Ymug5yLtPMoGSLj6JMEJOQ2dlkwBUyIyYWKFPc7krYiCrKjA2nYkPwl09ehdZFzbd8f1mt3xRxlOETuEcfLiCOtxBA5rAIJneIU3Z+y8O/Ox6K15BQzx/BHzucPm4aNZQ=</latexit><latexit sha1_base64="tFROaEGhmnMpUlcEPFjR2z/BNI=">AB6XicbZBNS8NAEIYn9avWr6hHL4tF8FQSEfRY9OKxiv2ANpTNdtIu3WzC7kYof/AiwdFvPqPvPlv3LY5aOsLCw/vzLAzb5gKro3nfTultfWNza3ydmVnd2/wD08aukUwybLBGJ6oRUo+ASm4YbgZ1UIY1Dge1wfDurt59QaZ7IRzNJMYjpUPKIM2qs9dDL+27Vq3lzkVXwC6hCoUbf/eoNEpbFKA0TVOu76UmyKkynAmcVnqZxpSyMR1i16KkMeogn286JWfWGZAoUfZJQ+bu74mcxlpP4tB2xtSM9HJtZv5X62Ymug5yLtPMoGSLj6JMEJOQ2dlkwBUyIyYWKFPc7krYiCrKjA2nYkPwl09ehdZFzbd8f1mt3xRxlOETuEcfLiCOtxBA5rAIJneIU3Z+y8O/Ox6K15BQzx/BHzucPm4aNZQ=</latexit>

Tasks

{

<latexit sha1_base64="tFROaEGhmnMpUlcEPFjR2z/BNI=">AB6XicbZBNS8NAEIYn9avWr6hHL4tF8FQSEfRY9OKxiv2ANpTNdtIu3WzC7kYof/AiwdFvPqPvPlv3LY5aOsLCw/vzLAzb5gKro3nfTultfWNza3ydmVnd2/wD08aukUwybLBGJ6oRUo+ASm4YbgZ1UIY1Dge1wfDurt59QaZ7IRzNJMYjpUPKIM2qs9dDL+27Vq3lzkVXwC6hCoUbf/eoNEpbFKA0TVOu76UmyKkynAmcVnqZxpSyMR1i16KkMeogn286JWfWGZAoUfZJQ+bu74mcxlpP4tB2xtSM9HJtZv5X62Ymug5yLtPMoGSLj6JMEJOQ2dlkwBUyIyYWKFPc7krYiCrKjA2nYkPwl09ehdZFzbd8f1mt3xRxlOETuEcfLiCOtxBA5rAIJneIU3Z+y8O/Ox6K15BQzx/BHzucPm4aNZQ=</latexit><latexit sha1_base64="tFROaEGhmnMpUlcEPFjR2z/BNI=">AB6XicbZBNS8NAEIYn9avWr6hHL4tF8FQSEfRY9OKxiv2ANpTNdtIu3WzC7kYof/AiwdFvPqPvPlv3LY5aOsLCw/vzLAzb5gKro3nfTultfWNza3ydmVnd2/wD08aukUwybLBGJ6oRUo+ASm4YbgZ1UIY1Dge1wfDurt59QaZ7IRzNJMYjpUPKIM2qs9dDL+27Vq3lzkVXwC6hCoUbf/eoNEpbFKA0TVOu76UmyKkynAmcVnqZxpSyMR1i16KkMeogn286JWfWGZAoUfZJQ+bu74mcxlpP4tB2xtSM9HJtZv5X62Ymug5yLtPMoGSLj6JMEJOQ2dlkwBUyIyYWKFPc7krYiCrKjA2nYkPwl09ehdZFzbd8f1mt3xRxlOETuEcfLiCOtxBA5rAIJneIU3Z+y8O/Ox6K15BQzx/BHzucPm4aNZQ=</latexit><latexit sha1_base64="tFROaEGhmnMpUlcEPFjR2z/BNI=">AB6XicbZBNS8NAEIYn9avWr6hHL4tF8FQSEfRY9OKxiv2ANpTNdtIu3WzC7kYof/AiwdFvPqPvPlv3LY5aOsLCw/vzLAzb5gKro3nfTultfWNza3ydmVnd2/wD08aukUwybLBGJ6oRUo+ASm4YbgZ1UIY1Dge1wfDurt59QaZ7IRzNJMYjpUPKIM2qs9dDL+27Vq3lzkVXwC6hCoUbf/eoNEpbFKA0TVOu76UmyKkynAmcVnqZxpSyMR1i16KkMeogn286JWfWGZAoUfZJQ+bu74mcxlpP4tB2xtSM9HJtZv5X62Ymug5yLtPMoGSLj6JMEJOQ2dlkwBUyIyYWKFPc7krYiCrKjA2nYkPwl09ehdZFzbd8f1mt3xRxlOETuEcfLiCOtxBA5rAIJneIU3Z+y8O/Ox6K15BQzx/BHzucPm4aNZQ=</latexit><latexit sha1_base64="tFROaEGhmnMpUlcEPFjR2z/BNI=">AB6XicbZBNS8NAEIYn9avWr6hHL4tF8FQSEfRY9OKxiv2ANpTNdtIu3WzC7kYof/AiwdFvPqPvPlv3LY5aOsLCw/vzLAzb5gKro3nfTultfWNza3ydmVnd2/wD08aukUwybLBGJ6oRUo+ASm4YbgZ1UIY1Dge1wfDurt59QaZ7IRzNJMYjpUPKIM2qs9dDL+27Vq3lzkVXwC6hCoUbf/eoNEpbFKA0TVOu76UmyKkynAmcVnqZxpSyMR1i16KkMeogn286JWfWGZAoUfZJQ+bu74mcxlpP4tB2xtSM9HJtZv5X62Ymug5yLtPMoGSLj6JMEJOQ2dlkwBUyIyYWKFPc7krYiCrKjA2nYkPwl09ehdZFzbd8f1mt3xRxlOETuEcfLiCOtxBA5rAIJneIU3Z+y8O/Ox6K15BQzx/BHzucPm4aNZQ=</latexit>

a1

<latexit sha1_base64="i5nKWL2osYHJ2SU7NvALg/zpMfE=">AB6nicbZBNS8NAEIYn9avWr6hHL4tF8FQSEfRY9OKxov2ANpTJdtMu3WzC7kYoT/BiwdFvPqLvPlv3LY5aOsLCw/vzLAzb5gKro3nfTultfWNza3ydmVnd2/wD08aukU5Q1aSIS1QlRM8ElaxpuBOukimEcCtYOx7ezevuJKc0T+WgmKQtiHEoecYrGWg/Y9/tu1at5c5FV8AuoQqFG3/3qDRKaxUwaKlDru+lJshRGU4Fm1Z6mWYp0jEOWdeixJjpIJ+vOiVn1hmQKFH2SUPm7u+JHGOtJ3FoO2M0I71cm5n/1bqZia6DnMs0M0zSxUdRJohJyOxuMuCKUSMmFpAqbncldIQKqbHpVGwI/vLJq9C6qPmW7y+r9ZsijKcwCmcgw9XUIc7aEATKAzhGV7hzRHOi/PufCxaS04xcwx/5Hz+AOiZjYk=</latexit><latexit sha1_base64="i5nKWL2osYHJ2SU7NvALg/zpMfE=">AB6nicbZBNS8NAEIYn9avWr6hHL4tF8FQSEfRY9OKxov2ANpTJdtMu3WzC7kYoT/BiwdFvPqLvPlv3LY5aOsLCw/vzLAzb5gKro3nfTultfWNza3ydmVnd2/wD08aukU5Q1aSIS1QlRM8ElaxpuBOukimEcCtYOx7ezevuJKc0T+WgmKQtiHEoecYrGWg/Y9/tu1at5c5FV8AuoQqFG3/3qDRKaxUwaKlDru+lJshRGU4Fm1Z6mWYp0jEOWdeixJjpIJ+vOiVn1hmQKFH2SUPm7u+JHGOtJ3FoO2M0I71cm5n/1bqZia6DnMs0M0zSxUdRJohJyOxuMuCKUSMmFpAqbncldIQKqbHpVGwI/vLJq9C6qPmW7y+r9ZsijKcwCmcgw9XUIc7aEATKAzhGV7hzRHOi/PufCxaS04xcwx/5Hz+AOiZjYk=</latexit><latexit sha1_base64="i5nKWL2osYHJ2SU7NvALg/zpMfE=">AB6nicbZBNS8NAEIYn9avWr6hHL4tF8FQSEfRY9OKxov2ANpTJdtMu3WzC7kYoT/BiwdFvPqLvPlv3LY5aOsLCw/vzLAzb5gKro3nfTultfWNza3ydmVnd2/wD08aukU5Q1aSIS1QlRM8ElaxpuBOukimEcCtYOx7ezevuJKc0T+WgmKQtiHEoecYrGWg/Y9/tu1at5c5FV8AuoQqFG3/3qDRKaxUwaKlDru+lJshRGU4Fm1Z6mWYp0jEOWdeixJjpIJ+vOiVn1hmQKFH2SUPm7u+JHGOtJ3FoO2M0I71cm5n/1bqZia6DnMs0M0zSxUdRJohJyOxuMuCKUSMmFpAqbncldIQKqbHpVGwI/vLJq9C6qPmW7y+r9ZsijKcwCmcgw9XUIc7aEATKAzhGV7hzRHOi/PufCxaS04xcwx/5Hz+AOiZjYk=</latexit><latexit sha1_base64="i5nKWL2osYHJ2SU7NvALg/zpMfE=">AB6nicbZBNS8NAEIYn9avWr6hHL4tF8FQSEfRY9OKxov2ANpTJdtMu3WzC7kYoT/BiwdFvPqLvPlv3LY5aOsLCw/vzLAzb5gKro3nfTultfWNza3ydmVnd2/wD08aukU5Q1aSIS1QlRM8ElaxpuBOukimEcCtYOx7ezevuJKc0T+WgmKQtiHEoecYrGWg/Y9/tu1at5c5FV8AuoQqFG3/3qDRKaxUwaKlDru+lJshRGU4Fm1Z6mWYp0jEOWdeixJjpIJ+vOiVn1hmQKFH2SUPm7u+JHGOtJ3FoO2M0I71cm5n/1bqZia6DnMs0M0zSxUdRJohJyOxuMuCKUSMmFpAqbncldIQKqbHpVGwI/vLJq9C6qPmW7y+r9ZsijKcwCmcgw9XUIc7aEATKAzhGV7hzRHOi/PufCxaS04xcwx/5Hz+AOiZjYk=</latexit>

a2

<latexit sha1_base64="fHY8L0Lq6jayN2ybi5nRkuRgM8k=">AB6nicbZBNS8NAEIYn9avWr6pHL4tF8FSItRj0YvHivYD2lAm2027dLMJuxuhP4ELx4U8eov8ua/cdvmoK0vLDy8M8POvEiuDau+0UNja3tneKu6W9/YPDo/LxSVvHqaKsRWMRq26AmgkuWctwI1g3UQyjQLBOMLmd1ztPTGkey0czTZgf4UjykFM01nrAQW1QrhVdyGyDl4OFcjVHJS/+sOYphGThgrUue5ifEzVIZTwWalfqpZgnSCI9azKDFi2s8Wq87IhXWGJIyVfdKQhft7IsNI62kU2M4IzViv1ubmf7VeasJrP+MySQ2TdPlRmApiYjK/mwy5YtSIqQWkitdCR2jQmpsOiUbgrd68jq0a1XP8v1VpXGTx1GEMziHS/CgDg24gya0gMInuEV3hzhvDjvzseyteDkM6fwR87nD+odjYo=</latexit><latexit sha1_base64="fHY8L0Lq6jayN2ybi5nRkuRgM8k=">AB6nicbZBNS8NAEIYn9avWr6pHL4tF8FSItRj0YvHivYD2lAm2027dLMJuxuhP4ELx4U8eov8ua/cdvmoK0vLDy8M8POvEiuDau+0UNja3tneKu6W9/YPDo/LxSVvHqaKsRWMRq26AmgkuWctwI1g3UQyjQLBOMLmd1ztPTGkey0czTZgf4UjykFM01nrAQW1QrhVdyGyDl4OFcjVHJS/+sOYphGThgrUue5ifEzVIZTwWalfqpZgnSCI9azKDFi2s8Wq87IhXWGJIyVfdKQhft7IsNI62kU2M4IzViv1ubmf7VeasJrP+MySQ2TdPlRmApiYjK/mwy5YtSIqQWkitdCR2jQmpsOiUbgrd68jq0a1XP8v1VpXGTx1GEMziHS/CgDg24gya0gMInuEV3hzhvDjvzseyteDkM6fwR87nD+odjYo=</latexit><latexit sha1_base64="fHY8L0Lq6jayN2ybi5nRkuRgM8k=">AB6nicbZBNS8NAEIYn9avWr6pHL4tF8FSItRj0YvHivYD2lAm2027dLMJuxuhP4ELx4U8eov8ua/cdvmoK0vLDy8M8POvEiuDau+0UNja3tneKu6W9/YPDo/LxSVvHqaKsRWMRq26AmgkuWctwI1g3UQyjQLBOMLmd1ztPTGkey0czTZgf4UjykFM01nrAQW1QrhVdyGyDl4OFcjVHJS/+sOYphGThgrUue5ifEzVIZTwWalfqpZgnSCI9azKDFi2s8Wq87IhXWGJIyVfdKQhft7IsNI62kU2M4IzViv1ubmf7VeasJrP+MySQ2TdPlRmApiYjK/mwy5YtSIqQWkitdCR2jQmpsOiUbgrd68jq0a1XP8v1VpXGTx1GEMziHS/CgDg24gya0gMInuEV3hzhvDjvzseyteDkM6fwR87nD+odjYo=</latexit><latexit sha1_base64="ck8pdC+ekZH4nUmSP+ZG7r8lEyk=">AB2XicbZDNSgMxFIXv1L86Vq1rN8EiuCozbnQpuHFZwbZCO5RM5k4bmskMyR2hDH0BF25EfC93vo3pz0JbDwQ+zknIvSculLQUBN9ebWd3b/+gfugfNfzjk9Nmo2fz0gjsilzl5jnmFpXU2CVJCp8LgzyLFfbj6f0i7+gsTLXTzQrMr4WMtUCk7O6oyaraAdLMW2IVxDC9YaNb+GS7KDUJxa0dhEFBUcUNSaFw7g9LiwUXUz7GgUPNM7RtRxzi6dk7A0N+5oYkv394uKZ9bOstjdzDhN7Ga2MP/LBiWlt1EldVESarH6KC0Vo5wtdmaJNChIzRxwYaSblYkJN1yQa8Z3HYSbG29D7odOn4MoA7ncAFXEMIN3MEDdKALAhJ4hXdv4r15H6uat6tDP4I+/zBzjGijg=</latexit><latexit sha1_base64="5EnDFfMDBOio/2eay9En10Yo+dY=">AB3icbZBLSwMxFIXv+Ky1anXrJlgEV2WmG10KblxWtA9oh5J7ShmcyQ3BHK0J/gxoUi/it3/hvTx0JbDwQ+zknIvSfKlLTk+9/e1vbO7t5+6aB8WDk6PqmeVto2zY3AlkhVaroRt6ikxhZJUtjNDPIkUtiJnfzvPOMxspUP9E0wzDhIy1jKTg565EPGoNqza/7C7FNCFZQg5Wag+pXf5iKPEFNQnFre4GfUVhwQ1IonJX7ucWMiwkfYc+h5gnasFiMOmOXzhmyODXuaGIL9/eLgifWTpPI3Uw4je16Njf/y3o5xTdhIXWE2qx/CjOFaOUzfdmQ2lQkJo64MJINysTY264INdO2ZUQrK+8Ce1GPXD84EMJzuECriCAa7iFe2hCwSM4AXe4N1T3qv3saxry1v1dgZ/5H3+AM+5jDU=</latexit><latexit sha1_base64="5EnDFfMDBOio/2eay9En10Yo+dY=">AB3icbZBLSwMxFIXv+Ky1anXrJlgEV2WmG10KblxWtA9oh5J7ShmcyQ3BHK0J/gxoUi/it3/hvTx0JbDwQ+zknIvSfKlLTk+9/e1vbO7t5+6aB8WDk6PqmeVto2zY3AlkhVaroRt6ikxhZJUtjNDPIkUtiJnfzvPOMxspUP9E0wzDhIy1jKTg565EPGoNqza/7C7FNCFZQg5Wag+pXf5iKPEFNQnFre4GfUVhwQ1IonJX7ucWMiwkfYc+h5gnasFiMOmOXzhmyODXuaGIL9/eLgifWTpPI3Uw4je16Njf/y3o5xTdhIXWE2qx/CjOFaOUzfdmQ2lQkJo64MJINysTY264INdO2ZUQrK+8Ce1GPXD84EMJzuECriCAa7iFe2hCwSM4AXe4N1T3qv3saxry1v1dgZ/5H3+AM+5jDU=</latexit><latexit sha1_base64="p8tLOQbHVpAOVm/uqaDbqhYE8i4=">AB6nicbZBNS8NAEIYn9avWr6hHL4tF8FSXvRY9OKxov2ANpTJdtMu3WzC7kYoT/BiwdFvPqLvPlv3LY5aOsLCw/vzLAzb5gKro3nfTuljc2t7Z3ybmVv/+DwyD0+aeskU5S1aCIS1Q1RM8ElaxluBOumimEcCtYJ7fzeueJKc0T+WimKQtiHEkecYrGWg84qA/cqlfzFiLr4BdQhULNgfvVHyY0i5k0VKDWPd9LTZCjMpwKNqv0M81SpBMcsZ5FiTHTQb5YdUYurDMkUaLsk4Ys3N8TOcZaT+PQdsZoxnq1Njf/q/UyE10HOZdpZpiky4+iTBCTkPndZMgVo0ZMLSBV3O5K6BgVUmPTqdgQ/NWT16Fdr/mW71q46aIowxncA6X4MVNOAOmtACiN4hld4c4Tz4rw7H8vWklPMnMIfOZ8/6N2Nhg=</latexit><latexit sha1_base64="fHY8L0Lq6jayN2ybi5nRkuRgM8k=">AB6nicbZBNS8NAEIYn9avWr6pHL4tF8FSItRj0YvHivYD2lAm2027dLMJuxuhP4ELx4U8eov8ua/cdvmoK0vLDy8M8POvEiuDau+0UNja3tneKu6W9/YPDo/LxSVvHqaKsRWMRq26AmgkuWctwI1g3UQyjQLBOMLmd1ztPTGkey0czTZgf4UjykFM01nrAQW1QrhVdyGyDl4OFcjVHJS/+sOYphGThgrUue5ifEzVIZTwWalfqpZgnSCI9azKDFi2s8Wq87IhXWGJIyVfdKQhft7IsNI62kU2M4IzViv1ubmf7VeasJrP+MySQ2TdPlRmApiYjK/mwy5YtSIqQWkitdCR2jQmpsOiUbgrd68jq0a1XP8v1VpXGTx1GEMziHS/CgDg24gya0gMInuEV3hzhvDjvzseyteDkM6fwR87nD+odjYo=</latexit><latexit sha1_base64="fHY8L0Lq6jayN2ybi5nRkuRgM8k=">AB6nicbZBNS8NAEIYn9avWr6pHL4tF8FSItRj0YvHivYD2lAm2027dLMJuxuhP4ELx4U8eov8ua/cdvmoK0vLDy8M8POvEiuDau+0UNja3tneKu6W9/YPDo/LxSVvHqaKsRWMRq26AmgkuWctwI1g3UQyjQLBOMLmd1ztPTGkey0czTZgf4UjykFM01nrAQW1QrhVdyGyDl4OFcjVHJS/+sOYphGThgrUue5ifEzVIZTwWalfqpZgnSCI9azKDFi2s8Wq87IhXWGJIyVfdKQhft7IsNI62kU2M4IzViv1ubmf7VeasJrP+MySQ2TdPlRmApiYjK/mwy5YtSIqQWkitdCR2jQmpsOiUbgrd68jq0a1XP8v1VpXGTx1GEMziHS/CgDg24gya0gMInuEV3hzhvDjvzseyteDkM6fwR87nD+odjYo=</latexit><latexit sha1_base64="fHY8L0Lq6jayN2ybi5nRkuRgM8k=">AB6nicbZBNS8NAEIYn9avWr6pHL4tF8FSItRj0YvHivYD2lAm2027dLMJuxuhP4ELx4U8eov8ua/cdvmoK0vLDy8M8POvEiuDau+0UNja3tneKu6W9/YPDo/LxSVvHqaKsRWMRq26AmgkuWctwI1g3UQyjQLBOMLmd1ztPTGkey0czTZgf4UjykFM01nrAQW1QrhVdyGyDl4OFcjVHJS/+sOYphGThgrUue5ifEzVIZTwWalfqpZgnSCI9azKDFi2s8Wq87IhXWGJIyVfdKQhft7IsNI62kU2M4IzViv1ubmf7VeasJrP+MySQ2TdPlRmApiYjK/mwy5YtSIqQWkitdCR2jQmpsOiUbgrd68jq0a1XP8v1VpXGTx1GEMziHS/CgDg24gya0gMInuEV3hzhvDjvzseyteDkM6fwR87nD+odjYo=</latexit><latexit sha1_base64="fHY8L0Lq6jayN2ybi5nRkuRgM8k=">AB6nicbZBNS8NAEIYn9avWr6pHL4tF8FSItRj0YvHivYD2lAm2027dLMJuxuhP4ELx4U8eov8ua/cdvmoK0vLDy8M8POvEiuDau+0UNja3tneKu6W9/YPDo/LxSVvHqaKsRWMRq26AmgkuWctwI1g3UQyjQLBOMLmd1ztPTGkey0czTZgf4UjykFM01nrAQW1QrhVdyGyDl4OFcjVHJS/+sOYphGThgrUue5ifEzVIZTwWalfqpZgnSCI9azKDFi2s8Wq87IhXWGJIyVfdKQhft7IsNI62kU2M4IzViv1ubmf7VeasJrP+MySQ2TdPlRmApiYjK/mwy5YtSIqQWkitdCR2jQmpsOiUbgrd68jq0a1XP8v1VpXGTx1GEMziHS/CgDg24gya0gMInuEV3hzhvDjvzseyteDkM6fwR87nD+odjYo=</latexit><latexit sha1_base64="fHY8L0Lq6jayN2ybi5nRkuRgM8k=">AB6nicbZBNS8NAEIYn9avWr6pHL4tF8FSItRj0YvHivYD2lAm2027dLMJuxuhP4ELx4U8eov8ua/cdvmoK0vLDy8M8POvEiuDau+0UNja3tneKu6W9/YPDo/LxSVvHqaKsRWMRq26AmgkuWctwI1g3UQyjQLBOMLmd1ztPTGkey0czTZgf4UjykFM01nrAQW1QrhVdyGyDl4OFcjVHJS/+sOYphGThgrUue5ifEzVIZTwWalfqpZgnSCI9azKDFi2s8Wq87IhXWGJIyVfdKQhft7IsNI62kU2M4IzViv1ubmf7VeasJrP+MySQ2TdPlRmApiYjK/mwy5YtSIqQWkitdCR2jQmpsOiUbgrd68jq0a1XP8v1VpXGTx1GEMziHS/CgDg24gya0gMInuEV3hzhvDjvzseyteDkM6fwR87nD+odjYo=</latexit><latexit sha1_base64="fHY8L0Lq6jayN2ybi5nRkuRgM8k=">AB6nicbZBNS8NAEIYn9avWr6pHL4tF8FSItRj0YvHivYD2lAm2027dLMJuxuhP4ELx4U8eov8ua/cdvmoK0vLDy8M8POvEiuDau+0UNja3tneKu6W9/YPDo/LxSVvHqaKsRWMRq26AmgkuWctwI1g3UQyjQLBOMLmd1ztPTGkey0czTZgf4UjykFM01nrAQW1QrhVdyGyDl4OFcjVHJS/+sOYphGThgrUue5ifEzVIZTwWalfqpZgnSCI9azKDFi2s8Wq87IhXWGJIyVfdKQhft7IsNI62kU2M4IzViv1ubmf7VeasJrP+MySQ2TdPlRmApiYjK/mwy5YtSIqQWkitdCR2jQmpsOiUbgrd68jq0a1XP8v1VpXGTx1GEMziHS/CgDg24gya0gMInuEV3hzhvDjvzseyteDkM6fwR87nD+odjYo=</latexit>

am

<latexit sha1_base64="uGkhqy/YuhTKr1EIG2B45HDVWx0=">AB6nicbZBNSwMxEIZn61etX1WPXoJF8FR2RdBj0YvHivYD2qXMptk2NMkuSVYopT/BiwdFvPqLvPlvTNs9aOsLgYd3ZsjMG6WCG+v7315hbX1jc6u4XdrZ3ds/KB8eNU2SacoaNBGJbkdomOCKNSy3grVTzVBGgrWi0e2s3npi2vBEPdpxykKJA8VjTtE6wF7sleu+FV/LrIKQ4VyFXvlb+6/YRmkilLBRrTCfzUhPUlPBpqVuZliKdIQD1nGoUDITuarTsmZc/okTrR7ypK5+3tigtKYsYxcp0Q7NMu1mflfrZPZ+DqcJVmlim6+CjOBLEJmd1N+lwzasXYAVLN3a6EDlEjtS6dkgshWD5FZoX1cDx/WldpPHUYQTOIVzCOAKanAHdWgAhQE8wyu8ecJ78d69j0VrwctnjuGPvM8fQ5iNxQ=</latexit><latexit sha1_base64="uGkhqy/YuhTKr1EIG2B45HDVWx0=">AB6nicbZBNSwMxEIZn61etX1WPXoJF8FR2RdBj0YvHivYD2qXMptk2NMkuSVYopT/BiwdFvPqLvPlvTNs9aOsLgYd3ZsjMG6WCG+v7315hbX1jc6u4XdrZ3ds/KB8eNU2SacoaNBGJbkdomOCKNSy3grVTzVBGgrWi0e2s3npi2vBEPdpxykKJA8VjTtE6wF7sleu+FV/LrIKQ4VyFXvlb+6/YRmkilLBRrTCfzUhPUlPBpqVuZliKdIQD1nGoUDITuarTsmZc/okTrR7ypK5+3tigtKYsYxcp0Q7NMu1mflfrZPZ+DqcJVmlim6+CjOBLEJmd1N+lwzasXYAVLN3a6EDlEjtS6dkgshWD5FZoX1cDx/WldpPHUYQTOIVzCOAKanAHdWgAhQE8wyu8ecJ78d69j0VrwctnjuGPvM8fQ5iNxQ=</latexit><latexit sha1_base64="uGkhqy/YuhTKr1EIG2B45HDVWx0=">AB6nicbZBNSwMxEIZn61etX1WPXoJF8FR2RdBj0YvHivYD2qXMptk2NMkuSVYopT/BiwdFvPqLvPlvTNs9aOsLgYd3ZsjMG6WCG+v7315hbX1jc6u4XdrZ3ds/KB8eNU2SacoaNBGJbkdomOCKNSy3grVTzVBGgrWi0e2s3npi2vBEPdpxykKJA8VjTtE6wF7sleu+FV/LrIKQ4VyFXvlb+6/YRmkilLBRrTCfzUhPUlPBpqVuZliKdIQD1nGoUDITuarTsmZc/okTrR7ypK5+3tigtKYsYxcp0Q7NMu1mflfrZPZ+DqcJVmlim6+CjOBLEJmd1N+lwzasXYAVLN3a6EDlEjtS6dkgshWD5FZoX1cDx/WldpPHUYQTOIVzCOAKanAHdWgAhQE8wyu8ecJ78d69j0VrwctnjuGPvM8fQ5iNxQ=</latexit><latexit sha1_base64="uGkhqy/YuhTKr1EIG2B45HDVWx0=">AB6nicbZBNSwMxEIZn61etX1WPXoJF8FR2RdBj0YvHivYD2qXMptk2NMkuSVYopT/BiwdFvPqLvPlvTNs9aOsLgYd3ZsjMG6WCG+v7315hbX1jc6u4XdrZ3ds/KB8eNU2SacoaNBGJbkdomOCKNSy3grVTzVBGgrWi0e2s3npi2vBEPdpxykKJA8VjTtE6wF7sleu+FV/LrIKQ4VyFXvlb+6/YRmkilLBRrTCfzUhPUlPBpqVuZliKdIQD1nGoUDITuarTsmZc/okTrR7ypK5+3tigtKYsYxcp0Q7NMu1mflfrZPZ+DqcJVmlim6+CjOBLEJmd1N+lwzasXYAVLN3a6EDlEjtS6dkgshWD5FZoX1cDx/WldpPHUYQTOIVzCOAKanAHdWgAhQE8wyu8ecJ78d69j0VrwctnjuGPvM8fQ5iNxQ=</latexit>

b P(yi > rk) = s m X

j

wjaj + bk

• <latexit sha1_base64="ODmb6Tbw4kN7qLYf/45mjVP376E=">ACJXicbZDLSsNAFIYnXmu9RV26GSxCi1ASEXShUnTjsoK9QFPDZDp51JwszEUkJfxo2v4saFRQRXvorTNAtPXDg4/PYeb8XsSoVJb1ZSwtr6yurec28ptb2zu75t5+XYaxwKSGQxaKpockYTQgNUVI81IEMQ9Rhre4HbqN56IkDQMHtQoIm2OugH1KUZKS656Qxph/SQSqrj4sil8BoKd1CV1A6Hu12i46Mudt/5HDo9iHSfQI9d5B6JdcsWGUrLbgIdgYFkFXVNSdOJ8QxJ4HCDEnZsq1ItRMkFMWMjPNOLEmE8AB1SUtjgDiR7S9cgyPtdKBfih0Bwqm6u+NBHEpR9zTkxypnpz3puJ/XitW/kU7oUEUKxLg2UN+zKAK4TQy2KGCYMVGhAWVP8V4h4SCsdbF6HYM+fvAj107Kt+f6sULnJ4siBQ3AEisAG56AC7kAV1AGz+AVvIOJ8WK8GR/G52x0ych2DsCfMr5/AMaYo4o=</latexit><latexit sha1_base64="ODmb6Tbw4kN7qLYf/45mjVP376E=">ACJXicbZDLSsNAFIYnXmu9RV26GSxCi1ASEXShUnTjsoK9QFPDZDp51JwszEUkJfxo2v4saFRQRXvorTNAtPXDg4/PYeb8XsSoVJb1ZSwtr6yurec28ptb2zu75t5+XYaxwKSGQxaKpockYTQgNUVI81IEMQ9Rhre4HbqN56IkDQMHtQoIm2OugH1KUZKS656Qxph/SQSqrj4sil8BoKd1CV1A6Hu12i46Mudt/5HDo9iHSfQI9d5B6JdcsWGUrLbgIdgYFkFXVNSdOJ8QxJ4HCDEnZsq1ItRMkFMWMjPNOLEmE8AB1SUtjgDiR7S9cgyPtdKBfih0Bwqm6u+NBHEpR9zTkxypnpz3puJ/XitW/kU7oUEUKxLg2UN+zKAK4TQy2KGCYMVGhAWVP8V4h4SCsdbF6HYM+fvAj107Kt+f6sULnJ4siBQ3AEisAG56AC7kAV1AGz+AVvIOJ8WK8GR/G52x0ych2DsCfMr5/AMaYo4o=</latexit><latexit sha1_base64="ODmb6Tbw4kN7qLYf/45mjVP376E=">ACJXicbZDLSsNAFIYnXmu9RV26GSxCi1ASEXShUnTjsoK9QFPDZDp51JwszEUkJfxo2v4saFRQRXvorTNAtPXDg4/PYeb8XsSoVJb1ZSwtr6yurec28ptb2zu75t5+XYaxwKSGQxaKpockYTQgNUVI81IEMQ9Rhre4HbqN56IkDQMHtQoIm2OugH1KUZKS656Qxph/SQSqrj4sil8BoKd1CV1A6Hu12i46Mudt/5HDo9iHSfQI9d5B6JdcsWGUrLbgIdgYFkFXVNSdOJ8QxJ4HCDEnZsq1ItRMkFMWMjPNOLEmE8AB1SUtjgDiR7S9cgyPtdKBfih0Bwqm6u+NBHEpR9zTkxypnpz3puJ/XitW/kU7oUEUKxLg2UN+zKAK4TQy2KGCYMVGhAWVP8V4h4SCsdbF6HYM+fvAj107Kt+f6sULnJ4siBQ3AEisAG56AC7kAV1AGz+AVvIOJ8WK8GR/G52x0ych2DsCfMr5/AMaYo4o=</latexit><latexit sha1_base64="ODmb6Tbw4kN7qLYf/45mjVP376E=">ACJXicbZDLSsNAFIYnXmu9RV26GSxCi1ASEXShUnTjsoK9QFPDZDp51JwszEUkJfxo2v4saFRQRXvorTNAtPXDg4/PYeb8XsSoVJb1ZSwtr6yurec28ptb2zu75t5+XYaxwKSGQxaKpockYTQgNUVI81IEMQ9Rhre4HbqN56IkDQMHtQoIm2OugH1KUZKS656Qxph/SQSqrj4sil8BoKd1CV1A6Hu12i46Mudt/5HDo9iHSfQI9d5B6JdcsWGUrLbgIdgYFkFXVNSdOJ8QxJ4HCDEnZsq1ItRMkFMWMjPNOLEmE8AB1SUtjgDiR7S9cgyPtdKBfih0Bwqm6u+NBHEpR9zTkxypnpz3puJ/XitW/kU7oUEUKxLg2UN+zKAK4TQy2KGCYMVGhAWVP8V4h4SCsdbF6HYM+fvAj107Kt+f6sULnJ4siBQ3AEisAG56AC7kAV1AGz+AVvIOJ8WK8GR/G52x0ych2DsCfMr5/AMaYo4o=</latexit>

Age label

[30]

<latexit sha1_base64="M7vmw7VbJIUf0IZQmzGANVHtQBA=">AB63icbZDLSsNAFIZP6q3W9Wlm8EiuCqJCrosunFZwV4gDWUynbRDZyZhZiKU0Fdw40IRt76QO9/GSZqFtv4w8PGfc5hz/jDhTBvX/XYqa+sbm1vV7drO7t7+Qf3wqKvjVBHaITGPVT/EmnImacw2k/URSLkNeOL3L670nqjSL5aOZJTQeCxZxAg2ueVfusGw3nCbiG0Cl4JDSjVHta/BqOYpIJKQzjW2vfcxAQZVoYRTue1QapgskUj6lvUWJBdZAVu87RmXVGKIqVfdKgwv09kWGh9UyEtlNgM9HLtdz8r+anJroJMiaT1FBJFh9FKUcmRvnhaMQUJYbPLGCimN0VkQlWmBgbT82G4C2fvArdi6Zn+eGq0bot46jCZzCOXhwDS24hzZ0gMAEnuEV3hzhvDjvzseiteKUM8fwR87nD02+jb0=</latexit><latexit sha1_base64="M7vmw7VbJIUf0IZQmzGANVHtQBA=">AB63icbZDLSsNAFIZP6q3W9Wlm8EiuCqJCrosunFZwV4gDWUynbRDZyZhZiKU0Fdw40IRt76QO9/GSZqFtv4w8PGfc5hz/jDhTBvX/XYqa+sbm1vV7drO7t7+Qf3wqKvjVBHaITGPVT/EmnImacw2k/URSLkNeOL3L670nqjSL5aOZJTQeCxZxAg2ueVfusGw3nCbiG0Cl4JDSjVHta/BqOYpIJKQzjW2vfcxAQZVoYRTue1QapgskUj6lvUWJBdZAVu87RmXVGKIqVfdKgwv09kWGh9UyEtlNgM9HLtdz8r+anJroJMiaT1FBJFh9FKUcmRvnhaMQUJYbPLGCimN0VkQlWmBgbT82G4C2fvArdi6Zn+eGq0bot46jCZzCOXhwDS24hzZ0gMAEnuEV3hzhvDjvzseiteKUM8fwR87nD02+jb0=</latexit><latexit sha1_base64="M7vmw7VbJIUf0IZQmzGANVHtQBA=">AB63icbZDLSsNAFIZP6q3W9Wlm8EiuCqJCrosunFZwV4gDWUynbRDZyZhZiKU0Fdw40IRt76QO9/GSZqFtv4w8PGfc5hz/jDhTBvX/XYqa+sbm1vV7drO7t7+Qf3wqKvjVBHaITGPVT/EmnImacw2k/URSLkNeOL3L670nqjSL5aOZJTQeCxZxAg2ueVfusGw3nCbiG0Cl4JDSjVHta/BqOYpIJKQzjW2vfcxAQZVoYRTue1QapgskUj6lvUWJBdZAVu87RmXVGKIqVfdKgwv09kWGh9UyEtlNgM9HLtdz8r+anJroJMiaT1FBJFh9FKUcmRvnhaMQUJYbPLGCimN0VkQlWmBgbT82G4C2fvArdi6Zn+eGq0bot46jCZzCOXhwDS24hzZ0gMAEnuEV3hzhvDjvzseiteKUM8fwR87nD02+jb0=</latexit><latexit sha1_base64="M7vmw7VbJIUf0IZQmzGANVHtQBA=">AB63icbZDLSsNAFIZP6q3W9Wlm8EiuCqJCrosunFZwV4gDWUynbRDZyZhZiKU0Fdw40IRt76QO9/GSZqFtv4w8PGfc5hz/jDhTBvX/XYqa+sbm1vV7drO7t7+Qf3wqKvjVBHaITGPVT/EmnImacw2k/URSLkNeOL3L670nqjSL5aOZJTQeCxZxAg2ueVfusGw3nCbiG0Cl4JDSjVHta/BqOYpIJKQzjW2vfcxAQZVoYRTue1QapgskUj6lvUWJBdZAVu87RmXVGKIqVfdKgwv09kWGh9UyEtlNgM9HLtdz8r+anJroJMiaT1FBJFh9FKUcmRvnhaMQUJYbPLGCimN0VkQlWmBgbT82G4C2fvArdi6Zn+eGq0bot46jCZzCOXhwDS24hzZ0gMAEnuEV3hzhvDjvzseiteKUM8fwR87nD02+jb0=</latexit>

       1 1 . . .        ∈ ZK−1

2

<latexit sha1_base64="/XCD8nf7cWvcS8D4qcb5KAZy6w=">ACNXicbZBNSwMxEIazflu/qh69BIvgRdkVQY9FL4IeFKwtNrUk2WkNZrNLkhXLsn/Ki/Dkx48KOLVv2C2LaLWgYEn78yQmZclUhjr+8/e2PjE5NT0zGxpbn5hcam8vHJh4lRzqPFYxrBqAEpFNSsBIaiQYaMQl1dnNY1Ou3oI2I1bntJdCKaFeJjuDUOqldPiEMukJlLKJWi7u8FBDST3IbxtY48IvEBFT43YSJUJi4xzVj2WXe3rnKjreCvNQuV/xtvx94FIhVNAwTtvlRxLGPI1AWS6pMc3AT2wro9oKLiEvkdRAQvkN7ULToaIRmFbWvzrHG04JcSfWLpXFfXnREYjY3oRc53FquZvrRD/qzVT29lvZUIlqQXFBx91UoltjAsLcSg0cCt7DijXwu2K+TXVlFtndGFC8PfkUbjY2Q4cn+1WqgdDO2bQGlpHmyhAe6iKjtApqiGO7tETekVv3oP34r17H4PWMW84s4p+hf5BaWDqh4=</latexit><latexit sha1_base64="/XCD8nf7cWvcS8D4qcb5KAZy6w=">ACNXicbZBNSwMxEIazflu/qh69BIvgRdkVQY9FL4IeFKwtNrUk2WkNZrNLkhXLsn/Ki/Dkx48KOLVv2C2LaLWgYEn78yQmZclUhjr+8/e2PjE5NT0zGxpbn5hcam8vHJh4lRzqPFYxrBqAEpFNSsBIaiQYaMQl1dnNY1Ou3oI2I1bntJdCKaFeJjuDUOqldPiEMukJlLKJWi7u8FBDST3IbxtY48IvEBFT43YSJUJi4xzVj2WXe3rnKjreCvNQuV/xtvx94FIhVNAwTtvlRxLGPI1AWS6pMc3AT2wro9oKLiEvkdRAQvkN7ULToaIRmFbWvzrHG04JcSfWLpXFfXnREYjY3oRc53FquZvrRD/qzVT29lvZUIlqQXFBx91UoltjAsLcSg0cCt7DijXwu2K+TXVlFtndGFC8PfkUbjY2Q4cn+1WqgdDO2bQGlpHmyhAe6iKjtApqiGO7tETekVv3oP34r17H4PWMW84s4p+hf5BaWDqh4=</latexit><latexit sha1_base64="/XCD8nf7cWvcS8D4qcb5KAZy6w=">ACNXicbZBNSwMxEIazflu/qh69BIvgRdkVQY9FL4IeFKwtNrUk2WkNZrNLkhXLsn/Ki/Dkx48KOLVv2C2LaLWgYEn78yQmZclUhjr+8/e2PjE5NT0zGxpbn5hcam8vHJh4lRzqPFYxrBqAEpFNSsBIaiQYaMQl1dnNY1Ou3oI2I1bntJdCKaFeJjuDUOqldPiEMukJlLKJWi7u8FBDST3IbxtY48IvEBFT43YSJUJi4xzVj2WXe3rnKjreCvNQuV/xtvx94FIhVNAwTtvlRxLGPI1AWS6pMc3AT2wro9oKLiEvkdRAQvkN7ULToaIRmFbWvzrHG04JcSfWLpXFfXnREYjY3oRc53FquZvrRD/qzVT29lvZUIlqQXFBx91UoltjAsLcSg0cCt7DijXwu2K+TXVlFtndGFC8PfkUbjY2Q4cn+1WqgdDO2bQGlpHmyhAe6iKjtApqiGO7tETekVv3oP34r17H4PWMW84s4p+hf5BaWDqh4=</latexit><latexit sha1_base64="/XCD8nf7cWvcS8D4qcb5KAZy6w=">ACNXicbZBNSwMxEIazflu/qh69BIvgRdkVQY9FL4IeFKwtNrUk2WkNZrNLkhXLsn/Ki/Dkx48KOLVv2C2LaLWgYEn78yQmZclUhjr+8/e2PjE5NT0zGxpbn5hcam8vHJh4lRzqPFYxrBqAEpFNSsBIaiQYaMQl1dnNY1Ou3oI2I1bntJdCKaFeJjuDUOqldPiEMukJlLKJWi7u8FBDST3IbxtY48IvEBFT43YSJUJi4xzVj2WXe3rnKjreCvNQuV/xtvx94FIhVNAwTtvlRxLGPI1AWS6pMc3AT2wro9oKLiEvkdRAQvkN7ULToaIRmFbWvzrHG04JcSfWLpXFfXnREYjY3oRc53FquZvrRD/qzVT29lvZUIlqQXFBx91UoltjAsLcSg0cCt7DijXwu2K+TXVlFtndGFC8PfkUbjY2Q4cn+1WqgdDO2bQGlpHmyhAe6iKjtApqiGO7tETekVv3oP34r17H4PWMW84s4p+hf5BaWDqh4=</latexit>

Extended label

{

<latexit sha1_base64="tFROaEGhmnMpUlcEPFjR2z/BNI=">AB6XicbZBNS8NAEIYn9avWr6hHL4tF8FQSEfRY9OKxiv2ANpTNdtIu3WzC7kYof/AiwdFvPqPvPlv3LY5aOsLCw/vzLAzb5gKro3nfTultfWNza3ydmVnd2/wD08aukUwybLBGJ6oRUo+ASm4YbgZ1UIY1Dge1wfDurt59QaZ7IRzNJMYjpUPKIM2qs9dDL+27Vq3lzkVXwC6hCoUbf/eoNEpbFKA0TVOu76UmyKkynAmcVnqZxpSyMR1i16KkMeogn286JWfWGZAoUfZJQ+bu74mcxlpP4tB2xtSM9HJtZv5X62Ymug5yLtPMoGSLj6JMEJOQ2dlkwBUyIyYWKFPc7krYiCrKjA2nYkPwl09ehdZFzbd8f1mt3xRxlOETuEcfLiCOtxBA5rAIJneIU3Z+y8O/Ox6K15BQzx/BHzucPm4aNZQ=</latexit><latexit sha1_base64="tFROaEGhmnMpUlcEPFjR2z/BNI=">AB6XicbZBNS8NAEIYn9avWr6hHL4tF8FQSEfRY9OKxiv2ANpTNdtIu3WzC7kYof/AiwdFvPqPvPlv3LY5aOsLCw/vzLAzb5gKro3nfTultfWNza3ydmVnd2/wD08aukUwybLBGJ6oRUo+ASm4YbgZ1UIY1Dge1wfDurt59QaZ7IRzNJMYjpUPKIM2qs9dDL+27Vq3lzkVXwC6hCoUbf/eoNEpbFKA0TVOu76UmyKkynAmcVnqZxpSyMR1i16KkMeogn286JWfWGZAoUfZJQ+bu74mcxlpP4tB2xtSM9HJtZv5X62Ymug5yLtPMoGSLj6JMEJOQ2dlkwBUyIyYWKFPc7krYiCrKjA2nYkPwl09ehdZFzbd8f1mt3xRxlOETuEcfLiCOtxBA5rAIJneIU3Z+y8O/Ox6K15BQzx/BHzucPm4aNZQ=</latexit><latexit sha1_base64="tFROaEGhmnMpUlcEPFjR2z/BNI=">AB6XicbZBNS8NAEIYn9avWr6hHL4tF8FQSEfRY9OKxiv2ANpTNdtIu3WzC7kYof/AiwdFvPqPvPlv3LY5aOsLCw/vzLAzb5gKro3nfTultfWNza3ydmVnd2/wD08aukUwybLBGJ6oRUo+ASm4YbgZ1UIY1Dge1wfDurt59QaZ7IRzNJMYjpUPKIM2qs9dDL+27Vq3lzkVXwC6hCoUbf/eoNEpbFKA0TVOu76UmyKkynAmcVnqZxpSyMR1i16KkMeogn286JWfWGZAoUfZJQ+bu74mcxlpP4tB2xtSM9HJtZv5X62Ymug5yLtPMoGSLj6JMEJOQ2dlkwBUyIyYWKFPc7krYiCrKjA2nYkPwl09ehdZFzbd8f1mt3xRxlOETuEcfLiCOtxBA5rAIJneIU3Z+y8O/Ox6K15BQzx/BHzucPm4aNZQ=</latexit><latexit sha1_base64="tFROaEGhmnMpUlcEPFjR2z/BNI=">AB6XicbZBNS8NAEIYn9avWr6hHL4tF8FQSEfRY9OKxiv2ANpTNdtIu3WzC7kYof/AiwdFvPqPvPlv3LY5aOsLCw/vzLAzb5gKro3nfTultfWNza3ydmVnd2/wD08aukUwybLBGJ6oRUo+ASm4YbgZ1UIY1Dge1wfDurt59QaZ7IRzNJMYjpUPKIM2qs9dDL+27Vq3lzkVXwC6hCoUbf/eoNEpbFKA0TVOu76UmyKkynAmcVnqZxpSyMR1i16KkMeogn286JWfWGZAoUfZJQ+bu74mcxlpP4tB2xtSM9HJtZv5X62Ymug5yLtPMoGSLj6JMEJOQ2dlkwBUyIyYWKFPc7krYiCrKjA2nYkPwl09ehdZFzbd8f1mt3xRxlOETuEcfLiCOtxBA5rAIJneIU3Z+y8O/Ox6K15BQzx/BHzucPm4aNZQ=</latexit>

Label extension during training

Deep Learning & Ordinal Data

SLIDE 32

32

Wenzhi Cao, Vahid Mirjalili, and Sebastian Raschka. "Rank-consistent ordinal regression for neural networks." arXiv:1901.07884 (2019).

OR-C COR-C

A B

SLIDE 33

33

Table 2 Average numbers of inconsistencies occurred on the different test datasets for CORAL-CNN and Niu et al’s Ordinal CNN. The penultimate column and last column list the average numbers of inconsistencies focussing only on the correct and incorrect age predictions, respectively. CORAL-CNN Ordinal-CNN [1] Ordinal-CNN [1] Ordinal-CNN [1] All predictions All predictions Only correct predictions Only incorrect predictions Morph Seed 0 2.74 2.02 2.89 Seed 1 2.74 2.08 2.88 Seed 2 3.00 2.20 3.16 AFAD Seed 0 2.32 1.78 2.40 Seed 1 2.35 1.83 2.43 Seed 2 2.55 1.97 2.63 UTKFace Seed 0 4.79 3.64 4.92 Seed 1 5.73 4.05 5.95 Seed 2 5.07 3.84 5.21 CACD Seed 0 5.06 4.55 5.10 Seed 1 5.40 4.76 5.44 Seed 2 5.56 4.87 5.61

Wenzhi Cao, Vahid Mirjalili, and Sebastian Raschka. "Rank-consistent ordinal regression for neural networks." arXiv:1901.07884 (2019).

SLIDE 34

34

Input image 7x7 conv @64 stride=2

…

3x3 conv @512 stride=1

. . .

Weight sharing across k-1 tasks

b1

<latexit sha1_base64="oxRwFKiWcKO5u9w5ICndUHjxWI=">AB6nicbZBNS8NAEIYn9avWr6hHL4tF8FQSEfRY9OKxov2ANpTNdtMu3WzC7kQoT/BiwdFvPqLvPlv3LY5aOsLCw/vzLAzb5hKYdDzvp3S2vrG5lZ5u7Kzu7d/4B4etUySacabLJGJ7oTUcCkUb6JAyTup5jQOJW+H49tZvf3EtRGJesRJyoOYDpWIBKNorYew7/fdqlfz5iKr4BdQhUKNvVGyQsi7lCJqkxXd9LMcipRsEkn1Z6meEpZWM65F2LisbcBPl81Sk5s86ARIm2TyGZu78nchobM4lD2xlTHJnl2sz8r9bNMLoOcqHSDLli4+iTBJMyOxuMhCaM5QTC5RpYXclbEQ1ZWjTqdgQ/OWTV6F1UfMt319W6zdFHGU4gVM4Bx+uoA530IAmMBjCM7zCmyOdF+fd+Vi0lpxi5hj+yPn8AeofjYo=</latexit><latexit sha1_base64="oxRwFKiWcKO5u9w5ICndUHjxWI=">AB6nicbZBNS8NAEIYn9avWr6hHL4tF8FQSEfRY9OKxov2ANpTNdtMu3WzC7kQoT/BiwdFvPqLvPlv3LY5aOsLCw/vzLAzb5hKYdDzvp3S2vrG5lZ5u7Kzu7d/4B4etUySacabLJGJ7oTUcCkUb6JAyTup5jQOJW+H49tZvf3EtRGJesRJyoOYDpWIBKNorYew7/fdqlfz5iKr4BdQhUKNvVGyQsi7lCJqkxXd9LMcipRsEkn1Z6meEpZWM65F2LisbcBPl81Sk5s86ARIm2TyGZu78nchobM4lD2xlTHJnl2sz8r9bNMLoOcqHSDLli4+iTBJMyOxuMhCaM5QTC5RpYXclbEQ1ZWjTqdgQ/OWTV6F1UfMt319W6zdFHGU4gVM4Bx+uoA530IAmMBjCM7zCmyOdF+fd+Vi0lpxi5hj+yPn8AeofjYo=</latexit><latexit sha1_base64="oxRwFKiWcKO5u9w5ICndUHjxWI=">AB6nicbZBNS8NAEIYn9avWr6hHL4tF8FQSEfRY9OKxov2ANpTNdtMu3WzC7kQoT/BiwdFvPqLvPlv3LY5aOsLCw/vzLAzb5hKYdDzvp3S2vrG5lZ5u7Kzu7d/4B4etUySacabLJGJ7oTUcCkUb6JAyTup5jQOJW+H49tZvf3EtRGJesRJyoOYDpWIBKNorYew7/fdqlfz5iKr4BdQhUKNvVGyQsi7lCJqkxXd9LMcipRsEkn1Z6meEpZWM65F2LisbcBPl81Sk5s86ARIm2TyGZu78nchobM4lD2xlTHJnl2sz8r9bNMLoOcqHSDLli4+iTBJMyOxuMhCaM5QTC5RpYXclbEQ1ZWjTqdgQ/OWTV6F1UfMt319W6zdFHGU4gVM4Bx+uoA530IAmMBjCM7zCmyOdF+fd+Vi0lpxi5hj+yPn8AeofjYo=</latexit><latexit sha1_base64="oxRwFKiWcKO5u9w5ICndUHjxWI=">AB6nicbZBNS8NAEIYn9avWr6hHL4tF8FQSEfRY9OKxov2ANpTNdtMu3WzC7kQoT/BiwdFvPqLvPlv3LY5aOsLCw/vzLAzb5hKYdDzvp3S2vrG5lZ5u7Kzu7d/4B4etUySacabLJGJ7oTUcCkUb6JAyTup5jQOJW+H49tZvf3EtRGJesRJyoOYDpWIBKNorYew7/fdqlfz5iKr4BdQhUKNvVGyQsi7lCJqkxXd9LMcipRsEkn1Z6meEpZWM65F2LisbcBPl81Sk5s86ARIm2TyGZu78nchobM4lD2xlTHJnl2sz8r9bNMLoOcqHSDLli4+iTBJMyOxuMhCaM5QTC5RpYXclbEQ1ZWjTqdgQ/OWTV6F1UfMt319W6zdFHGU4gVM4Bx+uoA530IAmMBjCM7zCmyOdF+fd+Vi0lpxi5hj+yPn8AeofjYo=</latexit>

b2

<latexit sha1_base64="KmwIREzjRQ4yYecphZP2rZMfohI=">AB6nicbZBNS8NAEIYn9avWr6pHL4tF8FSItRj0YvHivYD2lA20m7dLMJuxuhP4ELx4U8eov8ua/cdvmoK0vLDy8M8POvEiuDau+0UNja3tneKu6W9/YPDo/LxSVvHqWLYrGIVTegGgWX2DLcCOwmCmkUCOwEk9t5vfOESvNYPpgn5ER5KHnFjrYdgUBuUK27VXYisg5dDBXI1B+Wv/jBmaYTSMEG17nluYvyMKsOZwFmpn2pMKJvQEfYsShqh9rPFqjNyYZ0hCWNlnzRk4f6eyGik9TQKbGdEzViv1ubmf7VeasJrP+MySQ1KtvwoTAUxMZnfTYZcITNiaoEyxe2uhI2poszYdEo2BG/15HVo16qe5furSuMmj6MIZ3AOl+BHRpwB01oAYMRPMrvDnCeXHenY9la8HJZ07hj5zPH+ujYs=</latexit><latexit sha1_base64="KmwIREzjRQ4yYecphZP2rZMfohI=">AB6nicbZBNS8NAEIYn9avWr6pHL4tF8FSItRj0YvHivYD2lA20m7dLMJuxuhP4ELx4U8eov8ua/cdvmoK0vLDy8M8POvEiuDau+0UNja3tneKu6W9/YPDo/LxSVvHqWLYrGIVTegGgWX2DLcCOwmCmkUCOwEk9t5vfOESvNYPpgn5ER5KHnFjrYdgUBuUK27VXYisg5dDBXI1B+Wv/jBmaYTSMEG17nluYvyMKsOZwFmpn2pMKJvQEfYsShqh9rPFqjNyYZ0hCWNlnzRk4f6eyGik9TQKbGdEzViv1ubmf7VeasJrP+MySQ1KtvwoTAUxMZnfTYZcITNiaoEyxe2uhI2poszYdEo2BG/15HVo16qe5furSuMmj6MIZ3AOl+BHRpwB01oAYMRPMrvDnCeXHenY9la8HJZ07hj5zPH+ujYs=</latexit><latexit sha1_base64="KmwIREzjRQ4yYecphZP2rZMfohI=">AB6nicbZBNS8NAEIYn9avWr6pHL4tF8FSItRj0YvHivYD2lA20m7dLMJuxuhP4ELx4U8eov8ua/cdvmoK0vLDy8M8POvEiuDau+0UNja3tneKu6W9/YPDo/LxSVvHqWLYrGIVTegGgWX2DLcCOwmCmkUCOwEk9t5vfOESvNYPpgn5ER5KHnFjrYdgUBuUK27VXYisg5dDBXI1B+Wv/jBmaYTSMEG17nluYvyMKsOZwFmpn2pMKJvQEfYsShqh9rPFqjNyYZ0hCWNlnzRk4f6eyGik9TQKbGdEzViv1ubmf7VeasJrP+MySQ1KtvwoTAUxMZnfTYZcITNiaoEyxe2uhI2poszYdEo2BG/15HVo16qe5furSuMmj6MIZ3AOl+BHRpwB01oAYMRPMrvDnCeXHenY9la8HJZ07hj5zPH+ujYs=</latexit><latexit sha1_base64="KmwIREzjRQ4yYecphZP2rZMfohI=">AB6nicbZBNS8NAEIYn9avWr6pHL4tF8FSItRj0YvHivYD2lA20m7dLMJuxuhP4ELx4U8eov8ua/cdvmoK0vLDy8M8POvEiuDau+0UNja3tneKu6W9/YPDo/LxSVvHqWLYrGIVTegGgWX2DLcCOwmCmkUCOwEk9t5vfOESvNYPpgn5ER5KHnFjrYdgUBuUK27VXYisg5dDBXI1B+Wv/jBmaYTSMEG17nluYvyMKsOZwFmpn2pMKJvQEfYsShqh9rPFqjNyYZ0hCWNlnzRk4f6eyGik9TQKbGdEzViv1ubmf7VeasJrP+MySQ1KtvwoTAUxMZnfTYZcITNiaoEyxe2uhI2poszYdEo2BG/15HVo16qe5furSuMmj6MIZ3AOl+BHRpwB01oAYMRPMrvDnCeXHenY9la8HJZ07hj5zPH+ujYs=</latexit>

w1

<latexit sha1_base64="ozSIzVA/SGXegmac4XRXthOpvw0=">AB6nicbZBNS8NAEIYn9avWr6pHL4tF8FQSEeqx6MVjRfsBbSib7aZdutmE3YlSQn+CFw+KePUXefPfuG1z0NYXFh7emWFn3iCRwqDrfjuFtfWNza3idmlnd2/oHx41DJxqhlvsljGuhNQw6VQvIkCJe8kmtMokLwdjG9m9fYj10bE6gEnCfcjOlQiFIyite6f+l6/XHGr7lxkFbwcKpCr0S9/9QYxSyOukElqTNdzE/QzqlEwyaelXmp4QtmYDnXoqIRN342X3VKzqwzIGs7VNI5u7viYxGxkyiwHZGFEdmuTYz/6t1Uwyv/EyoJEWu2OKjMJUEYzK7mwyE5gzlxAJlWthdCRtRTRnadEo2BG/5FVoXVQ9y3eXlfp1HkcRTuAUzsGDGtThFhrQBAZDeIZXeHOk8+K8Ox+L1oKTzxzDHzmfPwosjZ8=</latexit><latexit sha1_base64="ozSIzVA/SGXegmac4XRXthOpvw0=">AB6nicbZBNS8NAEIYn9avWr6pHL4tF8FQSEeqx6MVjRfsBbSib7aZdutmE3YlSQn+CFw+KePUXefPfuG1z0NYXFh7emWFn3iCRwqDrfjuFtfWNza3idmlnd2/oHx41DJxqhlvsljGuhNQw6VQvIkCJe8kmtMokLwdjG9m9fYj10bE6gEnCfcjOlQiFIyite6f+l6/XHGr7lxkFbwcKpCr0S9/9QYxSyOukElqTNdzE/QzqlEwyaelXmp4QtmYDnXoqIRN342X3VKzqwzIGs7VNI5u7viYxGxkyiwHZGFEdmuTYz/6t1Uwyv/EyoJEWu2OKjMJUEYzK7mwyE5gzlxAJlWthdCRtRTRnadEo2BG/5FVoXVQ9y3eXlfp1HkcRTuAUzsGDGtThFhrQBAZDeIZXeHOk8+K8Ox+L1oKTzxzDHzmfPwosjZ8=</latexit><latexit sha1_base64="ozSIzVA/SGXegmac4XRXthOpvw0=">AB6nicbZBNS8NAEIYn9avWr6pHL4tF8FQSEeqx6MVjRfsBbSib7aZdutmE3YlSQn+CFw+KePUXefPfuG1z0NYXFh7emWFn3iCRwqDrfjuFtfWNza3idmlnd2/oHx41DJxqhlvsljGuhNQw6VQvIkCJe8kmtMokLwdjG9m9fYj10bE6gEnCfcjOlQiFIyite6f+l6/XHGr7lxkFbwcKpCr0S9/9QYxSyOukElqTNdzE/QzqlEwyaelXmp4QtmYDnXoqIRN342X3VKzqwzIGs7VNI5u7viYxGxkyiwHZGFEdmuTYz/6t1Uwyv/EyoJEWu2OKjMJUEYzK7mwyE5gzlxAJlWthdCRtRTRnadEo2BG/5FVoXVQ9y3eXlfp1HkcRTuAUzsGDGtThFhrQBAZDeIZXeHOk8+K8Ox+L1oKTzxzDHzmfPwosjZ8=</latexit><latexit sha1_base64="ozSIzVA/SGXegmac4XRXthOpvw0=">AB6nicbZBNS8NAEIYn9avWr6pHL4tF8FQSEeqx6MVjRfsBbSib7aZdutmE3YlSQn+CFw+KePUXefPfuG1z0NYXFh7emWFn3iCRwqDrfjuFtfWNza3idmlnd2/oHx41DJxqhlvsljGuhNQw6VQvIkCJe8kmtMokLwdjG9m9fYj10bE6gEnCfcjOlQiFIyite6f+l6/XHGr7lxkFbwcKpCr0S9/9QYxSyOukElqTNdzE/QzqlEwyaelXmp4QtmYDnXoqIRN342X3VKzqwzIGs7VNI5u7viYxGxkyiwHZGFEdmuTYz/6t1Uwyv/EyoJEWu2OKjMJUEYzK7mwyE5gzlxAJlWthdCRtRTRnadEo2BG/5FVoXVQ9y3eXlfp1HkcRTuAUzsGDGtThFhrQBAZDeIZXeHOk8+K8Ox+L1oKTzxzDHzmfPwosjZ8=</latexit>

w2

<latexit sha1_base64="sAe26MpFncoK5AcSpzUnkA9I=">AB6nicbZBNS8NAEIYn9avWr6pHL4tF8FSIuix6MVjRfsBbSib7aZdutmE3YlSQn+CFw+KePUXefPfuG1z0NYXFh7emWFn3iCRwqDrfjuFtfWNza3idmlnd2/oHx41DJxqhlvsljGuhNQw6VQvIkCJe8kmtMokLwdjG9m9fYj10bE6gEnCfcjOlQiFIyite6f+rV+ueJW3bnIKng5VCBXo1/+6g1ilkZcIZPUmK7nJuhnVKNgk9LvdTwhLIxHfKuRUjbvxsvuqUnFlnQMJY26eQzN3fExmNjJlEge2MKI7Mcm1m/lfrphe+ZlQSYpcscVHYSoJxmR2NxkIzRnKiQXKtLC7EjaimjK06ZRsCN7yavQqlU9y3cXlfp1HkcRTuAUzsGDS6jDLTSgCQyG8Ayv8OZI58V5dz4WrQUnzmGP3I+fwALsI2g</latexit><latexit sha1_base64="sAe26MpFncoK5AcSpzUnkA9I=">AB6nicbZBNS8NAEIYn9avWr6pHL4tF8FSIuix6MVjRfsBbSib7aZdutmE3YlSQn+CFw+KePUXefPfuG1z0NYXFh7emWFn3iCRwqDrfjuFtfWNza3idmlnd2/oHx41DJxqhlvsljGuhNQw6VQvIkCJe8kmtMokLwdjG9m9fYj10bE6gEnCfcjOlQiFIyite6f+rV+ueJW3bnIKng5VCBXo1/+6g1ilkZcIZPUmK7nJuhnVKNgk9LvdTwhLIxHfKuRUjbvxsvuqUnFlnQMJY26eQzN3fExmNjJlEge2MKI7Mcm1m/lfrphe+ZlQSYpcscVHYSoJxmR2NxkIzRnKiQXKtLC7EjaimjK06ZRsCN7yavQqlU9y3cXlfp1HkcRTuAUzsGDS6jDLTSgCQyG8Ayv8OZI58V5dz4WrQUnzmGP3I+fwALsI2g</latexit><latexit sha1_base64="sAe26MpFncoK5AcSpzUnkA9I=">AB6nicbZBNS8NAEIYn9avWr6pHL4tF8FSIuix6MVjRfsBbSib7aZdutmE3YlSQn+CFw+KePUXefPfuG1z0NYXFh7emWFn3iCRwqDrfjuFtfWNza3idmlnd2/oHx41DJxqhlvsljGuhNQw6VQvIkCJe8kmtMokLwdjG9m9fYj10bE6gEnCfcjOlQiFIyite6f+rV+ueJW3bnIKng5VCBXo1/+6g1ilkZcIZPUmK7nJuhnVKNgk9LvdTwhLIxHfKuRUjbvxsvuqUnFlnQMJY26eQzN3fExmNjJlEge2MKI7Mcm1m/lfrphe+ZlQSYpcscVHYSoJxmR2NxkIzRnKiQXKtLC7EjaimjK06ZRsCN7yavQqlU9y3cXlfp1HkcRTuAUzsGDS6jDLTSgCQyG8Ayv8OZI58V5dz4WrQUnzmGP3I+fwALsI2g</latexit><latexit sha1_base64="sAe26MpFncoK5AcSpzUnkA9I=">AB6nicbZBNS8NAEIYn9avWr6pHL4tF8FSIuix6MVjRfsBbSib7aZdutmE3YlSQn+CFw+KePUXefPfuG1z0NYXFh7emWFn3iCRwqDrfjuFtfWNza3idmlnd2/oHx41DJxqhlvsljGuhNQw6VQvIkCJe8kmtMokLwdjG9m9fYj10bE6gEnCfcjOlQiFIyite6f+rV+ueJW3bnIKng5VCBXo1/+6g1ilkZcIZPUmK7nJuhnVKNgk9LvdTwhLIxHfKuRUjbvxsvuqUnFlnQMJY26eQzN3fExmNjJlEge2MKI7Mcm1m/lfrphe+ZlQSYpcscVHYSoJxmR2NxkIzRnKiQXKtLC7EjaimjK06ZRsCN7yavQqlU9y3cXlfp1HkcRTuAUzsGDS6jDLTSgCQyG8Ayv8OZI58V5dz4WrQUnzmGP3I+fwALsI2g</latexit>

. . .

ResNet-34

7x7 AvgPool stride=1

wm

<latexit sha1_base64="3SltFZgdSbEcduFdJMJ4sVJM+s=">AB6nicbZBNSwMxEIZn61etX1WPXoJF8FR2RdBj0YvHirYW2qVk07QNTbJLMquUpT/BiwdFvPqLvPlvTNs9aOsLgYd3ZsjMGyVSWPT9b6+wsrq2vlHcLG1t7+zulfcPmjZODeMNFsvYtCJquRSaN1Cg5K3EcKoiyR+i0fW0/vDIjRWxvsdxwkNFB1r0BaPorLunruqWK37Vn4ksQ5BDBXLVu+WvTi9mqeIamaTWtgM/wTCjBgWTfFLqpJYnlI3ogLcdaq4DbPZqhNy4pwe6cfGPY1k5v6eyKiydqwi16koDu1ibWr+V2un2L8M6GTFLlm84/6qSQYk+ndpCcMZyjHDigzwu1K2JAaytClU3IhBIsnL0PzrBo4vj2v1K7yOIpwBMdwCgFcQA1uoA4NYDCAZ3iFN096L9679zFvLXj5zCH8kf5A2Ucjds=</latexit><latexit sha1_base64="3SltFZgdSbEcduFdJMJ4sVJM+s=">AB6nicbZBNSwMxEIZn61etX1WPXoJF8FR2RdBj0YvHirYW2qVk07QNTbJLMquUpT/BiwdFvPqLvPlvTNs9aOsLgYd3ZsjMGyVSWPT9b6+wsrq2vlHcLG1t7+zulfcPmjZODeMNFsvYtCJquRSaN1Cg5K3EcKoiyR+i0fW0/vDIjRWxvsdxwkNFB1r0BaPorLunruqWK37Vn4ksQ5BDBXLVu+WvTi9mqeIamaTWtgM/wTCjBgWTfFLqpJYnlI3ogLcdaq4DbPZqhNy4pwe6cfGPY1k5v6eyKiydqwi16koDu1ibWr+V2un2L8M6GTFLlm84/6qSQYk+ndpCcMZyjHDigzwu1K2JAaytClU3IhBIsnL0PzrBo4vj2v1K7yOIpwBMdwCgFcQA1uoA4NYDCAZ3iFN096L9679zFvLXj5zCH8kf5A2Ucjds=</latexit><latexit sha1_base64="3SltFZgdSbEcduFdJMJ4sVJM+s=">AB6nicbZBNSwMxEIZn61etX1WPXoJF8FR2RdBj0YvHirYW2qVk07QNTbJLMquUpT/BiwdFvPqLvPlvTNs9aOsLgYd3ZsjMGyVSWPT9b6+wsrq2vlHcLG1t7+zulfcPmjZODeMNFsvYtCJquRSaN1Cg5K3EcKoiyR+i0fW0/vDIjRWxvsdxwkNFB1r0BaPorLunruqWK37Vn4ksQ5BDBXLVu+WvTi9mqeIamaTWtgM/wTCjBgWTfFLqpJYnlI3ogLcdaq4DbPZqhNy4pwe6cfGPY1k5v6eyKiydqwi16koDu1ibWr+V2un2L8M6GTFLlm84/6qSQYk+ndpCcMZyjHDigzwu1K2JAaytClU3IhBIsnL0PzrBo4vj2v1K7yOIpwBMdwCgFcQA1uoA4NYDCAZ3iFN096L9679zFvLXj5zCH8kf5A2Ucjds=</latexit><latexit sha1_base64="3SltFZgdSbEcduFdJMJ4sVJM+s=">AB6nicbZBNSwMxEIZn61etX1WPXoJF8FR2RdBj0YvHirYW2qVk07QNTbJLMquUpT/BiwdFvPqLvPlvTNs9aOsLgYd3ZsjMGyVSWPT9b6+wsrq2vlHcLG1t7+zulfcPmjZODeMNFsvYtCJquRSaN1Cg5K3EcKoiyR+i0fW0/vDIjRWxvsdxwkNFB1r0BaPorLunruqWK37Vn4ksQ5BDBXLVu+WvTi9mqeIamaTWtgM/wTCjBgWTfFLqpJYnlI3ogLcdaq4DbPZqhNy4pwe6cfGPY1k5v6eyKiydqwi16koDu1ibWr+V2un2L8M6GTFLlm84/6qSQYk+ndpCcMZyjHDigzwu1K2JAaytClU3IhBIsnL0PzrBo4vj2v1K7yOIpwBMdwCgFcQA1uoA4NYDCAZ3iFN096L9679zFvLXj5zCH8kf5A2Ucjds=</latexit>

{

<latexit sha1_base64="tFROaEGhmnMpUlcEPFjR2z/BNI=">AB6XicbZBNS8NAEIYn9avWr6hHL4tF8FQSEfRY9OKxiv2ANpTNdtIu3WzC7kYof/AiwdFvPqPvPlv3LY5aOsLCw/vzLAzb5gKro3nfTultfWNza3ydmVnd2/wD08aukUwybLBGJ6oRUo+ASm4YbgZ1UIY1Dge1wfDurt59QaZ7IRzNJMYjpUPKIM2qs9dDL+27Vq3lzkVXwC6hCoUbf/eoNEpbFKA0TVOu76UmyKkynAmcVnqZxpSyMR1i16KkMeogn286JWfWGZAoUfZJQ+bu74mcxlpP4tB2xtSM9HJtZv5X62Ymug5yLtPMoGSLj6JMEJOQ2dlkwBUyIyYWKFPc7krYiCrKjA2nYkPwl09ehdZFzbd8f1mt3xRxlOETuEcfLiCOtxBA5rAIJneIU3Z+y8O/Ox6K15BQzx/BHzucPm4aNZQ=</latexit><latexit sha1_base64="tFROaEGhmnMpUlcEPFjR2z/BNI=">AB6XicbZBNS8NAEIYn9avWr6hHL4tF8FQSEfRY9OKxiv2ANpTNdtIu3WzC7kYof/AiwdFvPqPvPlv3LY5aOsLCw/vzLAzb5gKro3nfTultfWNza3ydmVnd2/wD08aukUwybLBGJ6oRUo+ASm4YbgZ1UIY1Dge1wfDurt59QaZ7IRzNJMYjpUPKIM2qs9dDL+27Vq3lzkVXwC6hCoUbf/eoNEpbFKA0TVOu76UmyKkynAmcVnqZxpSyMR1i16KkMeogn286JWfWGZAoUfZJQ+bu74mcxlpP4tB2xtSM9HJtZv5X62Ymug5yLtPMoGSLj6JMEJOQ2dlkwBUyIyYWKFPc7krYiCrKjA2nYkPwl09ehdZFzbd8f1mt3xRxlOETuEcfLiCOtxBA5rAIJneIU3Z+y8O/Ox6K15BQzx/BHzucPm4aNZQ=</latexit><latexit sha1_base64="tFROaEGhmnMpUlcEPFjR2z/BNI=">AB6XicbZBNS8NAEIYn9avWr6hHL4tF8FQSEfRY9OKxiv2ANpTNdtIu3WzC7kYof/AiwdFvPqPvPlv3LY5aOsLCw/vzLAzb5gKro3nfTultfWNza3ydmVnd2/wD08aukUwybLBGJ6oRUo+ASm4YbgZ1UIY1Dge1wfDurt59QaZ7IRzNJMYjpUPKIM2qs9dDL+27Vq3lzkVXwC6hCoUbf/eoNEpbFKA0TVOu76UmyKkynAmcVnqZxpSyMR1i16KkMeogn286JWfWGZAoUfZJQ+bu74mcxlpP4tB2xtSM9HJtZv5X62Ymug5yLtPMoGSLj6JMEJOQ2dlkwBUyIyYWKFPc7krYiCrKjA2nYkPwl09ehdZFzbd8f1mt3xRxlOETuEcfLiCOtxBA5rAIJneIU3Z+y8O/Ox6K15BQzx/BHzucPm4aNZQ=</latexit><latexit sha1_base64="tFROaEGhmnMpUlcEPFjR2z/BNI=">AB6XicbZBNS8NAEIYn9avWr6hHL4tF8FQSEfRY9OKxiv2ANpTNdtIu3WzC7kYof/AiwdFvPqPvPlv3LY5aOsLCw/vzLAzb5gKro3nfTultfWNza3ydmVnd2/wD08aukUwybLBGJ6oRUo+ASm4YbgZ1UIY1Dge1wfDurt59QaZ7IRzNJMYjpUPKIM2qs9dDL+27Vq3lzkVXwC6hCoUbf/eoNEpbFKA0TVOu76UmyKkynAmcVnqZxpSyMR1i16KkMeogn286JWfWGZAoUfZJQ+bu74mcxlpP4tB2xtSM9HJtZv5X62Ymug5yLtPMoGSLj6JMEJOQ2dlkwBUyIyYWKFPc7krYiCrKjA2nYkPwl09ehdZFzbd8f1mt3xRxlOETuEcfLiCOtxBA5rAIJneIU3Z+y8O/Ox6K15BQzx/BHzucPm4aNZQ=</latexit>

K − 1

<latexit sha1_base64="r9EyNDc4ZkK0RbMq+OXc0Let6lU=">AB6nicbZBNS8NAEIYn9avWr6pHL4tF8GJRNBj0YvgpaL9gDaUzXbSLt1swu5GKE/wYsHRbz6i7z5b9y2OWjrCwsP78ywM2+QCK6N6347hZXVtfWN4mZpa3tnd6+8f9DUcaoYNlgsYtUOqEbBJTYMNwLbiUIaBQJbwehmWm89odI8lo9mnKAf0YHkIWfUWOvh7szrlStu1Z2JLIOXQwVy1Xvlr24/ZmE0jBte54bmL8jCrDmcBJqZtqTCgb0QF2LEoaofaz2aoTcmKdPgljZ80ZOb+nshopPU4CmxnRM1QL9am5n+1TmrCKz/jMkNSjb/KEwFMTGZ3k36XCEzYmyBMsXtroQNqaLM2HRKNgRv8eRlaJ5XPcv3F5XadR5HEY7gGE7Bg0uowS3UoQEMBvAMr/DmCOfFeXc+5q0FJ585hD9yPn8AexuNQ=</latexit><latexit sha1_base64="r9EyNDc4ZkK0RbMq+OXc0Let6lU=">AB6nicbZBNS8NAEIYn9avWr6pHL4tF8GJRNBj0YvgpaL9gDaUzXbSLt1swu5GKE/wYsHRbz6i7z5b9y2OWjrCwsP78ywM2+QCK6N6347hZXVtfWN4mZpa3tnd6+8f9DUcaoYNlgsYtUOqEbBJTYMNwLbiUIaBQJbwehmWm89odI8lo9mnKAf0YHkIWfUWOvh7szrlStu1Z2JLIOXQwVy1Xvlr24/ZmE0jBte54bmL8jCrDmcBJqZtqTCgb0QF2LEoaofaz2aoTcmKdPgljZ80ZOb+nshopPU4CmxnRM1QL9am5n+1TmrCKz/jMkNSjb/KEwFMTGZ3k36XCEzYmyBMsXtroQNqaLM2HRKNgRv8eRlaJ5XPcv3F5XadR5HEY7gGE7Bg0uowS3UoQEMBvAMr/DmCOfFeXc+5q0FJ585hD9yPn8AexuNQ=</latexit><latexit sha1_base64="r9EyNDc4ZkK0RbMq+OXc0Let6lU=">AB6nicbZBNS8NAEIYn9avWr6pHL4tF8GJRNBj0YvgpaL9gDaUzXbSLt1swu5GKE/wYsHRbz6i7z5b9y2OWjrCwsP78ywM2+QCK6N6347hZXVtfWN4mZpa3tnd6+8f9DUcaoYNlgsYtUOqEbBJTYMNwLbiUIaBQJbwehmWm89odI8lo9mnKAf0YHkIWfUWOvh7szrlStu1Z2JLIOXQwVy1Xvlr24/ZmE0jBte54bmL8jCrDmcBJqZtqTCgb0QF2LEoaofaz2aoTcmKdPgljZ80ZOb+nshopPU4CmxnRM1QL9am5n+1TmrCKz/jMkNSjb/KEwFMTGZ3k36XCEzYmyBMsXtroQNqaLM2HRKNgRv8eRlaJ5XPcv3F5XadR5HEY7gGE7Bg0uowS3UoQEMBvAMr/DmCOfFeXc+5q0FJ585hD9yPn8AexuNQ=</latexit><latexit sha1_base64="r9EyNDc4ZkK0RbMq+OXc0Let6lU=">AB6nicbZBNS8NAEIYn9avWr6pHL4tF8GJRNBj0YvgpaL9gDaUzXbSLt1swu5GKE/wYsHRbz6i7z5b9y2OWjrCwsP78ywM2+QCK6N6347hZXVtfWN4mZpa3tnd6+8f9DUcaoYNlgsYtUOqEbBJTYMNwLbiUIaBQJbwehmWm89odI8lo9mnKAf0YHkIWfUWOvh7szrlStu1Z2JLIOXQwVy1Xvlr24/ZmE0jBte54bmL8jCrDmcBJqZtqTCgb0QF2LEoaofaz2aoTcmKdPgljZ80ZOb+nshopPU4CmxnRM1QL9am5n+1TmrCKz/jMkNSjb/KEwFMTGZ3k36XCEzYmyBMsXtroQNqaLM2HRKNgRv8eRlaJ5XPcv3F5XadR5HEY7gGE7Bg0uowS3UoQEMBvAMr/DmCOfFeXc+5q0FJ585hD9yPn8AexuNQ=</latexit>

bK−1

<latexit sha1_base64="LwOEvZG6tsrLM9p/zFnxW8ucU=">AB7nicbZBNS8NAEIYn9avWr6pHL4tF8GJpKDHohfBSwX7AW0om+2mXbrZhN2JUEJ/hBcPinj193jz37htc9DWFxYe3plhZ94gkcKg6347hbX1jc2t4nZpZ3dv/6B8eNQycaoZb7JYxroTUMOlULyJAiXvJrTKJC8HYxvZ/X2E9dGxOoRJwn3IzpUIhSMorXaQT+7v/Cm/XLFrbpzkVXwcqhArka/NUbxCyNuEImqTFdz03Qz6hGwSflnqp4QlYzrkXYuKRtz42XzdKTmzoCEsbZPIZm7vycyGhkziQLbGVEcmeXazPyv1k0xvPYzoZIUuWKLj8JUEozJ7HYyEJozlBMLlGlhdyVsRDVlaBMq2RC85ZNXoXVZ9Sw/1Cr1mzyOIpzAKZyDB1dQhztoQBMYjOEZXuHNSZwX5935WLQWnHzmGP7I+fwBshSPIg=</latexit><latexit sha1_base64="LwOEvZG6tsrLM9p/zFnxW8ucU=">AB7nicbZBNS8NAEIYn9avWr6pHL4tF8GJpKDHohfBSwX7AW0om+2mXbrZhN2JUEJ/hBcPinj193jz37htc9DWFxYe3plhZ94gkcKg6347hbX1jc2t4nZpZ3dv/6B8eNQycaoZb7JYxroTUMOlULyJAiXvJrTKJC8HYxvZ/X2E9dGxOoRJwn3IzpUIhSMorXaQT+7v/Cm/XLFrbpzkVXwcqhArka/NUbxCyNuEImqTFdz03Qz6hGwSflnqp4QlYzrkXYuKRtz42XzdKTmzoCEsbZPIZm7vycyGhkziQLbGVEcmeXazPyv1k0xvPYzoZIUuWKLj8JUEozJ7HYyEJozlBMLlGlhdyVsRDVlaBMq2RC85ZNXoXVZ9Sw/1Cr1mzyOIpzAKZyDB1dQhztoQBMYjOEZXuHNSZwX5935WLQWnHzmGP7I+fwBshSPIg=</latexit><latexit sha1_base64="LwOEvZG6tsrLM9p/zFnxW8ucU=">AB7nicbZBNS8NAEIYn9avWr6pHL4tF8GJpKDHohfBSwX7AW0om+2mXbrZhN2JUEJ/hBcPinj193jz37htc9DWFxYe3plhZ94gkcKg6347hbX1jc2t4nZpZ3dv/6B8eNQycaoZb7JYxroTUMOlULyJAiXvJrTKJC8HYxvZ/X2E9dGxOoRJwn3IzpUIhSMorXaQT+7v/Cm/XLFrbpzkVXwcqhArka/NUbxCyNuEImqTFdz03Qz6hGwSflnqp4QlYzrkXYuKRtz42XzdKTmzoCEsbZPIZm7vycyGhkziQLbGVEcmeXazPyv1k0xvPYzoZIUuWKLj8JUEozJ7HYyEJozlBMLlGlhdyVsRDVlaBMq2RC85ZNXoXVZ9Sw/1Cr1mzyOIpzAKZyDB1dQhztoQBMYjOEZXuHNSZwX5935WLQWnHzmGP7I+fwBshSPIg=</latexit><latexit sha1_base64="LwOEvZG6tsrLM9p/zFnxW8ucU=">AB7nicbZBNS8NAEIYn9avWr6pHL4tF8GJpKDHohfBSwX7AW0om+2mXbrZhN2JUEJ/hBcPinj193jz37htc9DWFxYe3plhZ94gkcKg6347hbX1jc2t4nZpZ3dv/6B8eNQycaoZb7JYxroTUMOlULyJAiXvJrTKJC8HYxvZ/X2E9dGxOoRJwn3IzpUIhSMorXaQT+7v/Cm/XLFrbpzkVXwcqhArka/NUbxCyNuEImqTFdz03Qz6hGwSflnqp4QlYzrkXYuKRtz42XzdKTmzoCEsbZPIZm7vycyGhkziQLbGVEcmeXazPyv1k0xvPYzoZIUuWKLj8JUEozJ7HYyEJozlBMLlGlhdyVsRDVlaBMq2RC85ZNXoXVZ9Sw/1Cr1mzyOIpzAKZyDB1dQhztoQBMYjOEZXuHNSZwX5935WLQWnHzmGP7I+fwBshSPIg=</latexit>

b P(yi > r1)

<latexit sha1_base64="JEWZgW/c9FbzIeNeK3w5m6yuKL4=">AB/XicbZDLSsNAFIYn9VbrLV52boJFqJuSiKArKbpxWcFeoA1hMjlph04mYWaixFB8FTcuFHre7jzbZy2WjrDwMf/zmHc+b3E0alsu1vo7S0vLK6Vl6vbGxube+Yu3tGaeCQIvELBZdH0tglENLUcWgmwjAkc+g4+uJ/XOPQhJY36nsgTcCA84DSnBSluedB/oAEMscqb41rm0UvhOSeWbXr9lTWIjgFVFGhpmd+9YOYpBFwRiWsufYiXJzLBQlDMaVfiohwWSEB9DTyHE0s2n14+tY+0EVhgL/biypu7viRxHUmaRrzsjrIZyvjYx/6v1UhVeuDnlSaqAk9miMGWiq1JFZABRDFMg2YCKpvtcgQC0yUDqyiQ3Dmv7wI7dO6o/n2rNq4KuIo0N0hGrIQeogW5QE7UQY/oGb2iN+PJeDHejY9Za8koZvbRHxmfP9pzlM8=</latexit><latexit sha1_base64="JEWZgW/c9FbzIeNeK3w5m6yuKL4=">AB/XicbZDLSsNAFIYn9VbrLV52boJFqJuSiKArKbpxWcFeoA1hMjlph04mYWaixFB8FTcuFHre7jzbZy2WjrDwMf/zmHc+b3E0alsu1vo7S0vLK6Vl6vbGxube+Yu3tGaeCQIvELBZdH0tglENLUcWgmwjAkc+g4+uJ/XOPQhJY36nsgTcCA84DSnBSluedB/oAEMscqb41rm0UvhOSeWbXr9lTWIjgFVFGhpmd+9YOYpBFwRiWsufYiXJzLBQlDMaVfiohwWSEB9DTyHE0s2n14+tY+0EVhgL/biypu7viRxHUmaRrzsjrIZyvjYx/6v1UhVeuDnlSaqAk9miMGWiq1JFZABRDFMg2YCKpvtcgQC0yUDqyiQ3Dmv7wI7dO6o/n2rNq4KuIo0N0hGrIQeogW5QE7UQY/oGb2iN+PJeDHejY9Za8koZvbRHxmfP9pzlM8=</latexit><latexit sha1_base64="JEWZgW/c9FbzIeNeK3w5m6yuKL4=">AB/XicbZDLSsNAFIYn9VbrLV52boJFqJuSiKArKbpxWcFeoA1hMjlph04mYWaixFB8FTcuFHre7jzbZy2WjrDwMf/zmHc+b3E0alsu1vo7S0vLK6Vl6vbGxube+Yu3tGaeCQIvELBZdH0tglENLUcWgmwjAkc+g4+uJ/XOPQhJY36nsgTcCA84DSnBSluedB/oAEMscqb41rm0UvhOSeWbXr9lTWIjgFVFGhpmd+9YOYpBFwRiWsufYiXJzLBQlDMaVfiohwWSEB9DTyHE0s2n14+tY+0EVhgL/biypu7viRxHUmaRrzsjrIZyvjYx/6v1UhVeuDnlSaqAk9miMGWiq1JFZABRDFMg2YCKpvtcgQC0yUDqyiQ3Dmv7wI7dO6o/n2rNq4KuIo0N0hGrIQeogW5QE7UQY/oGb2iN+PJeDHejY9Za8koZvbRHxmfP9pzlM8=</latexit><latexit sha1_base64="JEWZgW/c9FbzIeNeK3w5m6yuKL4=">AB/XicbZDLSsNAFIYn9VbrLV52boJFqJuSiKArKbpxWcFeoA1hMjlph04mYWaixFB8FTcuFHre7jzbZy2WjrDwMf/zmHc+b3E0alsu1vo7S0vLK6Vl6vbGxube+Yu3tGaeCQIvELBZdH0tglENLUcWgmwjAkc+g4+uJ/XOPQhJY36nsgTcCA84DSnBSluedB/oAEMscqb41rm0UvhOSeWbXr9lTWIjgFVFGhpmd+9YOYpBFwRiWsufYiXJzLBQlDMaVfiohwWSEB9DTyHE0s2n14+tY+0EVhgL/biypu7viRxHUmaRrzsjrIZyvjYx/6v1UhVeuDnlSaqAk9miMGWiq1JFZABRDFMg2YCKpvtcgQC0yUDqyiQ3Dmv7wI7dO6o/n2rNq4KuIo0N0hGrIQeogW5QE7UQY/oGb2iN+PJeDHejY9Za8koZvbRHxmfP9pzlM8=</latexit>

b P(yi > r2)

<latexit sha1_base64="TPnmRaLx/eL+eApEw1jpI9oVN4=">AB/XicbZDLSsNAFIZPvNZ6i5edm2AR6qYkRdCVFN24rGAv0IYwmUzaoZNJmJkoMRfxY0LRdz6Hu58G6dtFtr6w8DHf87hnPn9hFGpbPvbWFpeWV1bL2UN7e2d3bNvf2jFOBSQvHLBZdH0nCKCctRUj3UQFPmMdPzR9aTeuSdC0pjfqSwhboQGnIYUI6UtzsP9CADJHKm+Nq5tFL4dVPbNi1+yprEVwCqhAoaZnfvWDGKcR4QozJGXPsRPl5kgoihkZl/upJAnCIzQgPY0cRUS6+fT6sXWincAKY6EfV9bU/T2Ro0jKLPJ1Z4TUM7XJuZ/tV6qwgs3pzxJFeF4tihMmaViaxKFVBsGKZBoQF1bdaeIgEwkoHVtYhOPNfXoR2veZovj2rNK6KOEpwBMdQBQfOoQE30IQWYHiEZ3iFN+PJeDHejY9Z65JRzBzAHxmfP9v4lNA=</latexit><latexit sha1_base64="TPnmRaLx/eL+eApEw1jpI9oVN4=">AB/XicbZDLSsNAFIZPvNZ6i5edm2AR6qYkRdCVFN24rGAv0IYwmUzaoZNJmJkoMRfxY0LRdz6Hu58G6dtFtr6w8DHf87hnPn9hFGpbPvbWFpeWV1bL2UN7e2d3bNvf2jFOBSQvHLBZdH0nCKCctRUj3UQFPmMdPzR9aTeuSdC0pjfqSwhboQGnIYUI6UtzsP9CADJHKm+Nq5tFL4dVPbNi1+yprEVwCqhAoaZnfvWDGKcR4QozJGXPsRPl5kgoihkZl/upJAnCIzQgPY0cRUS6+fT6sXWincAKY6EfV9bU/T2Ro0jKLPJ1Z4TUM7XJuZ/tV6qwgs3pzxJFeF4tihMmaViaxKFVBsGKZBoQF1bdaeIgEwkoHVtYhOPNfXoR2veZovj2rNK6KOEpwBMdQBQfOoQE30IQWYHiEZ3iFN+PJeDHejY9Z65JRzBzAHxmfP9v4lNA=</latexit><latexit sha1_base64="TPnmRaLx/eL+eApEw1jpI9oVN4=">AB/XicbZDLSsNAFIZPvNZ6i5edm2AR6qYkRdCVFN24rGAv0IYwmUzaoZNJmJkoMRfxY0LRdz6Hu58G6dtFtr6w8DHf87hnPn9hFGpbPvbWFpeWV1bL2UN7e2d3bNvf2jFOBSQvHLBZdH0nCKCctRUj3UQFPmMdPzR9aTeuSdC0pjfqSwhboQGnIYUI6UtzsP9CADJHKm+Nq5tFL4dVPbNi1+yprEVwCqhAoaZnfvWDGKcR4QozJGXPsRPl5kgoihkZl/upJAnCIzQgPY0cRUS6+fT6sXWincAKY6EfV9bU/T2Ro0jKLPJ1Z4TUM7XJuZ/tV6qwgs3pzxJFeF4tihMmaViaxKFVBsGKZBoQF1bdaeIgEwkoHVtYhOPNfXoR2veZovj2rNK6KOEpwBMdQBQfOoQE30IQWYHiEZ3iFN+PJeDHejY9Z65JRzBzAHxmfP9v4lNA=</latexit><latexit sha1_base64="TPnmRaLx/eL+eApEw1jpI9oVN4=">AB/XicbZDLSsNAFIZPvNZ6i5edm2AR6qYkRdCVFN24rGAv0IYwmUzaoZNJmJkoMRfxY0LRdz6Hu58G6dtFtr6w8DHf87hnPn9hFGpbPvbWFpeWV1bL2UN7e2d3bNvf2jFOBSQvHLBZdH0nCKCctRUj3UQFPmMdPzR9aTeuSdC0pjfqSwhboQGnIYUI6UtzsP9CADJHKm+Nq5tFL4dVPbNi1+yprEVwCqhAoaZnfvWDGKcR4QozJGXPsRPl5kgoihkZl/upJAnCIzQgPY0cRUS6+fT6sXWincAKY6EfV9bU/T2Ro0jKLPJ1Z4TUM7XJuZ/tV6qwgs3pzxJFeF4tihMmaViaxKFVBsGKZBoQF1bdaeIgEwkoHVtYhOPNfXoR2veZovj2rNK6KOEpwBMdQBQfOoQE30IQWYHiEZ3iFN+PJeDHejY9Z65JRzBzAHxmfP9v4lNA=</latexit>

b P(yi > rK−1)

<latexit sha1_base64="EsrSGPlu5mWGfLIJjs5E47reN0Y=">ACAXicbZDLSsNAFIYn9VbrLepGcDNYhLqwJCLoSopuBDcV7AXaECaTSTt0MgkzEyWEuPFV3LhQxK1v4c63cdpmoa0/DHz85xzOnN+LGZXKsr6N0sLi0vJKebWytr6xuWVu7RlAhMWjhikeh6SBJGOWkpqhjpxoKg0GOk42uxvXOPRGSRvxOpTFxQjTgNKAYKW25l7/gfpkiFTWzGupSy+Em90c2/mRa1atujURnAe7gCo1HTNr74f4SQkXGpOzZVqycDAlFMSN5pZ9IEiM8QgPS08hRSKSTS7I4aF2fBhEQj+u4MT9PZGhUMo09HRniNRQztbG5n+1XqKCcyejPE4U4Xi6KEgYVBEcxwF9KghWLNWAsKD6rxAPkUBY6dAqOgR79uR5aJ/Ubc23p9XGZRFHGeyDA1ADNjgDXANmqAFMHgEz+AVvBlPxovxbnxMW0tGMbML/sj4/AG94JZn</latexit><latexit sha1_base64="EsrSGPlu5mWGfLIJjs5E47reN0Y=">ACAXicbZDLSsNAFIYn9VbrLepGcDNYhLqwJCLoSopuBDcV7AXaECaTSTt0MgkzEyWEuPFV3LhQxK1v4c63cdpmoa0/DHz85xzOnN+LGZXKsr6N0sLi0vJKebWytr6xuWVu7RlAhMWjhikeh6SBJGOWkpqhjpxoKg0GOk42uxvXOPRGSRvxOpTFxQjTgNKAYKW25l7/gfpkiFTWzGupSy+Em90c2/mRa1atujURnAe7gCo1HTNr74f4SQkXGpOzZVqycDAlFMSN5pZ9IEiM8QgPS08hRSKSTS7I4aF2fBhEQj+u4MT9PZGhUMo09HRniNRQztbG5n+1XqKCcyejPE4U4Xi6KEgYVBEcxwF9KghWLNWAsKD6rxAPkUBY6dAqOgR79uR5aJ/Ubc23p9XGZRFHGeyDA1ADNjgDXANmqAFMHgEz+AVvBlPxovxbnxMW0tGMbML/sj4/AG94JZn</latexit><latexit sha1_base64="EsrSGPlu5mWGfLIJjs5E47reN0Y=">ACAXicbZDLSsNAFIYn9VbrLepGcDNYhLqwJCLoSopuBDcV7AXaECaTSTt0MgkzEyWEuPFV3LhQxK1v4c63cdpmoa0/DHz85xzOnN+LGZXKsr6N0sLi0vJKebWytr6xuWVu7RlAhMWjhikeh6SBJGOWkpqhjpxoKg0GOk42uxvXOPRGSRvxOpTFxQjTgNKAYKW25l7/gfpkiFTWzGupSy+Em90c2/mRa1atujURnAe7gCo1HTNr74f4SQkXGpOzZVqycDAlFMSN5pZ9IEiM8QgPS08hRSKSTS7I4aF2fBhEQj+u4MT9PZGhUMo09HRniNRQztbG5n+1XqKCcyejPE4U4Xi6KEgYVBEcxwF9KghWLNWAsKD6rxAPkUBY6dAqOgR79uR5aJ/Ubc23p9XGZRFHGeyDA1ADNjgDXANmqAFMHgEz+AVvBlPxovxbnxMW0tGMbML/sj4/AG94JZn</latexit><latexit sha1_base64="EsrSGPlu5mWGfLIJjs5E47reN0Y=">ACAXicbZDLSsNAFIYn9VbrLepGcDNYhLqwJCLoSopuBDcV7AXaECaTSTt0MgkzEyWEuPFV3LhQxK1v4c63cdpmoa0/DHz85xzOnN+LGZXKsr6N0sLi0vJKebWytr6xuWVu7RlAhMWjhikeh6SBJGOWkpqhjpxoKg0GOk42uxvXOPRGSRvxOpTFxQjTgNKAYKW25l7/gfpkiFTWzGupSy+Em90c2/mRa1atujURnAe7gCo1HTNr74f4SQkXGpOzZVqycDAlFMSN5pZ9IEiM8QgPS08hRSKSTS7I4aF2fBhEQj+u4MT9PZGhUMo09HRniNRQztbG5n+1XqKCcyejPE4U4Xi6KEgYVBEcxwF9KghWLNWAsKD6rxAPkUBY6dAqOgR79uR5aJ/Ubc23p9XGZRFHGeyDA1ADNjgDXANmqAFMHgEz+AVvBlPxovxbnxMW0tGMbML/sj4/AG94JZn</latexit>

{

<latexit sha1_base64="tFROaEGhmnMpUlcEPFjR2z/BNI=">AB6XicbZBNS8NAEIYn9avWr6hHL4tF8FQSEfRY9OKxiv2ANpTNdtIu3WzC7kYof/AiwdFvPqPvPlv3LY5aOsLCw/vzLAzb5gKro3nfTultfWNza3ydmVnd2/wD08aukUwybLBGJ6oRUo+ASm4YbgZ1UIY1Dge1wfDurt59QaZ7IRzNJMYjpUPKIM2qs9dDL+27Vq3lzkVXwC6hCoUbf/eoNEpbFKA0TVOu76UmyKkynAmcVnqZxpSyMR1i16KkMeogn286JWfWGZAoUfZJQ+bu74mcxlpP4tB2xtSM9HJtZv5X62Ymug5yLtPMoGSLj6JMEJOQ2dlkwBUyIyYWKFPc7krYiCrKjA2nYkPwl09ehdZFzbd8f1mt3xRxlOETuEcfLiCOtxBA5rAIJneIU3Z+y8O/Ox6K15BQzx/BHzucPm4aNZQ=</latexit><latexit sha1_base64="tFROaEGhmnMpUlcEPFjR2z/BNI=">AB6XicbZBNS8NAEIYn9avWr6hHL4tF8FQSEfRY9OKxiv2ANpTNdtIu3WzC7kYof/AiwdFvPqPvPlv3LY5aOsLCw/vzLAzb5gKro3nfTultfWNza3ydmVnd2/wD08aukUwybLBGJ6oRUo+ASm4YbgZ1UIY1Dge1wfDurt59QaZ7IRzNJMYjpUPKIM2qs9dDL+27Vq3lzkVXwC6hCoUbf/eoNEpbFKA0TVOu76UmyKkynAmcVnqZxpSyMR1i16KkMeogn286JWfWGZAoUfZJQ+bu74mcxlpP4tB2xtSM9HJtZv5X62Ymug5yLtPMoGSLj6JMEJOQ2dlkwBUyIyYWKFPc7krYiCrKjA2nYkPwl09ehdZFzbd8f1mt3xRxlOETuEcfLiCOtxBA5rAIJneIU3Z+y8O/Ox6K15BQzx/BHzucPm4aNZQ=</latexit><latexit sha1_base64="tFROaEGhmnMpUlcEPFjR2z/BNI=">AB6XicbZBNS8NAEIYn9avWr6hHL4tF8FQSEfRY9OKxiv2ANpTNdtIu3WzC7kYof/AiwdFvPqPvPlv3LY5aOsLCw/vzLAzb5gKro3nfTultfWNza3ydmVnd2/wD08aukUwybLBGJ6oRUo+ASm4YbgZ1UIY1Dge1wfDurt59QaZ7IRzNJMYjpUPKIM2qs9dDL+27Vq3lzkVXwC6hCoUbf/eoNEpbFKA0TVOu76UmyKkynAmcVnqZxpSyMR1i16KkMeogn286JWfWGZAoUfZJQ+bu74mcxlpP4tB2xtSM9HJtZv5X62Ymug5yLtPMoGSLj6JMEJOQ2dlkwBUyIyYWKFPc7krYiCrKjA2nYkPwl09ehdZFzbd8f1mt3xRxlOETuEcfLiCOtxBA5rAIJneIU3Z+y8O/Ox6K15BQzx/BHzucPm4aNZQ=</latexit><latexit sha1_base64="tFROaEGhmnMpUlcEPFjR2z/BNI=">AB6XicbZBNS8NAEIYn9avWr6hHL4tF8FQSEfRY9OKxiv2ANpTNdtIu3WzC7kYof/AiwdFvPqPvPlv3LY5aOsLCw/vzLAzb5gKro3nfTultfWNza3ydmVnd2/wD08aukUwybLBGJ6oRUo+ASm4YbgZ1UIY1Dge1wfDurt59QaZ7IRzNJMYjpUPKIM2qs9dDL+27Vq3lzkVXwC6hCoUbf/eoNEpbFKA0TVOu76UmyKkynAmcVnqZxpSyMR1i16KkMeogn286JWfWGZAoUfZJQ+bu74mcxlpP4tB2xtSM9HJtZv5X62Ymug5yLtPMoGSLj6JMEJOQ2dlkwBUyIyYWKFPc7krYiCrKjA2nYkPwl09ehdZFzbd8f1mt3xRxlOETuEcfLiCOtxBA5rAIJneIU3Z+y8O/Ox6K15BQzx/BHzucPm4aNZQ=</latexit>

Tasks

{

<latexit sha1_base64="tFROaEGhmnMpUlcEPFjR2z/BNI=">AB6XicbZBNS8NAEIYn9avWr6hHL4tF8FQSEfRY9OKxiv2ANpTNdtIu3WzC7kYof/AiwdFvPqPvPlv3LY5aOsLCw/vzLAzb5gKro3nfTultfWNza3ydmVnd2/wD08aukUwybLBGJ6oRUo+ASm4YbgZ1UIY1Dge1wfDurt59QaZ7IRzNJMYjpUPKIM2qs9dDL+27Vq3lzkVXwC6hCoUbf/eoNEpbFKA0TVOu76UmyKkynAmcVnqZxpSyMR1i16KkMeogn286JWfWGZAoUfZJQ+bu74mcxlpP4tB2xtSM9HJtZv5X62Ymug5yLtPMoGSLj6JMEJOQ2dlkwBUyIyYWKFPc7krYiCrKjA2nYkPwl09ehdZFzbd8f1mt3xRxlOETuEcfLiCOtxBA5rAIJneIU3Z+y8O/Ox6K15BQzx/BHzucPm4aNZQ=</latexit><latexit sha1_base64="tFROaEGhmnMpUlcEPFjR2z/BNI=">AB6XicbZBNS8NAEIYn9avWr6hHL4tF8FQSEfRY9OKxiv2ANpTNdtIu3WzC7kYof/AiwdFvPqPvPlv3LY5aOsLCw/vzLAzb5gKro3nfTultfWNza3ydmVnd2/wD08aukUwybLBGJ6oRUo+ASm4YbgZ1UIY1Dge1wfDurt59QaZ7IRzNJMYjpUPKIM2qs9dDL+27Vq3lzkVXwC6hCoUbf/eoNEpbFKA0TVOu76UmyKkynAmcVnqZxpSyMR1i16KkMeogn286JWfWGZAoUfZJQ+bu74mcxlpP4tB2xtSM9HJtZv5X62Ymug5yLtPMoGSLj6JMEJOQ2dlkwBUyIyYWKFPc7krYiCrKjA2nYkPwl09ehdZFzbd8f1mt3xRxlOETuEcfLiCOtxBA5rAIJneIU3Z+y8O/Ox6K15BQzx/BHzucPm4aNZQ=</latexit><latexit sha1_base64="tFROaEGhmnMpUlcEPFjR2z/BNI=">AB6XicbZBNS8NAEIYn9avWr6hHL4tF8FQSEfRY9OKxiv2ANpTNdtIu3WzC7kYof/AiwdFvPqPvPlv3LY5aOsLCw/vzLAzb5gKro3nfTultfWNza3ydmVnd2/wD08aukUwybLBGJ6oRUo+ASm4YbgZ1UIY1Dge1wfDurt59QaZ7IRzNJMYjpUPKIM2qs9dDL+27Vq3lzkVXwC6hCoUbf/eoNEpbFKA0TVOu76UmyKkynAmcVnqZxpSyMR1i16KkMeogn286JWfWGZAoUfZJQ+bu74mcxlpP4tB2xtSM9HJtZv5X62Ymug5yLtPMoGSLj6JMEJOQ2dlkwBUyIyYWKFPc7krYiCrKjA2nYkPwl09ehdZFzbd8f1mt3xRxlOETuEcfLiCOtxBA5rAIJneIU3Z+y8O/Ox6K15BQzx/BHzucPm4aNZQ=</latexit><latexit sha1_base64="tFROaEGhmnMpUlcEPFjR2z/BNI=">AB6XicbZBNS8NAEIYn9avWr6hHL4tF8FQSEfRY9OKxiv2ANpTNdtIu3WzC7kYof/AiwdFvPqPvPlv3LY5aOsLCw/vzLAzb5gKro3nfTultfWNza3ydmVnd2/wD08aukUwybLBGJ6oRUo+ASm4YbgZ1UIY1Dge1wfDurt59QaZ7IRzNJMYjpUPKIM2qs9dDL+27Vq3lzkVXwC6hCoUbf/eoNEpbFKA0TVOu76UmyKkynAmcVnqZxpSyMR1i16KkMeogn286JWfWGZAoUfZJQ+bu74mcxlpP4tB2xtSM9HJtZv5X62Ymug5yLtPMoGSLj6JMEJOQ2dlkwBUyIyYWKFPc7krYiCrKjA2nYkPwl09ehdZFzbd8f1mt3xRxlOETuEcfLiCOtxBA5rAIJneIU3Z+y8O/Ox6K15BQzx/BHzucPm4aNZQ=</latexit>

a1

<latexit sha1_base64="i5nKWL2osYHJ2SU7NvALg/zpMfE=">AB6nicbZBNS8NAEIYn9avWr6hHL4tF8FQSEfRY9OKxov2ANpTJdtMu3WzC7kYoT/BiwdFvPqLvPlv3LY5aOsLCw/vzLAzb5gKro3nfTultfWNza3ydmVnd2/wD08aukU5Q1aSIS1QlRM8ElaxpuBOukimEcCtYOx7ezevuJKc0T+WgmKQtiHEoecYrGWg/Y9/tu1at5c5FV8AuoQqFG3/3qDRKaxUwaKlDru+lJshRGU4Fm1Z6mWYp0jEOWdeixJjpIJ+vOiVn1hmQKFH2SUPm7u+JHGOtJ3FoO2M0I71cm5n/1bqZia6DnMs0M0zSxUdRJohJyOxuMuCKUSMmFpAqbncldIQKqbHpVGwI/vLJq9C6qPmW7y+r9ZsijKcwCmcgw9XUIc7aEATKAzhGV7hzRHOi/PufCxaS04xcwx/5Hz+AOiZjYk=</latexit><latexit sha1_base64="i5nKWL2osYHJ2SU7NvALg/zpMfE=">AB6nicbZBNS8NAEIYn9avWr6hHL4tF8FQSEfRY9OKxov2ANpTJdtMu3WzC7kYoT/BiwdFvPqLvPlv3LY5aOsLCw/vzLAzb5gKro3nfTultfWNza3ydmVnd2/wD08aukU5Q1aSIS1QlRM8ElaxpuBOukimEcCtYOx7ezevuJKc0T+WgmKQtiHEoecYrGWg/Y9/tu1at5c5FV8AuoQqFG3/3qDRKaxUwaKlDru+lJshRGU4Fm1Z6mWYp0jEOWdeixJjpIJ+vOiVn1hmQKFH2SUPm7u+JHGOtJ3FoO2M0I71cm5n/1bqZia6DnMs0M0zSxUdRJohJyOxuMuCKUSMmFpAqbncldIQKqbHpVGwI/vLJq9C6qPmW7y+r9ZsijKcwCmcgw9XUIc7aEATKAzhGV7hzRHOi/PufCxaS04xcwx/5Hz+AOiZjYk=</latexit><latexit sha1_base64="i5nKWL2osYHJ2SU7NvALg/zpMfE=">AB6nicbZBNS8NAEIYn9avWr6hHL4tF8FQSEfRY9OKxov2ANpTJdtMu3WzC7kYoT/BiwdFvPqLvPlv3LY5aOsLCw/vzLAzb5gKro3nfTultfWNza3ydmVnd2/wD08aukU5Q1aSIS1QlRM8ElaxpuBOukimEcCtYOx7ezevuJKc0T+WgmKQtiHEoecYrGWg/Y9/tu1at5c5FV8AuoQqFG3/3qDRKaxUwaKlDru+lJshRGU4Fm1Z6mWYp0jEOWdeixJjpIJ+vOiVn1hmQKFH2SUPm7u+JHGOtJ3FoO2M0I71cm5n/1bqZia6DnMs0M0zSxUdRJohJyOxuMuCKUSMmFpAqbncldIQKqbHpVGwI/vLJq9C6qPmW7y+r9ZsijKcwCmcgw9XUIc7aEATKAzhGV7hzRHOi/PufCxaS04xcwx/5Hz+AOiZjYk=</latexit><latexit sha1_base64="i5nKWL2osYHJ2SU7NvALg/zpMfE=">AB6nicbZBNS8NAEIYn9avWr6hHL4tF8FQSEfRY9OKxov2ANpTJdtMu3WzC7kYoT/BiwdFvPqLvPlv3LY5aOsLCw/vzLAzb5gKro3nfTultfWNza3ydmVnd2/wD08aukU5Q1aSIS1QlRM8ElaxpuBOukimEcCtYOx7ezevuJKc0T+WgmKQtiHEoecYrGWg/Y9/tu1at5c5FV8AuoQqFG3/3qDRKaxUwaKlDru+lJshRGU4Fm1Z6mWYp0jEOWdeixJjpIJ+vOiVn1hmQKFH2SUPm7u+JHGOtJ3FoO2M0I71cm5n/1bqZia6DnMs0M0zSxUdRJohJyOxuMuCKUSMmFpAqbncldIQKqbHpVGwI/vLJq9C6qPmW7y+r9ZsijKcwCmcgw9XUIc7aEATKAzhGV7hzRHOi/PufCxaS04xcwx/5Hz+AOiZjYk=</latexit>

a2

<latexit sha1_base64="fHY8L0Lq6jayN2ybi5nRkuRgM8k=">AB6nicbZBNS8NAEIYn9avWr6pHL4tF8FSItRj0YvHivYD2lAm2027dLMJuxuhP4ELx4U8eov8ua/cdvmoK0vLDy8M8POvEiuDau+0UNja3tneKu6W9/YPDo/LxSVvHqaKsRWMRq26AmgkuWctwI1g3UQyjQLBOMLmd1ztPTGkey0czTZgf4UjykFM01nrAQW1QrhVdyGyDl4OFcjVHJS/+sOYphGThgrUue5ifEzVIZTwWalfqpZgnSCI9azKDFi2s8Wq87IhXWGJIyVfdKQhft7IsNI62kU2M4IzViv1ubmf7VeasJrP+MySQ2TdPlRmApiYjK/mwy5YtSIqQWkitdCR2jQmpsOiUbgrd68jq0a1XP8v1VpXGTx1GEMziHS/CgDg24gya0gMInuEV3hzhvDjvzseyteDkM6fwR87nD+odjYo=</latexit><latexit sha1_base64="fHY8L0Lq6jayN2ybi5nRkuRgM8k=">AB6nicbZBNS8NAEIYn9avWr6pHL4tF8FSItRj0YvHivYD2lAm2027dLMJuxuhP4ELx4U8eov8ua/cdvmoK0vLDy8M8POvEiuDau+0UNja3tneKu6W9/YPDo/LxSVvHqaKsRWMRq26AmgkuWctwI1g3UQyjQLBOMLmd1ztPTGkey0czTZgf4UjykFM01nrAQW1QrhVdyGyDl4OFcjVHJS/+sOYphGThgrUue5ifEzVIZTwWalfqpZgnSCI9azKDFi2s8Wq87IhXWGJIyVfdKQhft7IsNI62kU2M4IzViv1ubmf7VeasJrP+MySQ2TdPlRmApiYjK/mwy5YtSIqQWkitdCR2jQmpsOiUbgrd68jq0a1XP8v1VpXGTx1GEMziHS/CgDg24gya0gMInuEV3hzhvDjvzseyteDkM6fwR87nD+odjYo=</latexit><latexit sha1_base64="fHY8L0Lq6jayN2ybi5nRkuRgM8k=">AB6nicbZBNS8NAEIYn9avWr6pHL4tF8FSItRj0YvHivYD2lAm2027dLMJuxuhP4ELx4U8eov8ua/cdvmoK0vLDy8M8POvEiuDau+0UNja3tneKu6W9/YPDo/LxSVvHqaKsRWMRq26AmgkuWctwI1g3UQyjQLBOMLmd1ztPTGkey0czTZgf4UjykFM01nrAQW1QrhVdyGyDl4OFcjVHJS/+sOYphGThgrUue5ifEzVIZTwWalfqpZgnSCI9azKDFi2s8Wq87IhXWGJIyVfdKQhft7IsNI62kU2M4IzViv1ubmf7VeasJrP+MySQ2TdPlRmApiYjK/mwy5YtSIqQWkitdCR2jQmpsOiUbgrd68jq0a1XP8v1VpXGTx1GEMziHS/CgDg24gya0gMInuEV3hzhvDjvzseyteDkM6fwR87nD+odjYo=</latexit><latexit sha1_base64="ck8pdC+ekZH4nUmSP+ZG7r8lEyk=">AB2XicbZDNSgMxFIXv1L86Vq1rN8EiuCozbnQpuHFZwbZCO5RM5k4bmskMyR2hDH0BF25EfC93vo3pz0JbDwQ+zknIvSculLQUBN9ebWd3b/+gfugfNfzjk9Nmo2fz0gjsilzl5jnmFpXU2CVJCp8LgzyLFfbj6f0i7+gsTLXTzQrMr4WMtUCk7O6oyaraAdLMW2IVxDC9YaNb+GS7KDUJxa0dhEFBUcUNSaFw7g9LiwUXUz7GgUPNM7RtRxzi6dk7A0N+5oYkv394uKZ9bOstjdzDhN7Ga2MP/LBiWlt1EldVESarH6KC0Vo5wtdmaJNChIzRxwYaSblYkJN1yQa8Z3HYSbG29D7odOn4MoA7ncAFXEMIN3MEDdKALAhJ4hXdv4r15H6uat6tDP4I+/zBzjGijg=</latexit><latexit sha1_base64="5EnDFfMDBOio/2eay9En10Yo+dY=">AB3icbZBLSwMxFIXv+Ky1anXrJlgEV2WmG10KblxWtA9oh5J7ShmcyQ3BHK0J/gxoUi/it3/hvTx0JbDwQ+zknIvSfKlLTk+9/e1vbO7t5+6aB8WDk6PqmeVto2zY3AlkhVaroRt6ikxhZJUtjNDPIkUtiJnfzvPOMxspUP9E0wzDhIy1jKTg565EPGoNqza/7C7FNCFZQg5Wag+pXf5iKPEFNQnFre4GfUVhwQ1IonJX7ucWMiwkfYc+h5gnasFiMOmOXzhmyODXuaGIL9/eLgifWTpPI3Uw4je16Njf/y3o5xTdhIXWE2qx/CjOFaOUzfdmQ2lQkJo64MJINysTY264INdO2ZUQrK+8Ce1GPXD84EMJzuECriCAa7iFe2hCwSM4AXe4N1T3qv3saxry1v1dgZ/5H3+AM+5jDU=</latexit><latexit sha1_base64="5EnDFfMDBOio/2eay9En10Yo+dY=">AB3icbZBLSwMxFIXv+Ky1anXrJlgEV2WmG10KblxWtA9oh5J7ShmcyQ3BHK0J/gxoUi/it3/hvTx0JbDwQ+zknIvSfKlLTk+9/e1vbO7t5+6aB8WDk6PqmeVto2zY3AlkhVaroRt6ikxhZJUtjNDPIkUtiJnfzvPOMxspUP9E0wzDhIy1jKTg565EPGoNqza/7C7FNCFZQg5Wag+pXf5iKPEFNQnFre4GfUVhwQ1IonJX7ucWMiwkfYc+h5gnasFiMOmOXzhmyODXuaGIL9/eLgifWTpPI3Uw4je16Njf/y3o5xTdhIXWE2qx/CjOFaOUzfdmQ2lQkJo64MJINysTY264INdO2ZUQrK+8Ce1GPXD84EMJzuECriCAa7iFe2hCwSM4AXe4N1T3qv3saxry1v1dgZ/5H3+AM+5jDU=</latexit><latexit sha1_base64="p8tLOQbHVpAOVm/uqaDbqhYE8i4=">AB6nicbZBNS8NAEIYn9avWr6hHL4tF8FSXvRY9OKxov2ANpTJdtMu3WzC7kYoT/BiwdFvPqLvPlv3LY5aOsLCw/vzLAzb5gKro3nfTuljc2t7Z3ybmVv/+DwyD0+aeskU5S1aCIS1Q1RM8ElaxluBOumimEcCtYJ7fzeueJKc0T+WimKQtiHEkecYrGWg84qA/cqlfzFiLr4BdQhULNgfvVHyY0i5k0VKDWPd9LTZCjMpwKNqv0M81SpBMcsZ5FiTHTQb5YdUYurDMkUaLsk4Ys3N8TOcZaT+PQdsZoxnq1Njf/q/UyE10HOZdpZpiky4+iTBCTkPndZMgVo0ZMLSBV3O5K6BgVUmPTqdgQ/NWT16Fdr/mW71q46aIowxncA6X4MVNOAOmtACiN4hld4c4Tz4rw7H8vWklPMnMIfOZ8/6N2Nhg=</latexit><latexit sha1_base64="fHY8L0Lq6jayN2ybi5nRkuRgM8k=">AB6nicbZBNS8NAEIYn9avWr6pHL4tF8FSItRj0YvHivYD2lAm2027dLMJuxuhP4ELx4U8eov8ua/cdvmoK0vLDy8M8POvEiuDau+0UNja3tneKu6W9/YPDo/LxSVvHqaKsRWMRq26AmgkuWctwI1g3UQyjQLBOMLmd1ztPTGkey0czTZgf4UjykFM01nrAQW1QrhVdyGyDl4OFcjVHJS/+sOYphGThgrUue5ifEzVIZTwWalfqpZgnSCI9azKDFi2s8Wq87IhXWGJIyVfdKQhft7IsNI62kU2M4IzViv1ubmf7VeasJrP+MySQ2TdPlRmApiYjK/mwy5YtSIqQWkitdCR2jQmpsOiUbgrd68jq0a1XP8v1VpXGTx1GEMziHS/CgDg24gya0gMInuEV3hzhvDjvzseyteDkM6fwR87nD+odjYo=</latexit><latexit sha1_base64="fHY8L0Lq6jayN2ybi5nRkuRgM8k=">AB6nicbZBNS8NAEIYn9avWr6pHL4tF8FSItRj0YvHivYD2lAm2027dLMJuxuhP4ELx4U8eov8ua/cdvmoK0vLDy8M8POvEiuDau+0UNja3tneKu6W9/YPDo/LxSVvHqaKsRWMRq26AmgkuWctwI1g3UQyjQLBOMLmd1ztPTGkey0czTZgf4UjykFM01nrAQW1QrhVdyGyDl4OFcjVHJS/+sOYphGThgrUue5ifEzVIZTwWalfqpZgnSCI9azKDFi2s8Wq87IhXWGJIyVfdKQhft7IsNI62kU2M4IzViv1ubmf7VeasJrP+MySQ2TdPlRmApiYjK/mwy5YtSIqQWkitdCR2jQmpsOiUbgrd68jq0a1XP8v1VpXGTx1GEMziHS/CgDg24gya0gMInuEV3hzhvDjvzseyteDkM6fwR87nD+odjYo=</latexit><latexit sha1_base64="fHY8L0Lq6jayN2ybi5nRkuRgM8k=">AB6nicbZBNS8NAEIYn9avWr6pHL4tF8FSItRj0YvHivYD2lAm2027dLMJuxuhP4ELx4U8eov8ua/cdvmoK0vLDy8M8POvEiuDau+0UNja3tneKu6W9/YPDo/LxSVvHqaKsRWMRq26AmgkuWctwI1g3UQyjQLBOMLmd1ztPTGkey0czTZgf4UjykFM01nrAQW1QrhVdyGyDl4OFcjVHJS/+sOYphGThgrUue5ifEzVIZTwWalfqpZgnSCI9azKDFi2s8Wq87IhXWGJIyVfdKQhft7IsNI62kU2M4IzViv1ubmf7VeasJrP+MySQ2TdPlRmApiYjK/mwy5YtSIqQWkitdCR2jQmpsOiUbgrd68jq0a1XP8v1VpXGTx1GEMziHS/CgDg24gya0gMInuEV3hzhvDjvzseyteDkM6fwR87nD+odjYo=</latexit><latexit sha1_base64="fHY8L0Lq6jayN2ybi5nRkuRgM8k=">AB6nicbZBNS8NAEIYn9avWr6pHL4tF8FSItRj0YvHivYD2lAm2027dLMJuxuhP4ELx4U8eov8ua/cdvmoK0vLDy8M8POvEiuDau+0UNja3tneKu6W9/YPDo/LxSVvHqaKsRWMRq26AmgkuWctwI1g3UQyjQLBOMLmd1ztPTGkey0czTZgf4UjykFM01nrAQW1QrhVdyGyDl4OFcjVHJS/+sOYphGThgrUue5ifEzVIZTwWalfqpZgnSCI9azKDFi2s8Wq87IhXWGJIyVfdKQhft7IsNI62kU2M4IzViv1ubmf7VeasJrP+MySQ2TdPlRmApiYjK/mwy5YtSIqQWkitdCR2jQmpsOiUbgrd68jq0a1XP8v1VpXGTx1GEMziHS/CgDg24gya0gMInuEV3hzhvDjvzseyteDkM6fwR87nD+odjYo=</latexit><latexit sha1_base64="fHY8L0Lq6jayN2ybi5nRkuRgM8k=">AB6nicbZBNS8NAEIYn9avWr6pHL4tF8FSItRj0YvHivYD2lAm2027dLMJuxuhP4ELx4U8eov8ua/cdvmoK0vLDy8M8POvEiuDau+0UNja3tneKu6W9/YPDo/LxSVvHqaKsRWMRq26AmgkuWctwI1g3UQyjQLBOMLmd1ztPTGkey0czTZgf4UjykFM01nrAQW1QrhVdyGyDl4OFcjVHJS/+sOYphGThgrUue5ifEzVIZTwWalfqpZgnSCI9azKDFi2s8Wq87IhXWGJIyVfdKQhft7IsNI62kU2M4IzViv1ubmf7VeasJrP+MySQ2TdPlRmApiYjK/mwy5YtSIqQWkitdCR2jQmpsOiUbgrd68jq0a1XP8v1VpXGTx1GEMziHS/CgDg24gya0gMInuEV3hzhvDjvzseyteDkM6fwR87nD+odjYo=</latexit><latexit sha1_base64="fHY8L0Lq6jayN2ybi5nRkuRgM8k=">AB6nicbZBNS8NAEIYn9avWr6pHL4tF8FSItRj0YvHivYD2lAm2027dLMJuxuhP4ELx4U8eov8ua/cdvmoK0vLDy8M8POvEiuDau+0UNja3tneKu6W9/YPDo/LxSVvHqaKsRWMRq26AmgkuWctwI1g3UQyjQLBOMLmd1ztPTGkey0czTZgf4UjykFM01nrAQW1QrhVdyGyDl4OFcjVHJS/+sOYphGThgrUue5ifEzVIZTwWalfqpZgnSCI9azKDFi2s8Wq87IhXWGJIyVfdKQhft7IsNI62kU2M4IzViv1ubmf7VeasJrP+MySQ2TdPlRmApiYjK/mwy5YtSIqQWkitdCR2jQmpsOiUbgrd68jq0a1XP8v1VpXGTx1GEMziHS/CgDg24gya0gMInuEV3hzhvDjvzseyteDkM6fwR87nD+odjYo=</latexit>

am

<latexit sha1_base64="uGkhqy/YuhTKr1EIG2B45HDVWx0=">AB6nicbZBNSwMxEIZn61etX1WPXoJF8FR2RdBj0YvHivYD2qXMptk2NMkuSVYopT/BiwdFvPqLvPlvTNs9aOsLgYd3ZsjMG6WCG+v7315hbX1jc6u4XdrZ3ds/KB8eNU2SacoaNBGJbkdomOCKNSy3grVTzVBGgrWi0e2s3npi2vBEPdpxykKJA8VjTtE6wF7sleu+FV/LrIKQ4VyFXvlb+6/YRmkilLBRrTCfzUhPUlPBpqVuZliKdIQD1nGoUDITuarTsmZc/okTrR7ypK5+3tigtKYsYxcp0Q7NMu1mflfrZPZ+DqcJVmlim6+CjOBLEJmd1N+lwzasXYAVLN3a6EDlEjtS6dkgshWD5FZoX1cDx/WldpPHUYQTOIVzCOAKanAHdWgAhQE8wyu8ecJ78d69j0VrwctnjuGPvM8fQ5iNxQ=</latexit><latexit sha1_base64="uGkhqy/YuhTKr1EIG2B45HDVWx0=">AB6nicbZBNSwMxEIZn61etX1WPXoJF8FR2RdBj0YvHivYD2qXMptk2NMkuSVYopT/BiwdFvPqLvPlvTNs9aOsLgYd3ZsjMG6WCG+v7315hbX1jc6u4XdrZ3ds/KB8eNU2SacoaNBGJbkdomOCKNSy3grVTzVBGgrWi0e2s3npi2vBEPdpxykKJA8VjTtE6wF7sleu+FV/LrIKQ4VyFXvlb+6/YRmkilLBRrTCfzUhPUlPBpqVuZliKdIQD1nGoUDITuarTsmZc/okTrR7ypK5+3tigtKYsYxcp0Q7NMu1mflfrZPZ+DqcJVmlim6+CjOBLEJmd1N+lwzasXYAVLN3a6EDlEjtS6dkgshWD5FZoX1cDx/WldpPHUYQTOIVzCOAKanAHdWgAhQE8wyu8ecJ78d69j0VrwctnjuGPvM8fQ5iNxQ=</latexit><latexit sha1_base64="uGkhqy/YuhTKr1EIG2B45HDVWx0=">AB6nicbZBNSwMxEIZn61etX1WPXoJF8FR2RdBj0YvHivYD2qXMptk2NMkuSVYopT/BiwdFvPqLvPlvTNs9aOsLgYd3ZsjMG6WCG+v7315hbX1jc6u4XdrZ3ds/KB8eNU2SacoaNBGJbkdomOCKNSy3grVTzVBGgrWi0e2s3npi2vBEPdpxykKJA8VjTtE6wF7sleu+FV/LrIKQ4VyFXvlb+6/YRmkilLBRrTCfzUhPUlPBpqVuZliKdIQD1nGoUDITuarTsmZc/okTrR7ypK5+3tigtKYsYxcp0Q7NMu1mflfrZPZ+DqcJVmlim6+CjOBLEJmd1N+lwzasXYAVLN3a6EDlEjtS6dkgshWD5FZoX1cDx/WldpPHUYQTOIVzCOAKanAHdWgAhQE8wyu8ecJ78d69j0VrwctnjuGPvM8fQ5iNxQ=</latexit><latexit sha1_base64="uGkhqy/YuhTKr1EIG2B45HDVWx0=">AB6nicbZBNSwMxEIZn61etX1WPXoJF8FR2RdBj0YvHivYD2qXMptk2NMkuSVYopT/BiwdFvPqLvPlvTNs9aOsLgYd3ZsjMG6WCG+v7315hbX1jc6u4XdrZ3ds/KB8eNU2SacoaNBGJbkdomOCKNSy3grVTzVBGgrWi0e2s3npi2vBEPdpxykKJA8VjTtE6wF7sleu+FV/LrIKQ4VyFXvlb+6/YRmkilLBRrTCfzUhPUlPBpqVuZliKdIQD1nGoUDITuarTsmZc/okTrR7ypK5+3tigtKYsYxcp0Q7NMu1mflfrZPZ+DqcJVmlim6+CjOBLEJmd1N+lwzasXYAVLN3a6EDlEjtS6dkgshWD5FZoX1cDx/WldpPHUYQTOIVzCOAKanAHdWgAhQE8wyu8ecJ78d69j0VrwctnjuGPvM8fQ5iNxQ=</latexit>

b P(yi > rk) = s m X

j

wjaj + bk

• <latexit sha1_base64="ODmb6Tbw4kN7qLYf/45mjVP376E=">ACJXicbZDLSsNAFIYnXmu9RV26GSxCi1ASEXShUnTjsoK9QFPDZDp51JwszEUkJfxo2v4saFRQRXvorTNAtPXDg4/PYeb8XsSoVJb1ZSwtr6yurec28ptb2zu75t5+XYaxwKSGQxaKpockYTQgNUVI81IEMQ9Rhre4HbqN56IkDQMHtQoIm2OugH1KUZKS656Qxph/SQSqrj4sil8BoKd1CV1A6Hu12i46Mudt/5HDo9iHSfQI9d5B6JdcsWGUrLbgIdgYFkFXVNSdOJ8QxJ4HCDEnZsq1ItRMkFMWMjPNOLEmE8AB1SUtjgDiR7S9cgyPtdKBfih0Bwqm6u+NBHEpR9zTkxypnpz3puJ/XitW/kU7oUEUKxLg2UN+zKAK4TQy2KGCYMVGhAWVP8V4h4SCsdbF6HYM+fvAj107Kt+f6sULnJ4siBQ3AEisAG56AC7kAV1AGz+AVvIOJ8WK8GR/G52x0ych2DsCfMr5/AMaYo4o=</latexit><latexit sha1_base64="ODmb6Tbw4kN7qLYf/45mjVP376E=">ACJXicbZDLSsNAFIYnXmu9RV26GSxCi1ASEXShUnTjsoK9QFPDZDp51JwszEUkJfxo2v4saFRQRXvorTNAtPXDg4/PYeb8XsSoVJb1ZSwtr6yurec28ptb2zu75t5+XYaxwKSGQxaKpockYTQgNUVI81IEMQ9Rhre4HbqN56IkDQMHtQoIm2OugH1KUZKS656Qxph/SQSqrj4sil8BoKd1CV1A6Hu12i46Mudt/5HDo9iHSfQI9d5B6JdcsWGUrLbgIdgYFkFXVNSdOJ8QxJ4HCDEnZsq1ItRMkFMWMjPNOLEmE8AB1SUtjgDiR7S9cgyPtdKBfih0Bwqm6u+NBHEpR9zTkxypnpz3puJ/XitW/kU7oUEUKxLg2UN+zKAK4TQy2KGCYMVGhAWVP8V4h4SCsdbF6HYM+fvAj107Kt+f6sULnJ4siBQ3AEisAG56AC7kAV1AGz+AVvIOJ8WK8GR/G52x0ych2DsCfMr5/AMaYo4o=</latexit><latexit sha1_base64="ODmb6Tbw4kN7qLYf/45mjVP376E=">ACJXicbZDLSsNAFIYnXmu9RV26GSxCi1ASEXShUnTjsoK9QFPDZDp51JwszEUkJfxo2v4saFRQRXvorTNAtPXDg4/PYeb8XsSoVJb1ZSwtr6yurec28ptb2zu75t5+XYaxwKSGQxaKpockYTQgNUVI81IEMQ9Rhre4HbqN56IkDQMHtQoIm2OugH1KUZKS656Qxph/SQSqrj4sil8BoKd1CV1A6Hu12i46Mudt/5HDo9iHSfQI9d5B6JdcsWGUrLbgIdgYFkFXVNSdOJ8QxJ4HCDEnZsq1ItRMkFMWMjPNOLEmE8AB1SUtjgDiR7S9cgyPtdKBfih0Bwqm6u+NBHEpR9zTkxypnpz3puJ/XitW/kU7oUEUKxLg2UN+zKAK4TQy2KGCYMVGhAWVP8V4h4SCsdbF6HYM+fvAj107Kt+f6sULnJ4siBQ3AEisAG56AC7kAV1AGz+AVvIOJ8WK8GR/G52x0ych2DsCfMr5/AMaYo4o=</latexit><latexit sha1_base64="ODmb6Tbw4kN7qLYf/45mjVP376E=">ACJXicbZDLSsNAFIYnXmu9RV26GSxCi1ASEXShUnTjsoK9QFPDZDp51JwszEUkJfxo2v4saFRQRXvorTNAtPXDg4/PYeb8XsSoVJb1ZSwtr6yurec28ptb2zu75t5+XYaxwKSGQxaKpockYTQgNUVI81IEMQ9Rhre4HbqN56IkDQMHtQoIm2OugH1KUZKS656Qxph/SQSqrj4sil8BoKd1CV1A6Hu12i46Mudt/5HDo9iHSfQI9d5B6JdcsWGUrLbgIdgYFkFXVNSdOJ8QxJ4HCDEnZsq1ItRMkFMWMjPNOLEmE8AB1SUtjgDiR7S9cgyPtdKBfih0Bwqm6u+NBHEpR9zTkxypnpz3puJ/XitW/kU7oUEUKxLg2UN+zKAK4TQy2KGCYMVGhAWVP8V4h4SCsdbF6HYM+fvAj107Kt+f6sULnJ4siBQ3AEisAG56AC7kAV1AGz+AVvIOJ8WK8GR/G52x0ych2DsCfMr5/AMaYo4o=</latexit>

Age label

[30]

<latexit sha1_base64="M7vmw7VbJIUf0IZQmzGANVHtQBA=">AB63icbZDLSsNAFIZP6q3W9Wlm8EiuCqJCrosunFZwV4gDWUynbRDZyZhZiKU0Fdw40IRt76QO9/GSZqFtv4w8PGfc5hz/jDhTBvX/XYqa+sbm1vV7drO7t7+Qf3wqKvjVBHaITGPVT/EmnImacw2k/URSLkNeOL3L670nqjSL5aOZJTQeCxZxAg2ueVfusGw3nCbiG0Cl4JDSjVHta/BqOYpIJKQzjW2vfcxAQZVoYRTue1QapgskUj6lvUWJBdZAVu87RmXVGKIqVfdKgwv09kWGh9UyEtlNgM9HLtdz8r+anJroJMiaT1FBJFh9FKUcmRvnhaMQUJYbPLGCimN0VkQlWmBgbT82G4C2fvArdi6Zn+eGq0bot46jCZzCOXhwDS24hzZ0gMAEnuEV3hzhvDjvzseiteKUM8fwR87nD02+jb0=</latexit><latexit sha1_base64="M7vmw7VbJIUf0IZQmzGANVHtQBA=">AB63icbZDLSsNAFIZP6q3W9Wlm8EiuCqJCrosunFZwV4gDWUynbRDZyZhZiKU0Fdw40IRt76QO9/GSZqFtv4w8PGfc5hz/jDhTBvX/XYqa+sbm1vV7drO7t7+Qf3wqKvjVBHaITGPVT/EmnImacw2k/URSLkNeOL3L670nqjSL5aOZJTQeCxZxAg2ueVfusGw3nCbiG0Cl4JDSjVHta/BqOYpIJKQzjW2vfcxAQZVoYRTue1QapgskUj6lvUWJBdZAVu87RmXVGKIqVfdKgwv09kWGh9UyEtlNgM9HLtdz8r+anJroJMiaT1FBJFh9FKUcmRvnhaMQUJYbPLGCimN0VkQlWmBgbT82G4C2fvArdi6Zn+eGq0bot46jCZzCOXhwDS24hzZ0gMAEnuEV3hzhvDjvzseiteKUM8fwR87nD02+jb0=</latexit><latexit sha1_base64="M7vmw7VbJIUf0IZQmzGANVHtQBA=">AB63icbZDLSsNAFIZP6q3W9Wlm8EiuCqJCrosunFZwV4gDWUynbRDZyZhZiKU0Fdw40IRt76QO9/GSZqFtv4w8PGfc5hz/jDhTBvX/XYqa+sbm1vV7drO7t7+Qf3wqKvjVBHaITGPVT/EmnImacw2k/URSLkNeOL3L670nqjSL5aOZJTQeCxZxAg2ueVfusGw3nCbiG0Cl4JDSjVHta/BqOYpIJKQzjW2vfcxAQZVoYRTue1QapgskUj6lvUWJBdZAVu87RmXVGKIqVfdKgwv09kWGh9UyEtlNgM9HLtdz8r+anJroJMiaT1FBJFh9FKUcmRvnhaMQUJYbPLGCimN0VkQlWmBgbT82G4C2fvArdi6Zn+eGq0bot46jCZzCOXhwDS24hzZ0gMAEnuEV3hzhvDjvzseiteKUM8fwR87nD02+jb0=</latexit><latexit sha1_base64="M7vmw7VbJIUf0IZQmzGANVHtQBA=">AB63icbZDLSsNAFIZP6q3W9Wlm8EiuCqJCrosunFZwV4gDWUynbRDZyZhZiKU0Fdw40IRt76QO9/GSZqFtv4w8PGfc5hz/jDhTBvX/XYqa+sbm1vV7drO7t7+Qf3wqKvjVBHaITGPVT/EmnImacw2k/URSLkNeOL3L670nqjSL5aOZJTQeCxZxAg2ueVfusGw3nCbiG0Cl4JDSjVHta/BqOYpIJKQzjW2vfcxAQZVoYRTue1QapgskUj6lvUWJBdZAVu87RmXVGKIqVfdKgwv09kWGh9UyEtlNgM9HLtdz8r+anJroJMiaT1FBJFh9FKUcmRvnhaMQUJYbPLGCimN0VkQlWmBgbT82G4C2fvArdi6Zn+eGq0bot46jCZzCOXhwDS24hzZ0gMAEnuEV3hzhvDjvzseiteKUM8fwR87nD02+jb0=</latexit>

       1 1 . . .        ∈ ZK−1

2

<latexit sha1_base64="/XCD8nf7cWvcS8D4qcb5KAZy6w=">ACNXicbZBNSwMxEIazflu/qh69BIvgRdkVQY9FL4IeFKwtNrUk2WkNZrNLkhXLsn/Ki/Dkx48KOLVv2C2LaLWgYEn78yQmZclUhjr+8/e2PjE5NT0zGxpbn5hcam8vHJh4lRzqPFYxrBqAEpFNSsBIaiQYaMQl1dnNY1Ou3oI2I1bntJdCKaFeJjuDUOqldPiEMukJlLKJWi7u8FBDST3IbxtY48IvEBFT43YSJUJi4xzVj2WXe3rnKjreCvNQuV/xtvx94FIhVNAwTtvlRxLGPI1AWS6pMc3AT2wro9oKLiEvkdRAQvkN7ULToaIRmFbWvzrHG04JcSfWLpXFfXnREYjY3oRc53FquZvrRD/qzVT29lvZUIlqQXFBx91UoltjAsLcSg0cCt7DijXwu2K+TXVlFtndGFC8PfkUbjY2Q4cn+1WqgdDO2bQGlpHmyhAe6iKjtApqiGO7tETekVv3oP34r17H4PWMW84s4p+hf5BaWDqh4=</latexit><latexit sha1_base64="/XCD8nf7cWvcS8D4qcb5KAZy6w=">ACNXicbZBNSwMxEIazflu/qh69BIvgRdkVQY9FL4IeFKwtNrUk2WkNZrNLkhXLsn/Ki/Dkx48KOLVv2C2LaLWgYEn78yQmZclUhjr+8/e2PjE5NT0zGxpbn5hcam8vHJh4lRzqPFYxrBqAEpFNSsBIaiQYaMQl1dnNY1Ou3oI2I1bntJdCKaFeJjuDUOqldPiEMukJlLKJWi7u8FBDST3IbxtY48IvEBFT43YSJUJi4xzVj2WXe3rnKjreCvNQuV/xtvx94FIhVNAwTtvlRxLGPI1AWS6pMc3AT2wro9oKLiEvkdRAQvkN7ULToaIRmFbWvzrHG04JcSfWLpXFfXnREYjY3oRc53FquZvrRD/qzVT29lvZUIlqQXFBx91UoltjAsLcSg0cCt7DijXwu2K+TXVlFtndGFC8PfkUbjY2Q4cn+1WqgdDO2bQGlpHmyhAe6iKjtApqiGO7tETekVv3oP34r17H4PWMW84s4p+hf5BaWDqh4=</latexit><latexit sha1_base64="/XCD8nf7cWvcS8D4qcb5KAZy6w=">ACNXicbZBNSwMxEIazflu/qh69BIvgRdkVQY9FL4IeFKwtNrUk2WkNZrNLkhXLsn/Ki/Dkx48KOLVv2C2LaLWgYEn78yQmZclUhjr+8/e2PjE5NT0zGxpbn5hcam8vHJh4lRzqPFYxrBqAEpFNSsBIaiQYaMQl1dnNY1Ou3oI2I1bntJdCKaFeJjuDUOqldPiEMukJlLKJWi7u8FBDST3IbxtY48IvEBFT43YSJUJi4xzVj2WXe3rnKjreCvNQuV/xtvx94FIhVNAwTtvlRxLGPI1AWS6pMc3AT2wro9oKLiEvkdRAQvkN7ULToaIRmFbWvzrHG04JcSfWLpXFfXnREYjY3oRc53FquZvrRD/qzVT29lvZUIlqQXFBx91UoltjAsLcSg0cCt7DijXwu2K+TXVlFtndGFC8PfkUbjY2Q4cn+1WqgdDO2bQGlpHmyhAe6iKjtApqiGO7tETekVv3oP34r17H4PWMW84s4p+hf5BaWDqh4=</latexit><latexit sha1_base64="/XCD8nf7cWvcS8D4qcb5KAZy6w=">ACNXicbZBNSwMxEIazflu/qh69BIvgRdkVQY9FL4IeFKwtNrUk2WkNZrNLkhXLsn/Ki/Dkx48KOLVv2C2LaLWgYEn78yQmZclUhjr+8/e2PjE5NT0zGxpbn5hcam8vHJh4lRzqPFYxrBqAEpFNSsBIaiQYaMQl1dnNY1Ou3oI2I1bntJdCKaFeJjuDUOqldPiEMukJlLKJWi7u8FBDST3IbxtY48IvEBFT43YSJUJi4xzVj2WXe3rnKjreCvNQuV/xtvx94FIhVNAwTtvlRxLGPI1AWS6pMc3AT2wro9oKLiEvkdRAQvkN7ULToaIRmFbWvzrHG04JcSfWLpXFfXnREYjY3oRc53FquZvrRD/qzVT29lvZUIlqQXFBx91UoltjAsLcSg0cCt7DijXwu2K+TXVlFtndGFC8PfkUbjY2Q4cn+1WqgdDO2bQGlpHmyhAe6iKjtApqiGO7tETekVv3oP34r17H4PWMW84s4p+hf5BaWDqh4=</latexit>

Extended label

{

<latexit sha1_base64="tFROaEGhmnMpUlcEPFjR2z/BNI=">AB6XicbZBNS8NAEIYn9avWr6hHL4tF8FQSEfRY9OKxiv2ANpTNdtIu3WzC7kYof/AiwdFvPqPvPlv3LY5aOsLCw/vzLAzb5gKro3nfTultfWNza3ydmVnd2/wD08aukUwybLBGJ6oRUo+ASm4YbgZ1UIY1Dge1wfDurt59QaZ7IRzNJMYjpUPKIM2qs9dDL+27Vq3lzkVXwC6hCoUbf/eoNEpbFKA0TVOu76UmyKkynAmcVnqZxpSyMR1i16KkMeogn286JWfWGZAoUfZJQ+bu74mcxlpP4tB2xtSM9HJtZv5X62Ymug5yLtPMoGSLj6JMEJOQ2dlkwBUyIyYWKFPc7krYiCrKjA2nYkPwl09ehdZFzbd8f1mt3xRxlOETuEcfLiCOtxBA5rAIJneIU3Z+y8O/Ox6K15BQzx/BHzucPm4aNZQ=</latexit><latexit sha1_base64="tFROaEGhmnMpUlcEPFjR2z/BNI=">AB6XicbZBNS8NAEIYn9avWr6hHL4tF8FQSEfRY9OKxiv2ANpTNdtIu3WzC7kYof/AiwdFvPqPvPlv3LY5aOsLCw/vzLAzb5gKro3nfTultfWNza3ydmVnd2/wD08aukUwybLBGJ6oRUo+ASm4YbgZ1UIY1Dge1wfDurt59QaZ7IRzNJMYjpUPKIM2qs9dDL+27Vq3lzkVXwC6hCoUbf/eoNEpbFKA0TVOu76UmyKkynAmcVnqZxpSyMR1i16KkMeogn286JWfWGZAoUfZJQ+bu74mcxlpP4tB2xtSM9HJtZv5X62Ymug5yLtPMoGSLj6JMEJOQ2dlkwBUyIyYWKFPc7krYiCrKjA2nYkPwl09ehdZFzbd8f1mt3xRxlOETuEcfLiCOtxBA5rAIJneIU3Z+y8O/Ox6K15BQzx/BHzucPm4aNZQ=</latexit><latexit sha1_base64="tFROaEGhmnMpUlcEPFjR2z/BNI=">AB6XicbZBNS8NAEIYn9avWr6hHL4tF8FQSEfRY9OKxiv2ANpTNdtIu3WzC7kYof/AiwdFvPqPvPlv3LY5aOsLCw/vzLAzb5gKro3nfTultfWNza3ydmVnd2/wD08aukUwybLBGJ6oRUo+ASm4YbgZ1UIY1Dge1wfDurt59QaZ7IRzNJMYjpUPKIM2qs9dDL+27Vq3lzkVXwC6hCoUbf/eoNEpbFKA0TVOu76UmyKkynAmcVnqZxpSyMR1i16KkMeogn286JWfWGZAoUfZJQ+bu74mcxlpP4tB2xtSM9HJtZv5X62Ymug5yLtPMoGSLj6JMEJOQ2dlkwBUyIyYWKFPc7krYiCrKjA2nYkPwl09ehdZFzbd8f1mt3xRxlOETuEcfLiCOtxBA5rAIJneIU3Z+y8O/Ox6K15BQzx/BHzucPm4aNZQ=</latexit><latexit sha1_base64="tFROaEGhmnMpUlcEPFjR2z/BNI=">AB6XicbZBNS8NAEIYn9avWr6hHL4tF8FQSEfRY9OKxiv2ANpTNdtIu3WzC7kYof/AiwdFvPqPvPlv3LY5aOsLCw/vzLAzb5gKro3nfTultfWNza3ydmVnd2/wD08aukUwybLBGJ6oRUo+ASm4YbgZ1UIY1Dge1wfDurt59QaZ7IRzNJMYjpUPKIM2qs9dDL+27Vq3lzkVXwC6hCoUbf/eoNEpbFKA0TVOu76UmyKkynAmcVnqZxpSyMR1i16KkMeogn286JWfWGZAoUfZJQ+bu74mcxlpP4tB2xtSM9HJtZv5X62Ymug5yLtPMoGSLj6JMEJOQ2dlkwBUyIyYWKFPc7krYiCrKjA2nYkPwl09ehdZFzbd8f1mt3xRxlOETuEcfLiCOtxBA5rAIJneIU3Z+y8O/Ox6K15BQzx/BHzucPm4aNZQ=</latexit>

Label extension during training Wenzhi Cao, Vahid Mirjalili, and Sebastian Raschka. "Rank-consistent ordinal regression for neural networks." arXiv:1901.07884 (2019).

Deep Learning & Ordinal Data

Age prediction errors on the test sets without task importance Method Random Seed MORPH-2 MAE RMSE CE-CNN 3.40 4.88 1 3.39 4.87 2 3.37 4.87 AVG ± SD 3.39 ± 0.02 4.89 ± 0.01 3.98 OR-CNN

[1]

2.98 4.26 1 2.98 4.26 2 2.96 4.20 AVG ± SD 2.97 ± 0.01 4.24 ± 0.03 3.68 CORAL-CNN (ours) 2.68 3.75 1 2.63 3.66 2 2.61 3.64 AVG ± SD 2.64 ± 0.04 3.68 ± 0.06 3.49

SLIDE 35

35

Technologies

Challenges

"Hot" research areas

SLIDE 36

Transformers

36

Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez AN, Kaiser Ł, Polosukhin I. Attention is all you need. InAdvances in neural information processing systems 2017 (pp. 5998-6008).

Figure 1: The Transformer - model architecture.

SLIDE 37

37

https://www.theguardian.com/technology/2019/feb/14/elon- musk-backed-ai-writes-convincing-news-fiction

SLIDE 38

38

"Google researchers have trained a chatbot with uncannily good conversational skills. The bot, named Meena, is a 2.6 billion parameter language model trained on 341GB of text data, filtered from public domain social media conversations. Meena uses a seq2seq model (the same sort

• f technology that powers Google's "Smart Compose" feature in gmail),

paired with an Evolved Transformer encoder and decoder - it's interesting to see something like this depend so much on a component developed via neural architecture search."

https://mailchi.mp/jack-clark/import-ai-183-curve-fitting-conversation-with- meena-gans-paint-a-climate-change-future-and-what-compute-data-arbitrage- means-for-simulators

SLIDE 39

39

Human: Hi! Meena: Hey there! What's up? Human: Same old, Monday blues...what are you up to? Meena: I'm procrastinating on a paper I'm supposed to be writing. Human: are you thinking of pulling an all nighter? Meena: Possibly. What's your paper on? Human: Nah, i just do the grunge work around here...one day I wish to write a paper that will change the world! Meena: What's the topic? Human: not sure, any suggestions? Meena: Well what's the assignment on? Human: breathing under water

https://github.com/google-research/ google-research/blob/master/meena/ meena.txt

SLIDE 40

40

Image Source: https://medium.com/huggingface/distilbert-8cf3380435b5

SLIDE 41

41

https://github.com/nslatysheva/BERT_papers/blob/master/BERT_Papers.csv

SLIDE 42

• Despite LSTM, it may be hard to

memorize long sequences/ sentences (e.g., for language translation)

Recurrent Neural Networks

SLIDE 43

43

Attention Mechanism

• originally developed for language translation: 


Bahdanau, D., Cho, K., & Bengio, Y. (2014). Neural machine translation by jointly learning to align and translate. arXiv preprint arXiv:1409.0473.

"... allowing a model to automatically (soft-)search for parts of a source sentence that are relevant to predicting a target word ..."

10 20 30 40 50 60

Sentence length

5 10 15 20 25 30

BLEU score

RNNsearch-50 RNNsearch-30 RNNenc-50 RNNenc-30

Figure 2: The BLEU scores

• f the generated translations
• n the test set with respect

to the lengths of the sen- tences. The results are on the full test set which in- cludes sentences having un- known words to the models.

"traditional" 
 encoder+decoder 
 RNN

SLIDE 44

44

Attention Mechanism

• originally developed for language translation: 


Bahdanau, D., Cho, K., & Bengio, Y. (2014). Neural machine translation by jointly learning to align and translate. arXiv preprint arXiv:1409.0473.

"... allowing a model to automatically (soft-)search for parts of a source sentence that are relevant to predicting a target word ..."

Assign attention weight to each word to know how much "attention" the model should pay to each word (i.e., for each word, the network learns a "context")

SLIDE 45

45

Attention Mechanism

• originally developed for language translation: 


Bahdanau, D., Cho, K., & Bengio, Y. (2014). Neural machine translation by jointly learning to align and translate. arXiv preprint arXiv:1409.0473.

Assign attention weight to each word to know how much "attention" the model should pay to each word (i.e., for each word, the network learns a "context")

• Figure 1: The graphical illus-

tration of the proposed model trying to generate the t-th tar- get word yt given a source sentence (x1, x2, . . . , xT ). probability (4) ap- distinct annotations Each sequence the com- these (5)

hidden state attention weight

SLIDE 46

46

x1 x2 xT−1 xT x0

hF,1 hF,2

hF,T−1 hB,T−1

hB,2 hB,1

{

Bidirectional RNN

{

Added attention

c1

S0

S1 α1,1 α1,2 α1,T−1

...

c1 =

T

∑

t=1

α1,t ht

where the context vector is defined as

c1

... ...

̂ y1

(looks like a standard RNN but with context vectors as in-/output)

RNN Attention Mechanism

SLIDE 47

47

x1 x2 xT−1 xT x0

hF,1 hF,2

hF,T−1 hB,T−1

hB,2 hB,1

Computing attention weights

c1

S0

S1 α1,1 α1,2 α1,T−1

... ... ...

̂ y1

RNN Attention Mechanism

h′

t

St−1

neural net

et,t′ αt,t′ = exp(et,t′ ) ∑T

t′ =1 exp(et,t′ )

SLIDE 48

Computing attention weights

RNN Attention Mechanism

h′

t

St−1

neural net

et,t′ αt,t′ = exp(et,t′ ) ∑T

t′ =1 exp(et,t′ ) (a)

SLIDE 49

49

Self-Attention Mechanism

Attention Is All You Need

Ashish Vaswani∗ Google Brain avaswani@google.com Noam Shazeer∗ Google Brain noam@google.com Niki Parmar∗ Google Research nikip@google.com Jakob Uszkoreit∗ Google Research usz@google.com Llion Jones∗ Google Research llion@google.com Aidan N. Gomez∗ † University of Toronto aidan@cs.toronto.edu Łukasz Kaiser∗ Google Brain lukaszkaiser@google.com Illia Polosukhin∗ ‡ illia.polosukhin@gmail.com

https://arxiv.org/abs/1706.03762

Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., ... & Polosukhin, I. (2017). Attention is all you need. In Advances in neural information processing systems (pp. 5998-6008).

SLIDE 50

50

Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., ... & Polosukhin, I. (2017). Attention is all you need. In Advances in neural information processing systems (pp. 5998-6008).

Figure 1: The Transformer - model architecture.

SLIDE 51

51

https://deepmind.com/blog/article/alphastar-mastering-real-time-strategy-game-starcraft-ii

AlphaStar applies a transformer torso to the units

SLIDE 52

52

Transformer on a Diet

Chenguang Wang Zihao Ye Aston Zhang Zheng Zhang Alexander J. Smola Amazon Web Services {chgwang, yeziha, astonz, zhaz, smola}@amazon.com Abstract

Transformer has been widely used thanks to its ability to capture sequence information in an efficient way. However, recent develop- ments, such as BERT and GPT-2, deliver only heavy architectures with a focus on effective-

• ness. In this paper, we explore three carefully-

designed light Transformer architectures to fig- ure out whether the Transformer with less computations could produce competitive re-

size of the model. Therefore a light version of the standard Transformer architecture is expected to relieve the heavy computation issue and compress the model to ease the deployment in real world applications. In this paper, we carefully design several light Transformer architectures. The intuition behind the light Transformers is: preserving the Transformer connections that are useful to capture the essential sequence information, while omitting the ones with

arXiv:2002.06170v1 [cs.CL] 14 Feb 2020

(a) Full Transformer. (b) Dilated Transformer. (c) Dilated Transformer with memory. (d) Cascade Transformer.

SLIDE 53

53 Published as a conference paper at ICLR 2018

GRAPH ATTENTION NETWORKS

Petar Veliˇ ckovi´ c∗ Department of Computer Science and Technology University of Cambridge petar.velickovic@cst.cam.ac.uk Guillem Cucurull∗ Centre de Visi´

• per Computador, UAB

gcucurull@gmail.com Arantxa Casanova∗ Centre de Visi´

• per Computador, UAB

ar.casanova.8@gmail.com Adriana Romero Montr´ eal Institute for Learning Algorithms adriana.romero.soriano@umontreal.ca Pietro Li`

• Department of Computer Science and Technology

University of Cambridge pietro.lio@cst.cam.ac.uk Yoshua Bengio Montr´ eal Institute for Learning Algorithms yoshua.umontreal@gmail.com

https://arxiv.org/abs/1710.10903

↵ij ~ a softmaxj W~ hi W~ hj ~ h1 ~ h2 ~ h3 ~ h4 ~ h5 ~ h6 ~ ↵16 ~ ↵11 ~ ↵12 ~ ↵13 ~ ↵14 ~ ↵15 ~ h0

1

concat/avg Figure 1: Left: The attention mechanism a(W~ hi, W~ hj) employed by our model, parametrized by a weight vector ~ a 2 R2F 0, applying a LeakyReLU activation. Right: An illustration of multi- head attention (with K = 3 heads) by node 1 on its neighborhood. Different arrow styles and colors denote independent attention computations. The aggregated features from each head are concatenated or averaged to obtain ~ h0

1.

applying a nonlinearity, ):

SLIDE 54

54

Graph Neural Networks -- Why Graphs?

SLIDE 55

55

CC(=O)Oc1ccccc1C(=O)O 0 0 0 1 . . . 0 0 1

3D voxelization SMILES string Molecular fingerprint Graph encoding

+ +

=

A B C D

O C O

Sebastian Raschka and Benjamin Kaufman (2020)
 Machine learning and AI-based approaches for bioactive ligand discovery and GPCR-ligand recognition arXiv:2001.06545

SLIDE 56

56

https://github.com/rusty1s/pytorch_geometric

SLIDE 57

57

SLIDE 58

Self-Supervised Learning

58

SLIDE 59

Self-Supervised Learning: Image Colorization

59

Zhang R, Isola P , Efros AA. Colorful image colorization. InEuropean conference on computer vision 2016 Oct 8 (pp. 649-666). Springer, Cham.

Input Frame Target Colors Reference Frame Reference Colors Pointer Copy

• Fig. 1. Self-supervised Tracking: We capitalize on large amounts of unlabeled video

to learn a self-supervised model for tracking. The model learns to predict the target colors for a gray-scale input frame by pointing to a colorful reference frame, and copying the color channels. Although we train without ground-truth labels, experiments and visualizations suggest that tracking emerges automatically in this model.

Vondrick C, Shrivastava A, Fathi A, Guadarrama S, Murphy K. Tracking emerges by colorizing videos. InProceedings of the European Conference on Computer Vision (ECCV) 2018 (pp. 391-408). Larsson G, Maire M, Shakhnarovich G. Learning representations for automatic colorization. InEuropean Conference on Computer Vision 2016 Oct 8 (pp. 577-593). Springer, Cham.

SLIDE 60

Self-Supervised Learning: Inpainting

60

}

(a) Input context (b) Human artist (c) Context Encoder (L2 loss) (d) Context Encoder (L2 + Adversarial loss)

Pathak D, Krahenbuhl P , Donahue J, Darrell T, Efros AA. Context encoders: Feature learning by inpainting. InProceedings of the IEEE conference on computer vision and pattern recognition 2016 (pp. 2536-2544).

SLIDE 61

Self-Supervised Learning: Jigsaw Puzzles & Context Predictions

61

Noroozi M, Favaro P . Unsupervised learning of visual representations by solving jigsaw puzzles. In European Conference on Computer Vision 2016 Oct 8 (pp. 69-84). Springer, Cham. Doersch C, Gupta A, Efros AA. Unsupervised visual representation learning by context prediction. InProceedings

• f the IEEE International Conference on Computer Vision 2015

(pp. 1422-1430).

po- that g- fea- xt xam- ds , R- ver state-of-the- ascal-

_ _ ? ?

Example: Question 1: Question 2:

Figure 1. Our task for learning patch representations involves ran- domly sampling a patch (blue) and then one of eight possible neighbors (red). Can you guess the spatial configuration for the two pairs of patches? Note that the task is much easier once you have recognized the object! Answer key: Q1: Bottom right Q2: Top center

SLIDE 62

Self-Supervised Learning: Recognizing Artifacts

62

Jenni S, Favaro P . Self-supervised feature learning by learning to spot artifacts. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition 2018 (pp. 2733-2742).

Figure 1. A mixture of real images (green border) and images with synthetic artifacts (red border). Is a good object representation necessary to tell them apart?

SLIDE 63

63

https://www.pewresearch.org/fact-tank/2020/04/14/half-of-americans-have-decided-not-to- use-a-product-or-service-because-of-privacy-concerns/

SLIDE 64

64

Vahid Mirjalili, Sebastian Raschka, Arun Ross (2020)
 PrivacyNet: Semi-Adversarial Networks for Multi-attribute Face Privacy 
 arXiv:2001.00561

Generator Discriminator Attribute Classifier Auxiliary Face Matcher Target Labels Input Image

!"

# #′

Output Image Match score Gender, Age, Race Real / Synthesized

LD

<latexit sha1_base64="7+0IjgJn4DAPQ+LbgVg/SZ8/Sno=">AC0HicjVHLgRBFD3T3u/B0qZjIrHqVI9hxm6ChYUFYoaEiVSXQke/dFeLyUTE1g/Y8lXiD/gLt0pPwkKoTnfOvecU3Xv9ZLAzxRjbyVrYHBoeGR0bHxicmp6pjw7187iPBWyJeIgTo8nsnAj2RL+SqQR0kqegF8tC72tT5wxuZn4cHahuIjshv4j8c19wRVDnJOTqUvCgt3N3unVarjCHrdXYet1mTnW1UVtxKVhlbnWF2a7DzKqgWLtx+RUnOEMgRwhJCIoigNwZPQcwVDQlgHPcJSinyTl7jDOGlzYklicEKv6HtBu+MCjWivPTOjFnRKQG9KShtLpImJl1KsT7NPjfOGv3Nu2c89d269PcKr5BQhUtC/9L1mf/V6VoUztEwNfhU2IQXZ0oXHLTFX1z+1tVihwSwnR8RvmUYmGU/T7bRpOZ2nVvucm/G6ZG9V4U3Bwf+pY04P4U7d+DdtVxmePu1SrNjWLUo1jAIpZpnU0sY1dtMj7Gk94xou1b91a9bDF9UqFZp5/FjW4yfjiJTW</latexit><latexit sha1_base64="7+0IjgJn4DAPQ+LbgVg/SZ8/Sno=">AC0HicjVHLgRBFD3T3u/B0qZjIrHqVI9hxm6ChYUFYoaEiVSXQke/dFeLyUTE1g/Y8lXiD/gLt0pPwkKoTnfOvecU3Xv9ZLAzxRjbyVrYHBoeGR0bHxicmp6pjw7187iPBWyJeIgTo8nsnAj2RL+SqQR0kqegF8tC72tT5wxuZn4cHahuIjshv4j8c19wRVDnJOTqUvCgt3N3unVarjCHrdXYet1mTnW1UVtxKVhlbnWF2a7DzKqgWLtx+RUnOEMgRwhJCIoigNwZPQcwVDQlgHPcJSinyTl7jDOGlzYklicEKv6HtBu+MCjWivPTOjFnRKQG9KShtLpImJl1KsT7NPjfOGv3Nu2c89d269PcKr5BQhUtC/9L1mf/V6VoUztEwNfhU2IQXZ0oXHLTFX1z+1tVihwSwnR8RvmUYmGU/T7bRpOZ2nVvucm/G6ZG9V4U3Bwf+pY04P4U7d+DdtVxmePu1SrNjWLUo1jAIpZpnU0sY1dtMj7Gk94xou1b91a9bDF9UqFZp5/FjW4yfjiJTW</latexit><latexit sha1_base64="7+0IjgJn4DAPQ+LbgVg/SZ8/Sno=">AC0HicjVHLgRBFD3T3u/B0qZjIrHqVI9hxm6ChYUFYoaEiVSXQke/dFeLyUTE1g/Y8lXiD/gLt0pPwkKoTnfOvecU3Xv9ZLAzxRjbyVrYHBoeGR0bHxicmp6pjw7187iPBWyJeIgTo8nsnAj2RL+SqQR0kqegF8tC72tT5wxuZn4cHahuIjshv4j8c19wRVDnJOTqUvCgt3N3unVarjCHrdXYet1mTnW1UVtxKVhlbnWF2a7DzKqgWLtx+RUnOEMgRwhJCIoigNwZPQcwVDQlgHPcJSinyTl7jDOGlzYklicEKv6HtBu+MCjWivPTOjFnRKQG9KShtLpImJl1KsT7NPjfOGv3Nu2c89d269PcKr5BQhUtC/9L1mf/V6VoUztEwNfhU2IQXZ0oXHLTFX1z+1tVihwSwnR8RvmUYmGU/T7bRpOZ2nVvucm/G6ZG9V4U3Bwf+pY04P4U7d+DdtVxmePu1SrNjWLUo1jAIpZpnU0sY1dtMj7Gk94xou1b91a9bDF9UqFZp5/FjW4yfjiJTW</latexit><latexit sha1_base64="G30nvnJkyKBlKcCymNnrqTxJgV4=">ACtXicjVLSgMxFD0dX7VWrWs3g0VwVTJudCnowmUF+4BaZCZNa+y8TDJCKf6AWz9O/AP9C2/iCGoRzTAzJ+fec5KbmyiPpTaMvVS8peWV1bXqem2jXtvc2m7UuzorFBcdnsWZ6kehFrFMRcdIE4t+rkSYRLHoRdNTG+/dC6Vl6aWS6GSThJ5Vjy0BDVvm40WYu54S+CoARNlCNrPOMKI2TgKJBAIUhHCOEpmeAw5cUPMiVOEpIsLPKBG2oKyBGWExE7pO6HZoGRTmltP7dScVonpVaT0sU+ajPIUYbua7+KFc7bsb95z52n3NqN/VHolxBrcEPuX7jPzvzpbi8EYx64GSTXljrHV8dKlcKdid+5/qcqQ06cxSOK8LcKT/P2Xca7Wq3Zxu6+KvLtKyd8zK3wJvdJfU3+NnNRdA9bAWsFVwVLGLPRxQG49wgnO0SHLER7x5J15t97dxz3wKuWF2MG34el34YWM3A=</latexit><latexit sha1_base64="4MrC+jrKE9Y6Uixdn4yn3AW3+G8=">ACxXicjVLSsNAFD2Nr1qrVrdugkVwVRI3uhR04cJFfuAtshkOm1D83IyEaWI+BVu9avEP9C/8M6YglpEJyQ5c+49Z+bOHS8J/FQ5zmvBmptfWFwqLpdWyqtr65WNcjONM8lFg8dBLNseS0XgR6KhfBWIdiIFC71AtLzxkY63roVM/Ti6ULeJ6IVsGPkDnzNFVK8bMjXiLJic3l0eX1aqTs0xw54Fbg6qyEc9rygiz5icGQIRBEQ7AkNLTgQsHCXE9TIiThHwTF7hDibQZQnKYMSO6TukWSdnI5prz9SoOa0S0CtJaWOHNDHlScJ6NdvEM+Os2d+8J8ZT7+2W/l7uFRKrMCL2L908786XYvCAemBp9qSgyjq+O5S2ZORe/c/lKVIoeEOI37FJeEuVFOz9k2mtTUrs+WmfibydSsnvM8N8O73iU12P3ZzlnQ3Ku5Ts09c1DEFraxS23cxyFOUEeDLK/wiCc8W+fWjX/eRWsQn4nNvFtWA8f6vCTXg=</latexit><latexit sha1_base64="bVHLM6rdJ3hojfyxBqy7/nN4hpM=">ACxXicjVLJTsMwFJyGfS9cuUQgJE6R01KWGxIcOHARBepVMgxhkZkI3EQVUhvoIrfBXiD+AveDapBIcKHCUZz3sz9vOzlwR+ph7L1lj4xOTU9Mzs3PzC4tL5eX5RhbnqZB1EQdx2vJ4JgM/knXlq0C2klTy0Atk07s90PHmvUwzP47OVS+RnZDfRP61L7giqnMRctUVPOgfDy4PL8vrzGHbW2xvx2ZOpba7VXUJ1JhbqTLbdZgZ6yjGSVx+wWuEMgRwiJCIpwAI6MnjZcMCTEdAnLiXkm7jEALOkzSlLUgYn9pa+NzRrF2xEc+2ZGbWgVQJ6U1La2CBNTHkpYb2abeK5cdbsKO+8dR769HfK7xCYhW6xP6lG2b+V6drUbjGrqnBp5oSw+jqROGSm1PRO7d/VKXISFO4yuKp4SFUQ7P2TazNSuz5ab+IfJ1KyeiyI3x6feJTV42EV7NGhUHJc57inDNFaxhk1q4w72cYQT1MnyDs94wat1Zj1Yj9XwSoVd2IFv4b19AVzU5Oa</latexit><latexit sha1_base64="PKmVZwp+bG9nGxjnb1GtIBSU4rk=">AC0HicjVHLTsMwEJyGd3kVOHKJqJA4RU53yrgwIFDQbQgQYUcYyBqXiQOoqoQ4soPcIWvQvwB/AVrk0pwQOAoyXp2Zuzd9ZLAzxRjbyVrYHBoeGR0rDw+MTk1XZmZbWVxngrZFHEQp8cez2TgR7KpfBXI4ySVPQCeR1tnX+6EamR9Hh6qbyHbILyP/whdcEdQ+Dbm6Ejzo7d2d7ZxVqsxhaytsc91mTm1Y2XZpWCVubVlZrsOM6uKYjXiyitOcY4YAjlCSERQFAfgyOg5gQuGhLA2eoSlFPkmL3GHMmlzYklicEI79L2k3UmBRrTXnplRCzoloDclpY1F0sTESynWp9kmnxtnjf7m3TOe+m5d+nuFV0iowhWhf+n6zP/qdC0KF9gwNfhU2IQXZ0oXHLTFX1z+1tVihwSwnR8TvmUYmGU/T7bRpOZ2nVvucm/G6ZG9V4U3Bwf+pY04P4U7d+DVs1xmePus2p9qxj1KOaxgCWa5zrq2EUDTfK+xhOe8WIdWLfWvfXwRbVKhWYOP5b1+AniSJTS</latexit><latexit sha1_base64="7+0IjgJn4DAPQ+LbgVg/SZ8/Sno=">AC0HicjVHLgRBFD3T3u/B0qZjIrHqVI9hxm6ChYUFYoaEiVSXQke/dFeLyUTE1g/Y8lXiD/gLt0pPwkKoTnfOvecU3Xv9ZLAzxRjbyVrYHBoeGR0bHxicmp6pjw7187iPBWyJeIgTo8nsnAj2RL+SqQR0kqegF8tC72tT5wxuZn4cHahuIjshv4j8c19wRVDnJOTqUvCgt3N3unVarjCHrdXYet1mTnW1UVtxKVhlbnWF2a7DzKqgWLtx+RUnOEMgRwhJCIoigNwZPQcwVDQlgHPcJSinyTl7jDOGlzYklicEKv6HtBu+MCjWivPTOjFnRKQG9KShtLpImJl1KsT7NPjfOGv3Nu2c89d269PcKr5BQhUtC/9L1mf/V6VoUztEwNfhU2IQXZ0oXHLTFX1z+1tVihwSwnR8RvmUYmGU/T7bRpOZ2nVvucm/G6ZG9V4U3Bwf+pY04P4U7d+DdtVxmePu1SrNjWLUo1jAIpZpnU0sY1dtMj7Gk94xou1b91a9bDF9UqFZp5/FjW4yfjiJTW</latexit><latexit sha1_base64="7+0IjgJn4DAPQ+LbgVg/SZ8/Sno=">AC0HicjVHLgRBFD3T3u/B0qZjIrHqVI9hxm6ChYUFYoaEiVSXQke/dFeLyUTE1g/Y8lXiD/gLt0pPwkKoTnfOvecU3Xv9ZLAzxRjbyVrYHBoeGR0bHxicmp6pjw7187iPBWyJeIgTo8nsnAj2RL+SqQR0kqegF8tC72tT5wxuZn4cHahuIjshv4j8c19wRVDnJOTqUvCgt3N3unVarjCHrdXYet1mTnW1UVtxKVhlbnWF2a7DzKqgWLtx+RUnOEMgRwhJCIoigNwZPQcwVDQlgHPcJSinyTl7jDOGlzYklicEKv6HtBu+MCjWivPTOjFnRKQG9KShtLpImJl1KsT7NPjfOGv3Nu2c89d269PcKr5BQhUtC/9L1mf/V6VoUztEwNfhU2IQXZ0oXHLTFX1z+1tVihwSwnR8RvmUYmGU/T7bRpOZ2nVvucm/G6ZG9V4U3Bwf+pY04P4U7d+DdtVxmePu1SrNjWLUo1jAIpZpnU0sY1dtMj7Gk94xou1b91a9bDF9UqFZp5/FjW4yfjiJTW</latexit><latexit sha1_base64="7+0IjgJn4DAPQ+LbgVg/SZ8/Sno=">AC0HicjVHLgRBFD3T3u/B0qZjIrHqVI9hxm6ChYUFYoaEiVSXQke/dFeLyUTE1g/Y8lXiD/gLt0pPwkKoTnfOvecU3Xv9ZLAzxRjbyVrYHBoeGR0bHxicmp6pjw7187iPBWyJeIgTo8nsnAj2RL+SqQR0kqegF8tC72tT5wxuZn4cHahuIjshv4j8c19wRVDnJOTqUvCgt3N3unVarjCHrdXYet1mTnW1UVtxKVhlbnWF2a7DzKqgWLtx+RUnOEMgRwhJCIoigNwZPQcwVDQlgHPcJSinyTl7jDOGlzYklicEKv6HtBu+MCjWivPTOjFnRKQG9KShtLpImJl1KsT7NPjfOGv3Nu2c89d269PcKr5BQhUtC/9L1mf/V6VoUztEwNfhU2IQXZ0oXHLTFX1z+1tVihwSwnR8RvmUYmGU/T7bRpOZ2nVvucm/G6ZG9V4U3Bwf+pY04P4U7d+DdtVxmePu1SrNjWLUo1jAIpZpnU0sY1dtMj7Gk94xou1b91a9bDF9UqFZp5/FjW4yfjiJTW</latexit><latexit sha1_base64="7+0IjgJn4DAPQ+LbgVg/SZ8/Sno=">AC0HicjVHLgRBFD3T3u/B0qZjIrHqVI9hxm6ChYUFYoaEiVSXQke/dFeLyUTE1g/Y8lXiD/gLt0pPwkKoTnfOvecU3Xv9ZLAzxRjbyVrYHBoeGR0bHxicmp6pjw7187iPBWyJeIgTo8nsnAj2RL+SqQR0kqegF8tC72tT5wxuZn4cHahuIjshv4j8c19wRVDnJOTqUvCgt3N3unVarjCHrdXYet1mTnW1UVtxKVhlbnWF2a7DzKqgWLtx+RUnOEMgRwhJCIoigNwZPQcwVDQlgHPcJSinyTl7jDOGlzYklicEKv6HtBu+MCjWivPTOjFnRKQG9KShtLpImJl1KsT7NPjfOGv3Nu2c89d269PcKr5BQhUtC/9L1mf/V6VoUztEwNfhU2IQXZ0oXHLTFX1z+1tVihwSwnR8RvmUYmGU/T7bRpOZ2nVvucm/G6ZG9V4U3Bwf+pY04P4U7d+DdtVxmePu1SrNjWLUo1jAIpZpnU0sY1dtMj7Gk94xou1b91a9bDF9UqFZp5/FjW4yfjiJTW</latexit><latexit sha1_base64="7+0IjgJn4DAPQ+LbgVg/SZ8/Sno=">AC0HicjVHLgRBFD3T3u/B0qZjIrHqVI9hxm6ChYUFYoaEiVSXQke/dFeLyUTE1g/Y8lXiD/gLt0pPwkKoTnfOvecU3Xv9ZLAzxRjbyVrYHBoeGR0bHxicmp6pjw7187iPBWyJeIgTo8nsnAj2RL+SqQR0kqegF8tC72tT5wxuZn4cHahuIjshv4j8c19wRVDnJOTqUvCgt3N3unVarjCHrdXYet1mTnW1UVtxKVhlbnWF2a7DzKqgWLtx+RUnOEMgRwhJCIoigNwZPQcwVDQlgHPcJSinyTl7jDOGlzYklicEKv6HtBu+MCjWivPTOjFnRKQG9KShtLpImJl1KsT7NPjfOGv3Nu2c89d269PcKr5BQhUtC/9L1mf/V6VoUztEwNfhU2IQXZ0oXHLTFX1z+1tVihwSwnR8RvmUYmGU/T7bRpOZ2nVvucm/G6ZG9V4U3Bwf+pY04P4U7d+DdtVxmePu1SrNjWLUo1jAIpZpnU0sY1dtMj7Gk94xou1b91a9bDF9UqFZp5/FjW4yfjiJTW</latexit><latexit sha1_base64="7+0IjgJn4DAPQ+LbgVg/SZ8/Sno=">AC0HicjVHLgRBFD3T3u/B0qZjIrHqVI9hxm6ChYUFYoaEiVSXQke/dFeLyUTE1g/Y8lXiD/gLt0pPwkKoTnfOvecU3Xv9ZLAzxRjbyVrYHBoeGR0bHxicmp6pjw7187iPBWyJeIgTo8nsnAj2RL+SqQR0kqegF8tC72tT5wxuZn4cHahuIjshv4j8c19wRVDnJOTqUvCgt3N3unVarjCHrdXYet1mTnW1UVtxKVhlbnWF2a7DzKqgWLtx+RUnOEMgRwhJCIoigNwZPQcwVDQlgHPcJSinyTl7jDOGlzYklicEKv6HtBu+MCjWivPTOjFnRKQG9KShtLpImJl1KsT7NPjfOGv3Nu2c89d269PcKr5BQhUtC/9L1mf/V6VoUztEwNfhU2IQXZ0oXHLTFX1z+1tVihwSwnR8RvmUYmGU/T7bRpOZ2nVvucm/G6ZG9V4U3Bwf+pY04P4U7d+DdtVxmePu1SrNjWLUo1jAIpZpnU0sY1dtMj7Gk94xou1b91a9bDF9UqFZp5/FjW4yfjiJTW</latexit>

LA

<latexit sha1_base64="ljXmF37Llrl81rV1fQ78Ns6CQk4=">AC0HicjVHLgRBFD3T3u/B0qZjIrHqVI9hxm6wsbBAzJAwkepS6OiX7moxmYjY+gFbvkr8AX/hVulJWAjV6e5b5zqu69XhL4mWLsrWQNDA4Nj4yOjU9MTk3PlGfn2lmcp0K2RBzE6ZHMxn4kWwpXwXyKEklD71AHnpXWzp/eCPTzI+jA9VNZCfkF5F/7guCOqchFxdCh70du5ON07LFeawtRpbr9vMqa42aisuBavMra4w23WYWRUazcuv+IEZ4ghkCOERARFcQCOjJ5juGBICOugR1hKkW/yEncYJ21OLEkMTugVfS9od1ygEe21Z2bUgk4J6E1JaWOJNDHxUor1abJ58Zo79594ynvluX/l7hFRKqcEnoX7o+8786XYvCORqmBp9qSgyiqxOFS26om9uf6tKkUNCmI7PKJ9SLIy32fbaDJTu+4tN/l3w9So3ouCm+ND35IG3J+i/XvQrjouc9y9WqW5WYx6FAtYxDLNs4mtrGLFnlf4wnPeLH2rVvr3nr4olqlQjOPH8t6/ATcaJT</latexit><latexit sha1_base64="ljXmF37Llrl81rV1fQ78Ns6CQk4=">AC0HicjVHLgRBFD3T3u/B0qZjIrHqVI9hxm6wsbBAzJAwkepS6OiX7moxmYjY+gFbvkr8AX/hVulJWAjV6e5b5zqu69XhL4mWLsrWQNDA4Nj4yOjU9MTk3PlGfn2lmcp0K2RBzE6ZHMxn4kWwpXwXyKEklD71AHnpXWzp/eCPTzI+jA9VNZCfkF5F/7guCOqchFxdCh70du5ON07LFeawtRpbr9vMqa42aisuBavMra4w23WYWRUazcuv+IEZ4ghkCOERARFcQCOjJ5juGBICOugR1hKkW/yEncYJ21OLEkMTugVfS9od1ygEe21Z2bUgk4J6E1JaWOJNDHxUor1abJ58Zo79594ynvluX/l7hFRKqcEnoX7o+8786XYvCORqmBp9qSgyiqxOFS26om9uf6tKkUNCmI7PKJ9SLIy32fbaDJTu+4tN/l3w9So3ouCm+ND35IG3J+i/XvQrjouc9y9WqW5WYx6FAtYxDLNs4mtrGLFnlf4wnPeLH2rVvr3nr4olqlQjOPH8t6/ATcaJT</latexit><latexit sha1_base64="ljXmF37Llrl81rV1fQ78Ns6CQk4=">AC0HicjVHLgRBFD3T3u/B0qZjIrHqVI9hxm6wsbBAzJAwkepS6OiX7moxmYjY+gFbvkr8AX/hVulJWAjV6e5b5zqu69XhL4mWLsrWQNDA4Nj4yOjU9MTk3PlGfn2lmcp0K2RBzE6ZHMxn4kWwpXwXyKEklD71AHnpXWzp/eCPTzI+jA9VNZCfkF5F/7guCOqchFxdCh70du5ON07LFeawtRpbr9vMqa42aisuBavMra4w23WYWRUazcuv+IEZ4ghkCOERARFcQCOjJ5juGBICOugR1hKkW/yEncYJ21OLEkMTugVfS9od1ygEe21Z2bUgk4J6E1JaWOJNDHxUor1abJ58Zo79594ynvluX/l7hFRKqcEnoX7o+8786XYvCORqmBp9qSgyiqxOFS26om9uf6tKkUNCmI7PKJ9SLIy32fbaDJTu+4tN/l3w9So3ouCm+ND35IG3J+i/XvQrjouc9y9WqW5WYx6FAtYxDLNs4mtrGLFnlf4wnPeLH2rVvr3nr4olqlQjOPH8t6/ATcaJT</latexit><latexit sha1_base64="ljXmF37Llrl81rV1fQ78Ns6CQk4=">AC0HicjVHLgRBFD3T3u/B0qZjIrHqVI9hxm6wsbBAzJAwkepS6OiX7moxmYjY+gFbvkr8AX/hVulJWAjV6e5b5zqu69XhL4mWLsrWQNDA4Nj4yOjU9MTk3PlGfn2lmcp0K2RBzE6ZHMxn4kWwpXwXyKEklD71AHnpXWzp/eCPTzI+jA9VNZCfkF5F/7guCOqchFxdCh70du5ON07LFeawtRpbr9vMqa42aisuBavMra4w23WYWRUazcuv+IEZ4ghkCOERARFcQCOjJ5juGBICOugR1hKkW/yEncYJ21OLEkMTugVfS9od1ygEe21Z2bUgk4J6E1JaWOJNDHxUor1abJ58Zo79594ynvluX/l7hFRKqcEnoX7o+8786XYvCORqmBp9qSgyiqxOFS26om9uf6tKkUNCmI7PKJ9SLIy32fbaDJTu+4tN/l3w9So3ouCm+ND35IG3J+i/XvQrjouc9y9WqW5WYx6FAtYxDLNs4mtrGLFnlf4wnPeLH2rVvr3nr4olqlQjOPH8t6/ATcaJT</latexit>

LM

<latexit sha1_base64="Z2C6iZioE2mvy0K3nmyxeWXg0lg=">AC0HicjVHLgRBFD3T3uM1WNp0TCRWnerxaHbCxoIEMUbCZFJdCh390l0tZCJi6wds+SrxB/yFW6UnYSFUp7tvnXvOqbr3+mkY5Iqxt4rV1z8wODQ8Uh0dG5+YrE1NH+ZJkQnZFEmYZEc+z2UYxLKpAhXKozSTPJD2fIvN3W+dS2zPEjiA3WbynbEz+PgLBcEdQ+ibi6EDzsbt91djq1OnO8lSVvdFmzuIyazCXArbWcD3Pdh1mVh3l2k1qrzjBKRIFIgEUNRHIjp+cYLhSwtroEpZRFJi8xB2qpC2IJYnBCb2k7zntjks0pr32zI1a0CkhvRkpbcyTJiFeRrE+zTb5wjhr9DfvrvHUd7ulv196RYQqXBD6l67H/K9O16JwhlVTQ0A1pQbR1YnSpTBd0Te3v1WlyCElTMenlM8oFkbZ67NtNLmpXfeWm/y7YWpU70XJLfChb0kD7k3R/j04bDguc9y9pfr6RjnqYcxiDgs0Tw/r2MIumuR9hSc848Xat26se+vhi2pVSs0Mfizr8RMUd5Tr</latexit><latexit sha1_base64="Z2C6iZioE2mvy0K3nmyxeWXg0lg=">AC0HicjVHLgRBFD3T3uM1WNp0TCRWnerxaHbCxoIEMUbCZFJdCh390l0tZCJi6wds+SrxB/yFW6UnYSFUp7tvnXvOqbr3+mkY5Iqxt4rV1z8wODQ8Uh0dG5+YrE1NH+ZJkQnZFEmYZEc+z2UYxLKpAhXKozSTPJD2fIvN3W+dS2zPEjiA3WbynbEz+PgLBcEdQ+ibi6EDzsbt91djq1OnO8lSVvdFmzuIyazCXArbWcD3Pdh1mVh3l2k1qrzjBKRIFIgEUNRHIjp+cYLhSwtroEpZRFJi8xB2qpC2IJYnBCb2k7zntjks0pr32zI1a0CkhvRkpbcyTJiFeRrE+zTb5wjhr9DfvrvHUd7ulv196RYQqXBD6l67H/K9O16JwhlVTQ0A1pQbR1YnSpTBd0Te3v1WlyCElTMenlM8oFkbZ67NtNLmpXfeWm/y7YWpU70XJLfChb0kD7k3R/j04bDguc9y9pfr6RjnqYcxiDgs0Tw/r2MIumuR9hSc848Xat26se+vhi2pVSs0Mfizr8RMUd5Tr</latexit><latexit sha1_base64="Z2C6iZioE2mvy0K3nmyxeWXg0lg=">AC0HicjVHLgRBFD3T3uM1WNp0TCRWnerxaHbCxoIEMUbCZFJdCh390l0tZCJi6wds+SrxB/yFW6UnYSFUp7tvnXvOqbr3+mkY5Iqxt4rV1z8wODQ8Uh0dG5+YrE1NH+ZJkQnZFEmYZEc+z2UYxLKpAhXKozSTPJD2fIvN3W+dS2zPEjiA3WbynbEz+PgLBcEdQ+ibi6EDzsbt91djq1OnO8lSVvdFmzuIyazCXArbWcD3Pdh1mVh3l2k1qrzjBKRIFIgEUNRHIjp+cYLhSwtroEpZRFJi8xB2qpC2IJYnBCb2k7zntjks0pr32zI1a0CkhvRkpbcyTJiFeRrE+zTb5wjhr9DfvrvHUd7ulv196RYQqXBD6l67H/K9O16JwhlVTQ0A1pQbR1YnSpTBd0Te3v1WlyCElTMenlM8oFkbZ67NtNLmpXfeWm/y7YWpU70XJLfChb0kD7k3R/j04bDguc9y9pfr6RjnqYcxiDgs0Tw/r2MIumuR9hSc848Xat26se+vhi2pVSs0Mfizr8RMUd5Tr</latexit><latexit sha1_base64="Z2C6iZioE2mvy0K3nmyxeWXg0lg=">AC0HicjVHLgRBFD3T3uM1WNp0TCRWnerxaHbCxoIEMUbCZFJdCh390l0tZCJi6wds+SrxB/yFW6UnYSFUp7tvnXvOqbr3+mkY5Iqxt4rV1z8wODQ8Uh0dG5+YrE1NH+ZJkQnZFEmYZEc+z2UYxLKpAhXKozSTPJD2fIvN3W+dS2zPEjiA3WbynbEz+PgLBcEdQ+ibi6EDzsbt91djq1OnO8lSVvdFmzuIyazCXArbWcD3Pdh1mVh3l2k1qrzjBKRIFIgEUNRHIjp+cYLhSwtroEpZRFJi8xB2qpC2IJYnBCb2k7zntjks0pr32zI1a0CkhvRkpbcyTJiFeRrE+zTb5wjhr9DfvrvHUd7ulv196RYQqXBD6l67H/K9O16JwhlVTQ0A1pQbR1YnSpTBd0Te3v1WlyCElTMenlM8oFkbZ67NtNLmpXfeWm/y7YWpU70XJLfChb0kD7k3R/j04bDguc9y9pfr6RjnqYcxiDgs0Tw/r2MIumuR9hSc848Xat26se+vhi2pVSs0Mfizr8RMUd5Tr</latexit>

Target Labels Input Image

!"

# #′

Output Image Original Labels

!;

A) B)

<=>? <@""> A B

Gender Age Race

M: G: A: R:

M: G: A: R: M: G: A: R: M: G: A: R: M: G: A: R: M: G: A: R: M: G: A: R: M: G: A: R:

Original image

PrivacyNet: Transforming face images for

• bfuscating soft-biometric

attributes

~2018-present: Increasing focus on user privacy

SLIDE 65

65

AI / DL and the "Small Data" Bottleneck

Transfer learning (pre-train on related dataset) One- and few-shot learning (train on many tasks with few examples per class) Human-in- the-loop Self-supervised learning (like transfer learning but pre-train

• n related task)

Synthetic data (GAN, autoencoder)