Strong Asymptotic Assertions for Discrete MDL in Regression and - PowerPoint PPT Presentation

Strong Asymptotic Assertions for Discrete MDL in Regression and Classification or A Strange Way of Proving Consistency of MDL Learning Jan Poland and Marcus Hutter IDSIA � Lugano � Switzerland

2 Focus of this Talk Regression Classification this talk this paper Sequence Prediction COLT‘04

3 Why Consistency? � ��

4 Setup � �� !� �� x �� n , y �� n � � �� x i � � �� y i � � � , � � �� i � n � �� x � � " �� # �� y � � $�� %��& �� # �� y � x � � � � � '��& �� '($" � �� " )))

5 Bayesian Framework � * �� ν ��! � �� # �� !�� , � � � �� !�� +�� ν � � �� w ν > � � ,�� %��& � ν ∈C w ν � � � +��!��& � ��

6 Proper/Online Learning .�� !��& � �� x � � �� !� �� !��! � �� y �� # �� -�� & �� x �� t , y <t � �� x <t , y <t �

7 Bayes Mixture � ��" �� x �� n , y �� n � " �� n �� t �� ν � y t � x t � ν w ν � � n ξ � y n �� x �� n �� , y �� n � � t �� ν � y t � x t � ν w ν � �� /�� !�� /�� !��" ��

8 Static MDL ��" �� !�� $*.�& ̺ �� y n �� x �� n �� , y �� n � � ν ∗ � x �� ,y �� y n �� x n �� ν ∗ ν ∈C � w ν ν � y �� n � x �� n � � � x �� ,y �� 0��!�� ! � )

9 Dynamic MDL �� ! �� $0� �� #��!��1�� !��1�� " �� & ̺ � y n � , y <n � � ̺ � y �� n � x �� n � ̺ � y <n � x <n � ̺ � y �� n � x �� n � � ̺ � y n � , y <n � � � y � ̺ � y �� n � x �� n � �� ̺ � y �� n � x �� n � � �� ν ∈C � w ν ν � y �� n � x �� n � � . �� !��& ��!�� !�� y n ) �� !�� $0� �� !�� !�� !�� 2�)

10 Distance and Convergence �� & �� h � � � y t � x �� t , y <t � � ψ � y t � x �� t , y <t � t � �, ψ � � . y � ∈{ � , � } �� ψ #�� #�� ∞ H � � � h � t � �, ψ �� < � . x � ∞ � �, ψ � � t �� !�� h � t � � �� )

11 Other Distance Measures � � � � � � y t � x �� t , y <t � � ψ � y t � x �� t , y <t � s t � �, ψ � � y � ∈{ � , � } �%�� y t � x �� t , y <t � � ψ � y t � x �� t , y <t � a t � �, ψ � � � y � ∈{ � , � } �� y t � x �� t , y <t � � �� y t � x �� t , y <t � d t � �, ψ � � ψ � y t � x �� t , y <t � y � ∈{ � , � } ,��3#��

12 Distance Measures: Properties � � �� %�� a t � 4�� h t & � d t � � � �!�� %�� a t � 5�� s t & � d t � � � �!�� %�� *�� a t & � d t � �� %�� ,��3#�� d t & � a t

13 Convergence Theorem 6�� " �� w � �� " �� ̺ �� ̺ ̺ � � � � � t a t � � w − � t a t � � w − � t d t � � w − � � � � � H � � �,̺ �� w − � �

14 Properties of the Proof � .�� -#��!� � �� '��!��7� �� # ��

15 Loss bounds � *��!� �� ℓ � y, � y � x � � �� x " �� y " �� # �� y � �� ) �� /��#��!�� L �� !�� & � L � ̺ � � L � � � � �� w − � �� w − � � � � � L � � � � � �� #�� 1�� # !��

Strong Asymptotic Assertions for Discrete MDL in Regression and - PowerPoint PPT Presentation

Strong Asymptotic Assertions for Discrete MDL in Regression and Classification or A Strange Way of Proving Consistency of MDL Learning Jan Poland and Marcus Hutter IDSIA Lugano Switzerland 2 Focus of this Talk Regression

Asymptotic approximations for options with discrete sampling Sam Howison Mathematical Institute

Program Analysis discrete math refresher READING: Textbook chapter 2 (p. 13-26)

Zero topological entropy and asymptotic pairs Czech-Slovak-Spanish-Polish Workshop on Discrete

High Order Asymptotic Preserving Schemes for Some Discrete-Velocity Kinetic Equations Fengyan Li

Program Analysis 1 Summary Summary analysis of algorithms asymptotic analysis

1 Java assert statement Java assert (cont.) an assert statement can be placed anywhere you

Assertions, pre/post- conditions Assertions: Section 4.2 in Savitch (p. 239) Programming as a

Assertions and Triggers Rose-Hulman Institute of Technology Curt Clifton Assertions Like

The Discovery of Asymptotic Freedom & The Emergence of QCD David Gross Nobel Lecture

The strong asymptotic freeness of large random and deterministic matrices Camille Male

Types of Formal Specifications for Assertions Concurrent and Reactive Systems Assertions

ASYNCHRONOUS ASSERTIONS What are assertions? public class ATM { public void

On the Asymptotic Variance of the Estimator of Kendalls Tau Barbara Dengler, Uwe Schmock

Asymptotic behavior of series of multiple integrals. K. K. Kozlowski CNRS, Institut de

10/6/2016 Give Asymptotic Analyses for the Following g( n ) = 45 n log n + 2 n 2 + 65 1. 2. g( n

1 Should debug code be left in shipped version Step through your code Pro: Maguire

Non-parameteric Estimation of Integral Probability Metrics Bharath K. Sriperumbudur , Kenji

Critical Asset & Portfolio Risk Analysis State of Practice and Challenges Bilal M. Ayyub,

Using Criminal History Records to Predict Recidivism Robert Brame Department of Criminology

A hierarchical model for micro-level E.A. Valdez stochastic loss reserving joint work with K.

A coherent structure approach for parameter estimation in Data Assimilation John Maclean 1 ,

Statistical Inference for Large Directed Graphs with Communities of Interest Deepak Agarwal

FLSA Collective Action Discovery Challenges Effective Approaches Before and After Conditional

The Kronig-Penney model extended to arbitrary potentials via numerical matrix mechanics Robert

Strong Asymptotic Assertions for Discrete MDL in Regression and - PowerPoint PPT Presentation

Strong Asymptotic Assertions for Discrete MDL in Regression and Classification or A Strange Way of Proving Consistency of MDL Learning Jan Poland and Marcus Hutter IDSIA Lugano Switzerland 2 Focus of this Talk Regression

Asymptotic approximations for options with discrete sampling Sam Howison Mathematical Institute

Program Analysis discrete math refresher READING: Textbook chapter 2 (p. 13-26)

Zero topological entropy and asymptotic pairs Czech-Slovak-Spanish-Polish Workshop on Discrete

High Order Asymptotic Preserving Schemes for Some Discrete-Velocity Kinetic Equations Fengyan Li

Program Analysis 1 Summary Summary analysis of algorithms asymptotic analysis

1 Java assert statement Java assert (cont.) an assert statement can be placed anywhere you

Assertions, pre/post- conditions Assertions: Section 4.2 in Savitch (p. 239) Programming as a

Assertions and Triggers Rose-Hulman Institute of Technology Curt Clifton Assertions Like

The Discovery of Asymptotic Freedom &amp; The Emergence of QCD David Gross Nobel Lecture

The strong asymptotic freeness of large random and deterministic matrices Camille Male

Types of Formal Specifications for Assertions Concurrent and Reactive Systems Assertions

ASYNCHRONOUS ASSERTIONS What are assertions? public class ATM { public void

On the Asymptotic Variance of the Estimator of Kendalls Tau Barbara Dengler, Uwe Schmock

Asymptotic behavior of series of multiple integrals. K. K. Kozlowski CNRS, Institut de

10/6/2016 Give Asymptotic Analyses for the Following g( n ) = 45 n log n + 2 n 2 + 65 1. 2. g( n

1 Should debug code be left in shipped version Step through your code Pro: Maguire

Non-parameteric Estimation of Integral Probability Metrics Bharath K. Sriperumbudur , Kenji

Critical Asset &amp; Portfolio Risk Analysis State of Practice and Challenges Bilal M. Ayyub,

Using Criminal History Records to Predict Recidivism Robert Brame Department of Criminology

A hierarchical model for micro-level E.A. Valdez stochastic loss reserving joint work with K.

A coherent structure approach for parameter estimation in Data Assimilation John Maclean 1 ,

Statistical Inference for Large Directed Graphs with Communities of Interest Deepak Agarwal

FLSA Collective Action Discovery Challenges Effective Approaches Before and After Conditional

The Kronig-Penney model extended to arbitrary potentials via numerical matrix mechanics Robert

The Discovery of Asymptotic Freedom & The Emergence of QCD David Gross Nobel Lecture

Critical Asset & Portfolio Risk Analysis State of Practice and Challenges Bilal M. Ayyub,