Stat 5101 Lecture Slides: Deck 6 Existence of Integrals and Infinite - PowerPoint PPT Presentation

Stat 5101 Lecture Slides: Deck 6 Existence of Integrals and Infinite Sums, Countable Additivity and Monotone Convergence, Existence of Moments, Correlation Charles J. Geyer School of Statistics University of Minnesota This work is licensed under a Creative Commons Attribution- ShareAlike 4.0 International License ( http://creativecommons.org/ licenses/by-sa/4.0/ ). 1

Existence of Integrals Just from the definition of integral as area under the curve, the integral � b a g ( x ) dx always exists when a and b are finite and g is bounded , which means there exists a finite c such that | g ( x ) | ≤ c, a < x < b. In this case � b � b � � � � a g ( x ) dx a | g ( x ) | dx ≤ c ( b − a ) � ≤ � � � � � 2

Existence of Integrals (cont.) It is a theorem of advanced calculus (which we will not prove) that every continuous function g having domain [ a, b ] where a and b are finite is bounded. So if we know g is continuous on [ a, b ], then we know � b a g ( x ) dx exists. It is important that the domain is a closed interval. The function x �→ 1 /x is continuous but unbounded on (0 , 1). So continuous on an open interval is not good enough. 3

Existence of Integrals (cont.) We are worried about non-existence. Clearly � b a g ( x ) dx may fail to exist in one of two cases (I) either a or b is infinite, or (II) g is unbounded (meaning not bounded). 4

Existence of Integrals: Case I So when does � ∞ g ( x ) dx a exist? In probability theory, we require absolute integrability, so � ∞ | g ( x ) | dx a must be finite. 5

Existence of Integrals: Case I (cont.) First we do a very important special case. Suppose a > 0, then � ∞ x α dx < ∞ a if and only if α < − 1. 6

Existence of Integrals: Case I (cont.) Case I of Case I, if α � = − 1, then b � b = b α +1 − a α +1 a x α dx = x α +1 � � � � α + 1 α + 1 a � If α > − 1, then this goes to infinity as b → ∞ . If α < − 1, then this goes to − a α +1 / ( α + 1) as b → ∞ . Case II of Case I, if α = − 1, then � b b � a x α dx = log( x ) � = log( b ) − log( a ) � � � a and this goes to infinity as b → ∞ . 7

Existence of Integrals: Comparison Principle Also obvious from the definition of integral as area under the curve, if | g ( x ) | ≤ | h ( x ) | , a < x < b, then � b � b a | g ( x ) | dx ≤ a | h ( x ) | dx including when either integral is infinite , that is, when the right- hand side is finite, then so is the left-hand side and when the left-hand side is infinite, then so is the right-hand side. 8

Existence of Integrals: Case I (cont.) Suppose a > 0, suppose g is continuous on [ a, ∞ ), and suppose | g ( x ) | lim = c x α x →∞ exists and is finite. If α < − 1, then � ∞ | g ( x ) | dx < ∞ . a Conversely, if c > 0 and α ≥ − 1, then � ∞ | g ( x ) | dx = ∞ . a 9

Existence of Integrals: Case I (cont.) From the definition of limit, we know there exists a finite r such that 2 ≤ | g ( x ) | c ≤ 1 + c, x ≥ r x α and we know that � r a | g ( x ) | dx < ∞ and � ∞ � ∞ � ∞ c x α dx ≤ x α dx | g ( x ) | dx ≤ (1 + c ) 2 r r r Hence the result about g ( x ) follows from the result about x α . 10

Existence of Integrals: Case I (cont.) There exists a constant c such that c f ( x ) = 1 + x 2 + 3( x − 1) 4 , −∞ < x < ∞ is a PDF. Compare f ( x ) | x | − 4 → c 3 as x → −∞ or x → + ∞ . Since − 4 < − 1, it follows that the integral of f is finite. 11

Existence of Integrals: Case I (cont.) In the preceding example we used two important principles. • Constants don’t matter. • In polynomials, only the term of highest degree matters. 12

Existence of Integrals: Case I (cont.) In more detail, if c is a constant � b � b a cg ( x ) dx = c a g ( x ) dx and both sides exist or neither does. And a 0 + a 1 x + a 2 x 2 + · · · + a k x k lim = a k x k x →∞ 13

Existence of Integrals: Case I (cont.) Returning to our example, suppose X has PDF c f ( x ) = 1 + x 2 + 3( x − 1) 4 , −∞ < x < ∞ , then for what positive values of β does E ( | X | β ) exist? Compare | x | β f ( x ) → c | x | β − 4 3 as x → −∞ or x → + ∞ . Since β − 4 < − 1, if and only if β < 3, it follows E ( | X | β ) exists if and only if β < 3 (when β > 0 is assumed). 14

The Cauchy Distribution There exists a constant c such that c f ( x ) = − ∞ < x < ∞ 1 + x 2 is a PDF. Compare f ( x ) | x | − 2 → c as x → −∞ or x → + ∞ . Since − 2 < − 1, it follows that the integral of f is finite. 15

The Cauchy Distribution (cont.) In this case we can actually determine the constant. � t t dx � � 1 + x 2 = atan( x ) = atan( t ) − atan( − t ) , � − t � − t where atan is the arctangent function, which goes from − π/ 2 to π/ 2 as its argument goes from −∞ to ∞ . Thus � ∞ dx � � 1 + x 2 = lim atan( t ) − atan( − t ) = π t →∞ −∞ and c = 1 /π . 16

The Cauchy Distribution (cont.) The distribution with PDF 1 f ( x ) = π (1 + x 2 ) , −∞ < x < ∞ is called the standard Cauchy distribution . The distribution with PDF f µ,σ ( x ) = σ 1 π · σ 2 + ( x − µ ) 2 , −∞ < x < ∞ is called the Cauchy distribution with location parameter µ and scale parameter σ and is abbreviated Cauchy( µ, σ ). 17

The Cauchy Distribution (cont.) The Cauchy( µ, σ ) distributions are a location-scale family. The Cauchy( µ, σ ) distribution is symmetric about µ , so the pa- rameter µ can be called the center of symmetry as well as the location parameter . 18

The Cauchy Distribution (cont.) If X has the Cauchy( µ, σ ) distribution, then for what positive values of β does E ( | X | β ) exist? Compare | x | β f ( x ) → σ | x | β − 2 π as x → −∞ or x → + ∞ . Since β − 2 < − 1, if and only if β < 1, it follows E ( | X | β ) exists if and only if β < 1 (when β > 0 is assumed). Summary: If X has the Cauchy( µ, σ ) distribution, then E ( X k ) exists for no positive integer k . The mean does not exist, neither does the variance. Hence µ cannot be the mean, and σ cannot be the standard deviation. 19

Existence of Integrals: Case I (cont.) The Maclaurin series e x = 1 + x + x 2 2 + · · · + x k k ! + · · · which we also know as the theorem that a Poisson PMF sums to one, shows that e x grows faster than any polynomial as x → ∞ . Similarly for e λx when λ > 0. Hence for any β ∈ R , any α ∈ R , any λ > 0, and any a > 0 x β e − λx x β − α lim = lim = 0 x α e λx x →∞ x →∞ and � ∞ x β e − λx dx < ∞ a 20

Existence of Integrals: Case II Now we turn to case II. The domain of integration is bounded, but the integrand is unbounded. Again we start with the monomial special case. If a > 0, then � a 0 x α dx < ∞ if and only if α > − 1. Note that the magic exponent − 1 is the same, but the inequality is reversed. 21

Existence of Integrals: Case II (cont.) The substitution x = 1 /y reduces this to the other case. � a � 1 /a � ∞ 0 x α dx = 1 /a y − α − 2 dy y − α � − y − 2 � dy = ∞ and we already know the latter is finite if and only if − α − 2 < − 1, which is the same as − α < 1 or α > − 1. 22

Existence of Integrals: Case II (cont.) We can move this theorem to any other point. If a < b , then � b a ( x − a ) α dx < ∞ if and only if α > − 1, and � b a ( b − x ) α dx < ∞ if and only if α > − 1. 23

Existence of Integrals: Case II (cont.) And we can analyze other integrals by comparison. Suppose g is continuous on ( a, b ] and suppose | g ( x ) | lim ( x − a ) α = c x ↓ a exists and is finite. If α > − 1, then � b a | g ( x ) | dx < ∞ . Conversely, if c > 0 and α ≤ − 1, then � b a | g ( x ) | dx = ∞ . 24

Existence of Integrals: Case II (cont.) The case where g is unbounded at b is an obvious modification. Suppose g is continuous on [ a, b ) and suppose | g ( x ) | lim ( b − x ) α = c x ↑ b exists and is finite. If α > − 1, then � b a | g ( x ) | dx < ∞ . Conversely, if c > 0 and α ≤ − 1, then � b a | g ( x ) | dx = ∞ . 25

Existence of Integrals: Case I and II Summary If g ( x ) is continuous on [ a, ∞ ), then � ∞ | g ( x ) | dx < ∞ a only if g ( x ) goes to zero as x → ∞ fast enough, faster than 1 /x . If g ( x ) is continuous on ( a, b ), then � b a | g ( x ) | dx < ∞ only if g ( x ) goes to infinity as x → a or as x → b slow enough, slower than 1 / ( x − a ) or 1 / ( b − x ). 26

Existence of Integrals: Gamma Distribution When does there exist a c such that f ( x ) = cx α − 1 e − λx , 0 < x < ∞ is a PDF? We have already been told this is the PDF of the Gam( α, λ ) distribution when α > 0 and λ > 0, but we haven’t proved it. If λ > 0, then we know from applying our theorem about case I that � ∞ x α − 1 e − λx dx < ∞ a for any real α and any a > 0. 27