Statistics and Steganalysis CSM25 Secure Information Hiding Dr Hans - - PowerPoint PPT Presentation

Statistics and Steganalysis

CSM25 Secure Information Hiding Dr Hans Georg Schaathun

University of Surrey

Spring 2007

Dr Hans Georg Schaathun Statistics and Steganalysis Spring 2007 1 / 52

Learning Outcomes

After this sessions, everyone should

understand what a statistical hypothesis test is know how hypothesis testing applies to steganalysis be able to implement the basic χ2 test of steganalysis

Dr Hans Georg Schaathun Statistics and Steganalysis Spring 2007 2 / 52

Statistics Reading

Suggested Reading Gouri K. Bhattacharyya and Richard A. Johnson: Statistical Concepts and Methods (Wiley Series in Probability and Statistics). Suggested Reading Jessica Fridrich: ‘Basic concepts from statistics’ from EECE 562: Fundamentals of Steganography http: //www.ws.binghamton.edu/fridrich/562/material.html

Dr Hans Georg Schaathun Statistics and Steganalysis Spring 2007 3 / 52

Probabilities Experiments and Events

Outline

1

Probabilities Experiments and Events Probabilities Random variables Independence

2

Probability Distributions The uniform distribution Some other distributions

3

Hypothesis tests Introduction

4

Pairs of Values Steganalysis Pairs of Values

5

Generalised χ2 test

Dr Hans Georg Schaathun Statistics and Steganalysis Spring 2007 4 / 52

Probabilities Experiments and Events

Experiment

Definition (Bhattacharyya and Johnson) An experiment is the process of collecting data relevant to a phenomenon that exhibits variation in its outcomes. Example When Wendy observes the image X transmitted by Alice to Bob, she is conducting an experiment. Experiment ∼ observe Random phenomena

Dr Hans Georg Schaathun Statistics and Steganalysis Spring 2007 5 / 52

Probabilities Experiments and Events

Experiment

Definition (Bhattacharyya and Johnson) An experiment is the process of collecting data relevant to a phenomenon that exhibits variation in its outcomes. Example When Wendy observes the image X transmitted by Alice to Bob, she is conducting an experiment. Experiment ∼ observe Random phenomena

Dr Hans Georg Schaathun Statistics and Steganalysis Spring 2007 5 / 52

Probabilities Experiments and Events

Experiment

Definition (Bhattacharyya and Johnson) An experiment is the process of collecting data relevant to a phenomenon that exhibits variation in its outcomes. Example When Wendy observes the image X transmitted by Alice to Bob, she is conducting an experiment. Wendy’s objective is to learn about the information Alice and Bob exchange. Experiment ∼ observe Random phenomena

Dr Hans Georg Schaathun Statistics and Steganalysis Spring 2007 5 / 52

Probabilities Experiments and Events

Experiment

Definition (Bhattacharyya and Johnson) An experiment is the process of collecting data relevant to a phenomenon that exhibits variation in its outcomes. Example When Wendy observes the image X transmitted by Alice to Bob, she is conducting an experiment. Wendy’s objective is to learn about the information Alice and Bob exchange. Experiment ∼ observe Random phenomena

Dr Hans Georg Schaathun Statistics and Steganalysis Spring 2007 5 / 52

Probabilities Experiments and Events

Experiment

Definition (Bhattacharyya and Johnson) An experiment is the process of collecting data relevant to a phenomenon that exhibits variation in its outcomes. Example When Wendy observes the image X transmitted by Alice to Bob, she is conducting an experiment. Wendy’s objective is to learn about the information Alice and Bob exchange. Experiment ∼ observe Random phenomena

Dr Hans Georg Schaathun Statistics and Steganalysis Spring 2007 5 / 52

Probabilities Experiments and Events

Sample Space

Definition The set of possible outcomes of an experiment is called the sample space S. Each element of the sample space is a simple event. Example The sample space in Wendy’s eavesdropping is the set of all images that Bob and Alice might exchange. Lenna in 256x256 JPEG with quality factor 76, would be one simple event.

Dr Hans Georg Schaathun Statistics and Steganalysis Spring 2007 6 / 52

Probabilities Experiments and Events

Sample Space

Definition The set of possible outcomes of an experiment is called the sample space S. Each element of the sample space is a simple event. Example The sample space in Wendy’s eavesdropping is the set of all images that Bob and Alice might exchange. Lenna in 256x256 JPEG with quality factor 76, would be one simple event.

Dr Hans Georg Schaathun Statistics and Steganalysis Spring 2007 6 / 52

Probabilities Experiments and Events

Sample Space

Definition The set of possible outcomes of an experiment is called the sample space S. Each element of the sample space is a simple event. Example The sample space in Wendy’s eavesdropping is the set of all images that Bob and Alice might exchange. Lenna in 256x256 JPEG with quality factor 76, would be one simple event.

Dr Hans Georg Schaathun Statistics and Steganalysis Spring 2007 6 / 52

Probabilities Experiments and Events

Event

Definition An event E is a collection of simple events characterised by some common descriptive feature. In other words, it is a subset E ⊂ S of the sample space.

Dr Hans Georg Schaathun Statistics and Steganalysis Spring 2007 7 / 52

Probabilities Probabilities

Outline

1

Probabilities Experiments and Events Probabilities Random variables Independence

2

Probability Distributions The uniform distribution Some other distributions

3

Hypothesis tests Introduction

4

Pairs of Values Steganalysis Pairs of Values

5

Generalised χ2 test

Dr Hans Georg Schaathun Statistics and Steganalysis Spring 2007 8 / 52

Probabilities Probabilities

Probabilities

The probability of an event e is written Pr(e). Pr : S → [0, 1]

• e∈SPr(e) = 1 (where S is the set of simple events).

If we could conduct the experiment N times (independently), we expect to see e N · Pr(e) times when N is large.

Dr Hans Georg Schaathun Statistics and Steganalysis Spring 2007 9 / 52

Probabilities Probabilities

Probabilities

The probability of an event e is written Pr(e). Pr : S → [0, 1]

• e∈SPr(e) = 1 (where S is the set of simple events).

If we could conduct the experiment N times (independently), we expect to see e N · Pr(e) times when N is large.

Dr Hans Georg Schaathun Statistics and Steganalysis Spring 2007 9 / 52

Probabilities Probabilities

Probability distribution

Let S be the feature observed in an experiment. Let S be the sample space. Definition The probability distribution of S is the function PS(s) = Pr(S = s) for all s ∈ S.

Dr Hans Georg Schaathun Statistics and Steganalysis Spring 2007 10 / 52

Probabilities Probabilities

Example: Probability Distributions

Stegogramme

Example Suppose Alice sends a stegogramme S from Alice to Bob. Wendy eavesdrop S. The sample space S is the set of possible stegogramme from their software Let PS be the probability distribution of S (the stegogramme distribution)

Dr Hans Georg Schaathun Statistics and Steganalysis Spring 2007 11 / 52

Probabilities Probabilities

Example: Probability Distributions

Innocent image

Example Suppose Alice sends an innocent message C to Bob. Wendy eavesdrop C. The sample space S is the set of possible images. Let PC be the probability distribution of C (the covertext distribution)

Dr Hans Georg Schaathun Statistics and Steganalysis Spring 2007 12 / 52

Probabilities Probabilities

Statistical Steganalysis

The goal of Statistical Steganalysis: Wendy observes an image X sent from Alice to Bob. Let x be the observed image. Is x a stegogramme (from S) or a pure image (from C)? Unfortunately, most of the time x ∈ C and x ∈ S are both possible. . . Compare Pr(S = x) and Pr(C = x). If Pr(S = x) > Pr(C = x), then x is most likely to be a stegogramme. How confident can we be about the conclusion?

Dr Hans Georg Schaathun Statistics and Steganalysis Spring 2007 13 / 52

Probabilities Probabilities

Statistical Steganalysis

The goal of Statistical Steganalysis: Wendy observes an image X sent from Alice to Bob. Let x be the observed image. Is x a stegogramme (from S) or a pure image (from C)? Unfortunately, most of the time x ∈ C and x ∈ S are both possible. . . Compare Pr(S = x) and Pr(C = x). If Pr(S = x) > Pr(C = x), then x is most likely to be a stegogramme. How confident can we be about the conclusion?

Dr Hans Georg Schaathun Statistics and Steganalysis Spring 2007 13 / 52

Probabilities Probabilities

Statistical Steganalysis

The goal of Statistical Steganalysis: Wendy observes an image X sent from Alice to Bob. Let x be the observed image. Is x a stegogramme (from S) or a pure image (from C)? Unfortunately, most of the time x ∈ C and x ∈ S are both possible. . . Compare Pr(S = x) and Pr(C = x). If Pr(S = x) > Pr(C = x), then x is most likely to be a stegogramme. How confident can we be about the conclusion?

Dr Hans Georg Schaathun Statistics and Steganalysis Spring 2007 13 / 52

Probabilities Probabilities

Statistical Steganalysis

The goal of Statistical Steganalysis: Wendy observes an image X sent from Alice to Bob. Let x be the observed image. Is x a stegogramme (from S) or a pure image (from C)? Unfortunately, most of the time x ∈ C and x ∈ S are both possible. . . Compare Pr(S = x) and Pr(C = x). If Pr(S = x) > Pr(C = x), then x is most likely to be a stegogramme. How confident can we be about the conclusion?

Dr Hans Georg Schaathun Statistics and Steganalysis Spring 2007 13 / 52

Probabilities Probabilities

Statistical Steganalysis

The goal of Statistical Steganalysis: Wendy observes an image X sent from Alice to Bob. Let x be the observed image. Is x a stegogramme (from S) or a pure image (from C)? Unfortunately, most of the time x ∈ C and x ∈ S are both possible. . . Compare Pr(S = x) and Pr(C = x). If Pr(S = x) > Pr(C = x), then x is most likely to be a stegogramme. How confident can we be about the conclusion?

Dr Hans Georg Schaathun Statistics and Steganalysis Spring 2007 13 / 52

Probabilities Probabilities

Statistical Steganalysis

The goal of Statistical Steganalysis: Wendy observes an image X sent from Alice to Bob. Let x be the observed image. Is x a stegogramme (from S) or a pure image (from C)? Unfortunately, most of the time x ∈ C and x ∈ S are both possible. . . Compare Pr(S = x) and Pr(C = x). If Pr(S = x) > Pr(C = x), then x is most likely to be a stegogramme. How confident can we be about the conclusion?

Dr Hans Georg Schaathun Statistics and Steganalysis Spring 2007 13 / 52

Probabilities Probabilities

Statistical Steganalysis

The goal of Statistical Steganalysis: Wendy observes an image X sent from Alice to Bob. Let x be the observed image. Is x a stegogramme (from S) or a pure image (from C)? Unfortunately, most of the time x ∈ C and x ∈ S are both possible. . . Compare Pr(S = x) and Pr(C = x). If Pr(S = x) > Pr(C = x), then x is most likely to be a stegogramme. How confident can we be about the conclusion?

Dr Hans Georg Schaathun Statistics and Steganalysis Spring 2007 13 / 52

Probabilities Probabilities

Statistical Steganalysis

If PS = PC,

i.e. if covertexts and stegogrammes have the same distributions

then no steganalysis is possible. If PS = PC, then statistics can help us. Remark The perfect stego-system gives PS = PC.

Dr Hans Georg Schaathun Statistics and Steganalysis Spring 2007 14 / 52

Probabilities Probabilities

Statistical Steganalysis

If PS = PC,

i.e. if covertexts and stegogrammes have the same distributions

then no steganalysis is possible. If PS = PC, then statistics can help us. Remark The perfect stego-system gives PS = PC.

Dr Hans Georg Schaathun Statistics and Steganalysis Spring 2007 14 / 52

Probabilities Probabilities

Statistical Steganalysis

If PS = PC,

i.e. if covertexts and stegogrammes have the same distributions

then no steganalysis is possible. If PS = PC, then statistics can help us. Remark The perfect stego-system gives PS = PC.

Dr Hans Georg Schaathun Statistics and Steganalysis Spring 2007 14 / 52

Probabilities Random variables

Outline

1

Probabilities Experiments and Events Probabilities Random variables Independence

2

Probability Distributions The uniform distribution Some other distributions

3

Hypothesis tests Introduction

4

Pairs of Values Steganalysis Pairs of Values

5

Generalised χ2 test

Dr Hans Georg Schaathun Statistics and Steganalysis Spring 2007 15 / 52

Probabilities Random variables

Random variable

Definition (Bhattacharyya and Johnson) A random variable X is a numerical valued function defined on a sample space. (. . . ) I.e. a number X(e) is assigned to each simple event. The random variable X is always a number. The event e could be anything (colour, multi-dimensional vector).

Dr Hans Georg Schaathun Statistics and Steganalysis Spring 2007 16 / 52

Probabilities Random variables

Random variable

Definition (Bhattacharyya and Johnson) A random variable X is a numerical valued function defined on a sample space. (. . . ) I.e. a number X(e) is assigned to each simple event. The random variable X is always a number. The event e could be anything (colour, multi-dimensional vector).

Dr Hans Georg Schaathun Statistics and Steganalysis Spring 2007 16 / 52

Probabilities Random variables

Random variable

Definition (Bhattacharyya and Johnson) A random variable X is a numerical valued function defined on a sample space. (. . . ) I.e. a number X(e) is assigned to each simple event. The random variable X is always a number. The event e could be anything (colour, multi-dimensional vector).

Dr Hans Georg Schaathun Statistics and Steganalysis Spring 2007 16 / 52

Probabilities Random variables

Random variable

Definition (Bhattacharyya and Johnson) A random variable X is a numerical valued function defined on a sample space. (. . . ) I.e. a number X(e) is assigned to each simple event. The random variable X is always a number. The event e could be anything (colour, multi-dimensional vector).

Dr Hans Georg Schaathun Statistics and Steganalysis Spring 2007 16 / 52

Probabilities Random variables

Discrete random variable

A discrete random variable X is characterised by The discrete set Ω = {x1, x2, . . .} from which it is drawn, The Probability Distribution pX : Ω → [0, 1], where pX(x) = Pr(X = x). Definition The cumulative density function (CDF) fX : Ω → [0, 1] of X is defined by FX(x) = Pr(X ≤ x). Remark FX(x) =

• y≤x

pX(x).

Dr Hans Georg Schaathun Statistics and Steganalysis Spring 2007 17 / 52

Probabilities Random variables

Discrete random variable

A discrete random variable X is characterised by The discrete set Ω = {x1, x2, . . .} from which it is drawn, The Probability Distribution pX : Ω → [0, 1], where pX(x) = Pr(X = x). Definition The cumulative density function (CDF) fX : Ω → [0, 1] of X is defined by FX(x) = Pr(X ≤ x). Remark FX(x) =

• y≤x

pX(x).

Dr Hans Georg Schaathun Statistics and Steganalysis Spring 2007 17 / 52

Probabilities Random variables

Discrete random variable

A discrete random variable X is characterised by The discrete set Ω = {x1, x2, . . .} from which it is drawn, The Probability Distribution pX : Ω → [0, 1], where pX(x) = Pr(X = x). Definition The cumulative density function (CDF) fX : Ω → [0, 1] of X is defined by FX(x) = Pr(X ≤ x). Remark FX(x) =

• y≤x

pX(x).

Dr Hans Georg Schaathun Statistics and Steganalysis Spring 2007 17 / 52

Probabilities Random variables

Discrete random variable

A discrete random variable X is characterised by The discrete set Ω = {x1, x2, . . .} from which it is drawn, The Probability Distribution pX : Ω → [0, 1], where pX(x) = Pr(X = x). Definition The cumulative density function (CDF) fX : Ω → [0, 1] of X is defined by FX(x) = Pr(X ≤ x). Remark FX(x) =

• y≤x

pX(x).

Dr Hans Georg Schaathun Statistics and Steganalysis Spring 2007 17 / 52

Probabilities Random variables

Continuous random variable

A continuous random variable Y is characterised by The continuous set Ω from which it is drawn, The probability density function (PDF) pY : Ω → [0, 1]. The CDF is FY(y) = Pr(Y ≤ y) = y pY(y)dy Clearly, Pr(Y = y) = 0 for all y.

Dr Hans Georg Schaathun Statistics and Steganalysis Spring 2007 18 / 52

Probabilities Random variables

Continuous random variable

A continuous random variable Y is characterised by The continuous set Ω from which it is drawn, The probability density function (PDF) pY : Ω → [0, 1]. The CDF is FY(y) = Pr(Y ≤ y) = y pY(y)dy Clearly, Pr(Y = y) = 0 for all y.

Dr Hans Georg Schaathun Statistics and Steganalysis Spring 2007 18 / 52

Probabilities Random variables

Continuous random variable

A continuous random variable Y is characterised by The continuous set Ω from which it is drawn, The probability density function (PDF) pY : Ω → [0, 1]. The CDF is FY(y) = Pr(Y ≤ y) = y pY(y)dy Clearly, Pr(Y = y) = 0 for all y.

Dr Hans Georg Schaathun Statistics and Steganalysis Spring 2007 18 / 52

Probabilities Random variables

Continuous random variable

A continuous random variable Y is characterised by The continuous set Ω from which it is drawn, The probability density function (PDF) pY : Ω → [0, 1]. The CDF is FY(y) = Pr(Y ≤ y) = y pY(y)dy Clearly, Pr(Y = y) = 0 for all y.

Dr Hans Georg Schaathun Statistics and Steganalysis Spring 2007 18 / 52

Probabilities Random variables

Example (random variables)

Consider a (random) M × N Grayscale Image X. X = [xxy]0<x≤M,0<y≤N, where each xij ∈ {0, 1, 2, . . . , 255}. The sample space is all M × N matrices of 8-bit integers. Let k = 0, . . . , 255 Define random variable: Yk = #{xxy|xxy = k}. We shall see later that the distribution of Yk, varies according to whether X is a stegogramme or a natural image.

Dr Hans Georg Schaathun Statistics and Steganalysis Spring 2007 19 / 52

Probabilities Random variables

Example (random variables)

Consider a (random) M × N Grayscale Image X. X = [xxy]0<x≤M,0<y≤N, where each xij ∈ {0, 1, 2, . . . , 255}. The sample space is all M × N matrices of 8-bit integers. Let k = 0, . . . , 255 Define random variable: Yk = #{xxy|xxy = k}. We shall see later that the distribution of Yk, varies according to whether X is a stegogramme or a natural image.

Dr Hans Georg Schaathun Statistics and Steganalysis Spring 2007 19 / 52

Probabilities Random variables

Example (random variables)

Consider a (random) M × N Grayscale Image X. X = [xxy]0<x≤M,0<y≤N, where each xij ∈ {0, 1, 2, . . . , 255}. The sample space is all M × N matrices of 8-bit integers. Let k = 0, . . . , 255 Define random variable: Yk = #{xxy|xxy = k}. We shall see later that the distribution of Yk, varies according to whether X is a stegogramme or a natural image.

Dr Hans Georg Schaathun Statistics and Steganalysis Spring 2007 19 / 52

Probabilities Random variables

Example (random variables)

Consider a (random) M × N Grayscale Image X. X = [xxy]0<x≤M,0<y≤N, where each xij ∈ {0, 1, 2, . . . , 255}. The sample space is all M × N matrices of 8-bit integers. Let k = 0, . . . , 255 Define random variable: Yk = #{xxy|xxy = k}. We shall see later that the distribution of Yk, varies according to whether X is a stegogramme or a natural image.

Dr Hans Georg Schaathun Statistics and Steganalysis Spring 2007 19 / 52

Probabilities Random variables

Example (random variables)

Consider a (random) M × N Grayscale Image X. X = [xxy]0<x≤M,0<y≤N, where each xij ∈ {0, 1, 2, . . . , 255}. The sample space is all M × N matrices of 8-bit integers. Let k = 0, . . . , 255 Define random variable: Yk = #{xxy|xxy = k}. We shall see later that the distribution of Yk, varies according to whether X is a stegogramme or a natural image.

Dr Hans Georg Schaathun Statistics and Steganalysis Spring 2007 19 / 52

Probabilities Random variables

Example (random variables)

Consider a (random) M × N Grayscale Image X. X = [xxy]0<x≤M,0<y≤N, where each xij ∈ {0, 1, 2, . . . , 255}. The sample space is all M × N matrices of 8-bit integers. Let k = 0, . . . , 255 Define random variable: Yk = #{xxy|xxy = k}. We shall see later that the distribution of Yk, varies according to whether X is a stegogramme or a natural image.

Dr Hans Georg Schaathun Statistics and Steganalysis Spring 2007 19 / 52

Probabilities Random variables

Example (random variables)

Consider a (random) M × N Grayscale Image X. X = [xxy]0<x≤M,0<y≤N, where each xij ∈ {0, 1, 2, . . . , 255}. The sample space is all M × N matrices of 8-bit integers. Let k = 0, . . . , 255 Define random variable: Yk = #{xxy|xxy = k}. We shall see later that the distribution of Yk, varies according to whether X is a stegogramme or a natural image.

Dr Hans Georg Schaathun Statistics and Steganalysis Spring 2007 19 / 52

Probabilities Random variables

Expectation

Definition The expectation of a discrete random variable X is E(X) =

• x∈X

xPr(X = x), where X is the set of possible values for X. For a continuous variable, this becomes E(X) = ∞

−∞

xpX(x)dx.

Dr Hans Georg Schaathun Statistics and Steganalysis Spring 2007 20 / 52

Probabilities Random variables

Expectation

Definition The expectation of a discrete random variable X is E(X) =

• x∈X

xPr(X = x), where X is the set of possible values for X. For a continuous variable, this becomes E(X) = ∞

−∞

xpX(x)dx.

Dr Hans Georg Schaathun Statistics and Steganalysis Spring 2007 20 / 52

Probabilities Random variables

Variance

Definition The variance of a discrete random variable X is Var(X) =

• x∈X

(x − E(X))2pX(x), where X is the set of possible values for X. For a continuous variable, it becomes Var(X) = ∞

−∞

(x − E(X))2pX(x)dx.

Dr Hans Georg Schaathun Statistics and Steganalysis Spring 2007 21 / 52

Probabilities Random variables

Variance

Definition The variance of a discrete random variable X is Var(X) =

• x∈X

(x − E(X))2pX(x), where X is the set of possible values for X. For a continuous variable, it becomes Var(X) = ∞

−∞

(x − E(X))2pX(x)dx.

Dr Hans Georg Schaathun Statistics and Steganalysis Spring 2007 21 / 52

Probabilities Random variables

Example: Expectation

Consider a stegogramme X as a grayscale image, Yk = #{xxy|xxy = k}, as discussed earlier. (2l, 2l + 1) is a pairs of values

The embedding changes 2l → 2l + 1 and vice versa.

Suppose the hidden message is a uniformly random bit string.

Then E(Y2k+1) = E(Y2k). This is not the case if X is a pure image.

Dr Hans Georg Schaathun Statistics and Steganalysis Spring 2007 22 / 52

Probabilities Random variables

Example: Expectation

Consider a stegogramme X as a grayscale image, Yk = #{xxy|xxy = k}, as discussed earlier. (2l, 2l + 1) is a pairs of values

The embedding changes 2l → 2l + 1 and vice versa.

Suppose the hidden message is a uniformly random bit string.

Then E(Y2k+1) = E(Y2k). This is not the case if X is a pure image.

Dr Hans Georg Schaathun Statistics and Steganalysis Spring 2007 22 / 52

Probabilities Random variables

Example: Expectation

Consider a stegogramme X as a grayscale image, Yk = #{xxy|xxy = k}, as discussed earlier. (2l, 2l + 1) is a pairs of values

The embedding changes 2l → 2l + 1 and vice versa.

Suppose the hidden message is a uniformly random bit string.

Then E(Y2k+1) = E(Y2k). This is not the case if X is a pure image.

Dr Hans Georg Schaathun Statistics and Steganalysis Spring 2007 22 / 52

Probabilities Random variables

Example: Expectation

Consider a stegogramme X as a grayscale image, Yk = #{xxy|xxy = k}, as discussed earlier. (2l, 2l + 1) is a pairs of values

The embedding changes 2l → 2l + 1 and vice versa.

Suppose the hidden message is a uniformly random bit string.

Then E(Y2k+1) = E(Y2k). This is not the case if X is a pure image.

Dr Hans Georg Schaathun Statistics and Steganalysis Spring 2007 22 / 52

Probabilities Random variables

Example: Expectation

Consider a stegogramme X as a grayscale image, Yk = #{xxy|xxy = k}, as discussed earlier. (2l, 2l + 1) is a pairs of values

The embedding changes 2l → 2l + 1 and vice versa.

Suppose the hidden message is a uniformly random bit string.

Then E(Y2k+1) = E(Y2k). This is not the case if X is a pure image.

Dr Hans Georg Schaathun Statistics and Steganalysis Spring 2007 22 / 52

Probabilities Random variables

Example: Expectation

Consider a stegogramme X as a grayscale image, Yk = #{xxy|xxy = k}, as discussed earlier. (2l, 2l + 1) is a pairs of values

The embedding changes 2l → 2l + 1 and vice versa.

Suppose the hidden message is a uniformly random bit string.

Then E(Y2k+1) = E(Y2k). This is not the case if X is a pure image.

Dr Hans Georg Schaathun Statistics and Steganalysis Spring 2007 22 / 52

Probabilities Independence

Outline

1

Probabilities Experiments and Events Probabilities Random variables Independence

2

Probability Distributions The uniform distribution Some other distributions

3

Hypothesis tests Introduction

4

Pairs of Values Steganalysis Pairs of Values

5

Generalised χ2 test

Dr Hans Georg Schaathun Statistics and Steganalysis Spring 2007 23 / 52

Probabilities Independence

Independence

The definition

Consider two events A and B. The probability that both happens is denoted Pr(A, B). Definition We say that A and B are independent if Pr(A, B) = Pr(A)Pr(B).

Dr Hans Georg Schaathun Statistics and Steganalysis Spring 2007 24 / 52

Probabilities Independence

Independence

The definition

Consider two events A and B. The probability that both happens is denoted Pr(A, B). Definition We say that A and B are independent if Pr(A, B) = Pr(A)Pr(B).

Dr Hans Georg Schaathun Statistics and Steganalysis Spring 2007 24 / 52

Probabilities Independence

Independence

Examples

Example You roll two dice, a green and a red one. Let A be the event of a 6 on the red die, and B a 6 on the green die. We have Pr(B) = Pr(A) = 1/6 and Pr(A, B) = 1/36. The events are independent. Example Let C be the event of rolling a sum of 6 on the two dice. Now Pr(C) = 5/36, but if A the sum is at least 7, so Pr(A, C) = 0. The events A and C are dependent.

Dr Hans Georg Schaathun Statistics and Steganalysis Spring 2007 25 / 52

Probabilities Independence

Independence

Examples

Example You roll two dice, a green and a red one. Let A be the event of a 6 on the red die, and B a 6 on the green die. We have Pr(B) = Pr(A) = 1/6 and Pr(A, B) = 1/36. The events are independent. Example Let C be the event of rolling a sum of 6 on the two dice. Now Pr(C) = 5/36, but if A the sum is at least 7, so Pr(A, C) = 0. The events A and C are dependent.

Dr Hans Georg Schaathun Statistics and Steganalysis Spring 2007 25 / 52

Probabilities Independence

Conditional probability

Definition Consider two events A and B. The probability of A happening in the case when B is known to happen is known as the conditional probability Pr(A|B). Theorem (Bayes formula) Pr(A|B)Pr(B) = Pr(B|A)Pr(A) = Pr(A, B)

Dr Hans Georg Schaathun Statistics and Steganalysis Spring 2007 26 / 52

Probability Distributions The uniform distribution

Outline

1

Probabilities Experiments and Events Probabilities Random variables Independence

2

Probability Distributions The uniform distribution Some other distributions

3

Hypothesis tests Introduction

4

Pairs of Values Steganalysis Pairs of Values

5

Generalised χ2 test

Dr Hans Georg Schaathun Statistics and Steganalysis Spring 2007 27 / 52

Probability Distributions The uniform distribution

The uniform distribution

Let X be a random variable from a set Ω. Let p(x) be the probability distribution of X.

• r the probability density function if X is continuous

If p(x) = p(y) for all x, y ∈ Ω, then

we say that X is uniformly distributed on Ω

Dr Hans Georg Schaathun Statistics and Steganalysis Spring 2007 28 / 52

Probability Distributions The uniform distribution

Distributions in graphics

Discrete random variable Probability distribution as histogramme Continuous random variable Probability density function as plot

Dr Hans Georg Schaathun Statistics and Steganalysis Spring 2007 29 / 52

Probability Distributions The uniform distribution

Distributions in graphics

Discrete random variable Probability distribution as histogramme Continuous random variable Probability density function as plot

Dr Hans Georg Schaathun Statistics and Steganalysis Spring 2007 29 / 52

Probability Distributions Some other distributions

Outline

1

Probabilities Experiments and Events Probabilities Random variables Independence

2

Probability Distributions The uniform distribution Some other distributions

3

Hypothesis tests Introduction

4

Pairs of Values Steganalysis Pairs of Values

5

Generalised χ2 test

Dr Hans Georg Schaathun Statistics and Steganalysis Spring 2007 30 / 52

Probability Distributions Some other distributions

The χ2 distribution

The χ2 distribution will be used later. Defined for k degrees of freedom k = 1, 2, . . .

Dr Hans Georg Schaathun Statistics and Steganalysis Spring 2007 31 / 52

Probability Distributions Some other distributions

Probability density function

The χ2 distribution

Dr Hans Georg Schaathun Statistics and Steganalysis Spring 2007 32 / 52

Probability Distributions Some other distributions

How to use it

(Usually) No need to learn the formulæ Look probabilities up in tables . . . or in Matlab

chi2pdf ( x, k ) gives the PDF at x for k degrees of freedom chi2cdf ( x, k ) gives P(X ≤ x) for k degrees of freedom Inverse function exists too (use help functions)

Dr Hans Georg Schaathun Statistics and Steganalysis Spring 2007 33 / 52

Probability Distributions Some other distributions

Random bit strings

We will refer to random bit strings

X = (X1, X2, . . . , XN) If X is an unbiased random bit string, then

Xi are identically distributed with P(Xi = 0) = P(Xi = 1) = 1

2; and

each Xi is independent of all other Xj.

If X is an biased random bit string, then

Xi are identically distributed with P(Xi = 0) = p = 1

2; and

each Xi is independent of all other Xj.

Note the significance of independence

Dr Hans Georg Schaathun Statistics and Steganalysis Spring 2007 34 / 52

Probability Distributions Some other distributions

Random bit strings

We will refer to random bit strings

X = (X1, X2, . . . , XN) If X is an unbiased random bit string, then

Xi are identically distributed with P(Xi = 0) = P(Xi = 1) = 1

2; and

each Xi is independent of all other Xj.

If X is an biased random bit string, then

Xi are identically distributed with P(Xi = 0) = p = 1

2; and

each Xi is independent of all other Xj.

Note the significance of independence

Dr Hans Georg Schaathun Statistics and Steganalysis Spring 2007 34 / 52

Probability Distributions Some other distributions

Random bit strings

We will refer to random bit strings

X = (X1, X2, . . . , XN) If X is an unbiased random bit string, then

Xi are identically distributed with P(Xi = 0) = P(Xi = 1) = 1

2; and

each Xi is independent of all other Xj.

If X is an biased random bit string, then

Xi are identically distributed with P(Xi = 0) = p = 1

2; and

each Xi is independent of all other Xj.

Note the significance of independence

Dr Hans Georg Schaathun Statistics and Steganalysis Spring 2007 34 / 52

Probability Distributions Some other distributions

Random bit strings

We will refer to random bit strings

X = (X1, X2, . . . , XN) If X is an unbiased random bit string, then

Xi are identically distributed with P(Xi = 0) = P(Xi = 1) = 1

2; and

each Xi is independent of all other Xj.

If X is an biased random bit string, then

Xi are identically distributed with P(Xi = 0) = p = 1

2; and

each Xi is independent of all other Xj.

Note the significance of independence

Dr Hans Georg Schaathun Statistics and Steganalysis Spring 2007 34 / 52

Probability Distributions Some other distributions

Random bit strings

We will refer to random bit strings

X = (X1, X2, . . . , XN) If X is an unbiased random bit string, then

Xi are identically distributed with P(Xi = 0) = P(Xi = 1) = 1

2; and

each Xi is independent of all other Xj.

If X is an biased random bit string, then

Xi are identically distributed with P(Xi = 0) = p = 1

2; and

each Xi is independent of all other Xj.

Note the significance of independence

Dr Hans Georg Schaathun Statistics and Steganalysis Spring 2007 34 / 52

Probability Distributions Some other distributions

Random bit strings

We will refer to random bit strings

X = (X1, X2, . . . , XN) If X is an unbiased random bit string, then

Xi are identically distributed with P(Xi = 0) = P(Xi = 1) = 1

2; and

each Xi is independent of all other Xj.

If X is an biased random bit string, then

Xi are identically distributed with P(Xi = 0) = p = 1

2; and

each Xi is independent of all other Xj.

Note the significance of independence

Dr Hans Georg Schaathun Statistics and Steganalysis Spring 2007 34 / 52

Probability Distributions Some other distributions

Random bit strings

We will refer to random bit strings

X = (X1, X2, . . . , XN) If X is an unbiased random bit string, then

Xi are identically distributed with P(Xi = 0) = P(Xi = 1) = 1

2; and

each Xi is independent of all other Xj.

If X is an biased random bit string, then

Xi are identically distributed with P(Xi = 0) = p = 1

2; and

each Xi is independent of all other Xj.

Note the significance of independence

Dr Hans Georg Schaathun Statistics and Steganalysis Spring 2007 34 / 52

Probability Distributions Some other distributions

Random bit strings

We will refer to random bit strings

X = (X1, X2, . . . , XN) If X is an unbiased random bit string, then

Xi are identically distributed with P(Xi = 0) = P(Xi = 1) = 1

2; and

each Xi is independent of all other Xj.

If X is an biased random bit string, then

Xi are identically distributed with P(Xi = 0) = p = 1

2; and

each Xi is independent of all other Xj.

Note the significance of independence

Dr Hans Georg Schaathun Statistics and Steganalysis Spring 2007 34 / 52

Probability Distributions Some other distributions

Random bit strings

We will refer to random bit strings

X = (X1, X2, . . . , XN) If X is an unbiased random bit string, then

Xi are identically distributed with P(Xi = 0) = P(Xi = 1) = 1

2; and

each Xi is independent of all other Xj.

If X is an biased random bit string, then

Xi are identically distributed with P(Xi = 0) = p = 1

2; and

each Xi is independent of all other Xj.

Note the significance of independence

Dr Hans Georg Schaathun Statistics and Steganalysis Spring 2007 34 / 52

Probability Distributions Some other distributions

Random bit strings

We will refer to random bit strings

X = (X1, X2, . . . , XN) If X is an unbiased random bit string, then

Xi are identically distributed with P(Xi = 0) = P(Xi = 1) = 1

2; and

each Xi is independent of all other Xj.

If X is an biased random bit string, then

Xi are identically distributed with P(Xi = 0) = p = 1

2; and

each Xi is independent of all other Xj.

Note the significance of independence

Dr Hans Georg Schaathun Statistics and Steganalysis Spring 2007 34 / 52

Hypothesis tests Introduction

Outline

1

Probabilities Experiments and Events Probabilities Random variables Independence

2

Probability Distributions The uniform distribution Some other distributions

3

Hypothesis tests Introduction

4

Pairs of Values Steganalysis Pairs of Values

5

Generalised χ2 test

Dr Hans Georg Schaathun Statistics and Steganalysis Spring 2007 35 / 52

Hypothesis tests Introduction

Hypothesis tests

Hypothesis testing is a recurring theme in statistics. Typical hypotheses

Treatment A makes patients recover more quickly than no treatment. The climate in South-East Britain is as warm today as it was a 100 years ago. The image sent by Alice is a stegogramme.

When the hypothesis has been phrased,

experiments can tell us whether it is plausible or not.

Dr Hans Georg Schaathun Statistics and Steganalysis Spring 2007 36 / 52

Hypothesis tests Introduction

Hypothesis tests

Hypothesis testing is a recurring theme in statistics. Typical hypotheses

Treatment A makes patients recover more quickly than no treatment. The climate in South-East Britain is as warm today as it was a 100 years ago. The image sent by Alice is a stegogramme.

When the hypothesis has been phrased,

experiments can tell us whether it is plausible or not.

Dr Hans Georg Schaathun Statistics and Steganalysis Spring 2007 36 / 52

Hypothesis tests Introduction

Hypothesis tests

Hypothesis testing is a recurring theme in statistics. Typical hypotheses

Treatment A makes patients recover more quickly than no treatment. The climate in South-East Britain is as warm today as it was a 100 years ago. The image sent by Alice is a stegogramme.

When the hypothesis has been phrased,

experiments can tell us whether it is plausible or not.

Dr Hans Georg Schaathun Statistics and Steganalysis Spring 2007 36 / 52

Hypothesis tests Introduction

Hypothesis tests

Hypothesis testing is a recurring theme in statistics. Typical hypotheses

Treatment A makes patients recover more quickly than no treatment. The climate in South-East Britain is as warm today as it was a 100 years ago. The image sent by Alice is a stegogramme.

When the hypothesis has been phrased,

experiments can tell us whether it is plausible or not.

Dr Hans Georg Schaathun Statistics and Steganalysis Spring 2007 36 / 52

Hypothesis tests Introduction

Hypothesis tests

Hypothesis testing is a recurring theme in statistics. Typical hypotheses

Treatment A makes patients recover more quickly than no treatment. The climate in South-East Britain is as warm today as it was a 100 years ago. The image sent by Alice is a stegogramme.

When the hypothesis has been phrased,

experiments can tell us whether it is plausible or not.

Dr Hans Georg Schaathun Statistics and Steganalysis Spring 2007 36 / 52

Hypothesis tests Introduction

Hypothesis tests

Hypothesis testing is a recurring theme in statistics. Typical hypotheses

Treatment A makes patients recover more quickly than no treatment. The climate in South-East Britain is as warm today as it was a 100 years ago. The image sent by Alice is a stegogramme.

When the hypothesis has been phrased,

experiments can tell us whether it is plausible or not.

Dr Hans Georg Schaathun Statistics and Steganalysis Spring 2007 36 / 52

Hypothesis tests Introduction

Hypothesis tests

Hypothesis testing is a recurring theme in statistics. Typical hypotheses

Treatment A makes patients recover more quickly than no treatment. The climate in South-East Britain is as warm today as it was a 100 years ago. The image sent by Alice is a stegogramme.

When the hypothesis has been phrased,

experiments can tell us whether it is plausible or not.

Dr Hans Georg Schaathun Statistics and Steganalysis Spring 2007 36 / 52

Hypothesis tests Introduction

The test statistic

Make a null hypothesis H0. Formulate an alternative hypothesis H1 (negating H0). Identify a random variable X with known distribution assuming H0 Observe the random variable. Is the observed value x a likely result under H0? We decide on a threshold t such that

Pr(X > t|H0) is small

If the observed x > t we reject H0. We can never prove H0, or reject H1.

Dr Hans Georg Schaathun Statistics and Steganalysis Spring 2007 37 / 52

Hypothesis tests Introduction

The test statistic

Make a null hypothesis H0. Formulate an alternative hypothesis H1 (negating H0). Identify a random variable X with known distribution assuming H0 Observe the random variable. Is the observed value x a likely result under H0? We decide on a threshold t such that

Pr(X > t|H0) is small

If the observed x > t we reject H0. We can never prove H0, or reject H1.

Dr Hans Georg Schaathun Statistics and Steganalysis Spring 2007 37 / 52

Hypothesis tests Introduction

The test statistic

Make a null hypothesis H0. Formulate an alternative hypothesis H1 (negating H0). Identify a random variable X with known distribution assuming H0 Observe the random variable. Is the observed value x a likely result under H0? We decide on a threshold t such that

Pr(X > t|H0) is small

If the observed x > t we reject H0. We can never prove H0, or reject H1.

Dr Hans Georg Schaathun Statistics and Steganalysis Spring 2007 37 / 52

Hypothesis tests Introduction

The test statistic

Make a null hypothesis H0. Formulate an alternative hypothesis H1 (negating H0). Identify a random variable X with known distribution assuming H0 Observe the random variable. Is the observed value x a likely result under H0? We decide on a threshold t such that

Pr(X > t|H0) is small

If the observed x > t we reject H0. We can never prove H0, or reject H1.

Dr Hans Georg Schaathun Statistics and Steganalysis Spring 2007 37 / 52

Hypothesis tests Introduction

The test statistic

Make a null hypothesis H0. Formulate an alternative hypothesis H1 (negating H0). Identify a random variable X with known distribution assuming H0 Observe the random variable. Is the observed value x a likely result under H0? We decide on a threshold t such that

Pr(X > t|H0) is small

If the observed x > t we reject H0. We can never prove H0, or reject H1.

Dr Hans Georg Schaathun Statistics and Steganalysis Spring 2007 37 / 52

Hypothesis tests Introduction

The test statistic

Make a null hypothesis H0. Formulate an alternative hypothesis H1 (negating H0). Identify a random variable X with known distribution assuming H0 Observe the random variable. Is the observed value x a likely result under H0? We decide on a threshold t such that

Pr(X > t|H0) is small

If the observed x > t we reject H0. We can never prove H0, or reject H1.

Dr Hans Georg Schaathun Statistics and Steganalysis Spring 2007 37 / 52

Hypothesis tests Introduction

The test statistic

Make a null hypothesis H0. Formulate an alternative hypothesis H1 (negating H0). Identify a random variable X with known distribution assuming H0 Observe the random variable. Is the observed value x a likely result under H0? We decide on a threshold t such that

Pr(X > t|H0) is small

If the observed x > t we reject H0. We can never prove H0, or reject H1.

Dr Hans Georg Schaathun Statistics and Steganalysis Spring 2007 37 / 52

Hypothesis tests Introduction

The test statistic

Make a null hypothesis H0. Formulate an alternative hypothesis H1 (negating H0). Identify a random variable X with known distribution assuming H0 Observe the random variable. Is the observed value x a likely result under H0? We decide on a threshold t such that

Pr(X > t|H0) is small

If the observed x > t we reject H0. We can never prove H0, or reject H1.

Dr Hans Georg Schaathun Statistics and Steganalysis Spring 2007 37 / 52

Hypothesis tests Introduction

The test statistic

Make a null hypothesis H0. Formulate an alternative hypothesis H1 (negating H0). Identify a random variable X with known distribution assuming H0 Observe the random variable. Is the observed value x a likely result under H0? We decide on a threshold t such that

Pr(X > t|H0) is small

If the observed x > t we reject H0. We can never prove H0, or reject H1.

Dr Hans Georg Schaathun Statistics and Steganalysis Spring 2007 37 / 52

Hypothesis tests Introduction

Example: A χ2 statistic

Consider k random variables: X1, . . . , Xk. Suppose E(Xi) = Ni for all i. Define S =

k

• i=1

(Xi − Ni)2 Ni . Now S is approximately χ2 distributed with k − 1 degrees of freedom.

Dr Hans Georg Schaathun Statistics and Steganalysis Spring 2007 38 / 52

Hypothesis tests Introduction

The two error types

H0 retained H0 rejected H0 true No error Error Type I H0 false Error Type II No error

Dr Hans Georg Schaathun Statistics and Steganalysis Spring 2007 39 / 52

Hypothesis tests Introduction

Calculating probability of Type I Errors

Definition A Type I Error is the event that H0 is true; and H0 is rejected. What is the error rate? We want to calculate the conditional probability Pr(Reject H0|H0) = Pr(X > t|H0). Because of H0, distribution of X is known. Hence the error probability can be looked up.

Dr Hans Georg Schaathun Statistics and Steganalysis Spring 2007 40 / 52

Hypothesis tests Introduction

Calculating probability of Type I Errors

Definition A Type I Error is the event that H0 is true; and H0 is rejected. What is the error rate? We want to calculate the conditional probability Pr(Reject H0|H0) = Pr(X > t|H0). Because of H0, distribution of X is known. Hence the error probability can be looked up.

Dr Hans Georg Schaathun Statistics and Steganalysis Spring 2007 40 / 52

Hypothesis tests Introduction

Calculating probability of Type I Errors

Definition A Type I Error is the event that H0 is true; and H0 is rejected. What is the error rate? We want to calculate the conditional probability Pr(Reject H0|H0) = Pr(X > t|H0). Because of H0, distribution of X is known. Hence the error probability can be looked up.

Dr Hans Georg Schaathun Statistics and Steganalysis Spring 2007 40 / 52

Hypothesis tests Introduction

Calculating probability of Type I Errors

Definition A Type I Error is the event that H0 is true; and H0 is rejected. What is the error rate? We want to calculate the conditional probability Pr(Reject H0|H0) = Pr(X > t|H0). Because of H0, distribution of X is known. Hence the error probability can be looked up.

Dr Hans Georg Schaathun Statistics and Steganalysis Spring 2007 40 / 52

Hypothesis tests Introduction

Calculating probability of Type I Errors

Definition A Type I Error is the event that H0 is true; and H0 is rejected. What is the error rate? We want to calculate the conditional probability Pr(Reject H0|H0) = Pr(X > t|H0). Because of H0, distribution of X is known. Hence the error probability can be looked up.

Dr Hans Georg Schaathun Statistics and Steganalysis Spring 2007 40 / 52

Hypothesis tests Introduction

Calculating probability of Type I Errors

Definition A Type I Error is the event that H0 is true; and H0 is rejected. What is the error rate? We want to calculate the conditional probability Pr(Reject H0|H0) = Pr(X > t|H0). Because of H0, distribution of X is known. Hence the error probability can be looked up.

Dr Hans Georg Schaathun Statistics and Steganalysis Spring 2007 40 / 52

Hypothesis tests Introduction

Calculating probability of Type I Errors

Definition A Type I Error is the event that H0 is true; and H0 is rejected. What is the error rate? We want to calculate the conditional probability Pr(Reject H0|H0) = Pr(X > t|H0). Because of H0, distribution of X is known. Hence the error probability can be looked up.

Dr Hans Georg Schaathun Statistics and Steganalysis Spring 2007 40 / 52

Hypothesis tests Introduction

Type II Errors

In theory: Similar to Type I Errors. In practice: What is the distribution of X when H0 is false?

Do we know this distribution at all?

Remark Very often, we will not know the error probability.

Dr Hans Georg Schaathun Statistics and Steganalysis Spring 2007 41 / 52

Hypothesis tests Introduction

Type II Errors

In theory: Similar to Type I Errors. In practice: What is the distribution of X when H0 is false?

Do we know this distribution at all?

Remark Very often, we will not know the error probability.

Dr Hans Georg Schaathun Statistics and Steganalysis Spring 2007 41 / 52

Hypothesis tests Introduction

Type II Errors

In theory: Similar to Type I Errors. In practice: What is the distribution of X when H0 is false?

Do we know this distribution at all?

Remark Very often, we will not know the error probability.

Dr Hans Georg Schaathun Statistics and Steganalysis Spring 2007 41 / 52

Hypothesis tests Introduction

Type II Errors

In theory: Similar to Type I Errors. In practice: What is the distribution of X when H0 is false?

Do we know this distribution at all?

Remark Very often, we will not know the error probability.

Dr Hans Georg Schaathun Statistics and Steganalysis Spring 2007 41 / 52

Hypothesis tests Introduction

Level of Significance

Choose the threshold t such that Pr(X > t|H0) < α. The level of significance in the test is α. If we observe X > t, we reject H0 at significance level α If we observe X < t, we could not reject H0 at a significance level α The probability that H0 is correct is not α. The probability that H0 is false is not α either. Remark No simple relation between level of significance and the probability of any hypothesis being right or wrong.

Dr Hans Georg Schaathun Statistics and Steganalysis Spring 2007 42 / 52

Hypothesis tests Introduction

Level of Significance

Choose the threshold t such that Pr(X > t|H0) < α. The level of significance in the test is α. If we observe X > t, we reject H0 at significance level α If we observe X < t, we could not reject H0 at a significance level α The probability that H0 is correct is not α. The probability that H0 is false is not α either. Remark No simple relation between level of significance and the probability of any hypothesis being right or wrong.

Dr Hans Georg Schaathun Statistics and Steganalysis Spring 2007 42 / 52

Hypothesis tests Introduction

Level of Significance

Choose the threshold t such that Pr(X > t|H0) < α. The level of significance in the test is α. If we observe X > t, we reject H0 at significance level α If we observe X < t, we could not reject H0 at a significance level α The probability that H0 is correct is not α. The probability that H0 is false is not α either. Remark No simple relation between level of significance and the probability of any hypothesis being right or wrong.

Dr Hans Georg Schaathun Statistics and Steganalysis Spring 2007 42 / 52

Hypothesis tests Introduction

Level of Significance

Choose the threshold t such that Pr(X > t|H0) < α. The level of significance in the test is α. If we observe X > t, we reject H0 at significance level α If we observe X < t, we could not reject H0 at a significance level α The probability that H0 is correct is not α. The probability that H0 is false is not α either. Remark No simple relation between level of significance and the probability of any hypothesis being right or wrong.

Dr Hans Georg Schaathun Statistics and Steganalysis Spring 2007 42 / 52

Hypothesis tests Introduction

Level of Significance

Choose the threshold t such that Pr(X > t|H0) < α. The level of significance in the test is α. If we observe X > t, we reject H0 at significance level α If we observe X < t, we could not reject H0 at a significance level α The probability that H0 is correct is not α. The probability that H0 is false is not α either. Remark No simple relation between level of significance and the probability of any hypothesis being right or wrong.

Dr Hans Georg Schaathun Statistics and Steganalysis Spring 2007 42 / 52

Hypothesis tests Introduction

Level of Significance

Choose the threshold t such that Pr(X > t|H0) < α. The level of significance in the test is α. If we observe X > t, we reject H0 at significance level α If we observe X < t, we could not reject H0 at a significance level α The probability that H0 is correct is not α. The probability that H0 is false is not α either. Remark No simple relation between level of significance and the probability of any hypothesis being right or wrong.

Dr Hans Georg Schaathun Statistics and Steganalysis Spring 2007 42 / 52

Hypothesis tests Introduction

Level of Significance

Choose the threshold t such that Pr(X > t|H0) < α. The level of significance in the test is α. If we observe X > t, we reject H0 at significance level α If we observe X < t, we could not reject H0 at a significance level α The probability that H0 is correct is not α. The probability that H0 is false is not α either. Remark No simple relation between level of significance and the probability of any hypothesis being right or wrong.

Dr Hans Georg Schaathun Statistics and Steganalysis Spring 2007 42 / 52

Hypothesis tests Introduction

The χ2 test

X1, X2, . . . , XN are independent, identically distributed random variables from Ω Let Fo = #{i|Xi = o} be the number of times o occurs in the sequence Xi Define S =

• ∈Ω

(Fo−E(Fo))2 E(Fo)

Let k = #Ω S is approximately χ2 distributed with k − 1 degrees of freedom. We have assumed E(Fo) > 4 for all o ∈ Ω

Dr Hans Georg Schaathun Statistics and Steganalysis Spring 2007 43 / 52

Pairs of Values Steganalysis Pairs of Values

Outline

1

Probabilities Experiments and Events Probabilities Random variables Independence

2

Probability Distributions The uniform distribution Some other distributions

3

Hypothesis tests Introduction

4

Pairs of Values Steganalysis Pairs of Values

5

Generalised χ2 test

Dr Hans Georg Schaathun Statistics and Steganalysis Spring 2007 44 / 52

Pairs of Values Steganalysis Pairs of Values

Suggested reading

Core Reading Jessica Fridrich, Miroslav Goljan, David Soukal: ‘Higher-Order Statistical Steganalysis of Palette Images’ Proc. SPIE Electronic Imaging, Jan 2003, pp. 178-190.

Dr Hans Georg Schaathun Statistics and Steganalysis Spring 2007 45 / 52

Pairs of Values Steganalysis Pairs of Values

Pairs of Values

The statistic

Image X. Random variable Yk = #{(x, y)|Xxy = k}. Recall that (2l, 2l + 1) is a pair of values.

First 7 pixel bits determined by image colour. Last bit (LSB) determined by message.

Sum Y2l + Y2l+1 is unaffected by embedding. For a random message E(Y2l) = 1 2(Y2l + Y2l+1). Can we make a χ2 statistic out of this?

Dr Hans Georg Schaathun Statistics and Steganalysis Spring 2007 46 / 52

Pairs of Values Steganalysis Pairs of Values

Pairs of Values

The statistic

Image X. Random variable Yk = #{(x, y)|Xxy = k}. Recall that (2l, 2l + 1) is a pair of values.

First 7 pixel bits determined by image colour. Last bit (LSB) determined by message.

Sum Y2l + Y2l+1 is unaffected by embedding. For a random message E(Y2l) = 1 2(Y2l + Y2l+1). Can we make a χ2 statistic out of this?

Dr Hans Georg Schaathun Statistics and Steganalysis Spring 2007 46 / 52

Pairs of Values Steganalysis Pairs of Values

Pairs of Values

The statistic

Image X. Random variable Yk = #{(x, y)|Xxy = k}. Recall that (2l, 2l + 1) is a pair of values.

First 7 pixel bits determined by image colour. Last bit (LSB) determined by message.

Sum Y2l + Y2l+1 is unaffected by embedding. For a random message E(Y2l) = 1 2(Y2l + Y2l+1). Can we make a χ2 statistic out of this?

Dr Hans Georg Schaathun Statistics and Steganalysis Spring 2007 46 / 52

Pairs of Values Steganalysis Pairs of Values

Pairs of Values

The statistic

Image X. Random variable Yk = #{(x, y)|Xxy = k}. Recall that (2l, 2l + 1) is a pair of values.

First 7 pixel bits determined by image colour. Last bit (LSB) determined by message.

Sum Y2l + Y2l+1 is unaffected by embedding. For a random message E(Y2l) = 1 2(Y2l + Y2l+1). Can we make a χ2 statistic out of this?

Dr Hans Georg Schaathun Statistics and Steganalysis Spring 2007 46 / 52

Pairs of Values Steganalysis Pairs of Values

Pairs of Values

The statistic

Image X. Random variable Yk = #{(x, y)|Xxy = k}. Recall that (2l, 2l + 1) is a pair of values.

First 7 pixel bits determined by image colour. Last bit (LSB) determined by message.

Sum Y2l + Y2l+1 is unaffected by embedding. For a random message E(Y2l) = 1 2(Y2l + Y2l+1). Can we make a χ2 statistic out of this?

Dr Hans Georg Schaathun Statistics and Steganalysis Spring 2007 46 / 52

Pairs of Values Steganalysis Pairs of Values

Pairs of Values

The statistic

Image X. Random variable Yk = #{(x, y)|Xxy = k}. Recall that (2l, 2l + 1) is a pair of values.

First 7 pixel bits determined by image colour. Last bit (LSB) determined by message.

Sum Y2l + Y2l+1 is unaffected by embedding. For a random message E(Y2l) = 1 2(Y2l + Y2l+1). Can we make a χ2 statistic out of this?

Dr Hans Georg Schaathun Statistics and Steganalysis Spring 2007 46 / 52

Pairs of Values Steganalysis Pairs of Values

Pairs of Values

The statistic

Image X. Random variable Yk = #{(x, y)|Xxy = k}. Recall that (2l, 2l + 1) is a pair of values.

First 7 pixel bits determined by image colour. Last bit (LSB) determined by message.

Sum Y2l + Y2l+1 is unaffected by embedding. For a random message E(Y2l) = 1 2(Y2l + Y2l+1). Can we make a χ2 statistic out of this?

Dr Hans Georg Schaathun Statistics and Steganalysis Spring 2007 46 / 52

Pairs of Values Steganalysis Pairs of Values

Pairs of Values

The definition

S =

• ∈Ω

(Fo − E(Fo))2 E(Fo) , (general χ2 statistic) S =

127

• l=0

(Y2l − 1

2(Y2l + Y2l+1))2 1 2(Y2l + Y2l+1)

. (pairs of values) Definition SPoV =

127

• l=0

(Y2l − Y2l+1)2 2(Y2l + Y2l+1).

Dr Hans Georg Schaathun Statistics and Steganalysis Spring 2007 47 / 52

Pairs of Values Steganalysis Pairs of Values

Pairs of Values

The definition

S =

• ∈Ω

(Fo − E(Fo))2 E(Fo) , (general χ2 statistic) S =

127

• l=0

(Y2l − 1

2(Y2l + Y2l+1))2 1 2(Y2l + Y2l+1)

. (pairs of values) Definition SPoV =

127

• l=0

(Y2l − Y2l+1)2 2(Y2l + Y2l+1).

Dr Hans Georg Schaathun Statistics and Steganalysis Spring 2007 47 / 52

Pairs of Values Steganalysis Pairs of Values

Pairs of Values

The definition

S =

• ∈Ω

(Fo − E(Fo))2 E(Fo) , (general χ2 statistic) S =

127

• l=0

(Y2l − 1

2(Y2l + Y2l+1))2 1 2(Y2l + Y2l+1)

. (pairs of values) Definition SPoV =

127

• l=0

(Y2l − Y2l+1)2 2(Y2l + Y2l+1).

Dr Hans Georg Schaathun Statistics and Steganalysis Spring 2007 47 / 52

Pairs of Values Steganalysis Pairs of Values

Probability distribution of S

Dr Hans Georg Schaathun Statistics and Steganalysis Spring 2007 48 / 52

Pairs of Values Steganalysis Pairs of Values

Exercise

Preliminaries

1

Implement a Matlab function getYs taking an image X as input, and which outputs a m × 1 matrix containing the statistics Y1, Y2, . . . , Ym, where Yi is the number of pixels with colour value i.

2

Modify the function getYs to take a second parameter N and count only the N first pixels of X.

3

Implement a Matlab function taking as inputs an image and a number N, and outputs the χ2 statistic obtained by counting pairs

• f values in the first N pixels of the image. Remember to use the

function you made for the previous problem.

Dr Hans Georg Schaathun Statistics and Steganalysis Spring 2007 49 / 52

Pairs of Values Steganalysis Pairs of Values

Exercise

Preliminaries

1

Take a couple of the stego-images you created for Exercise set 2, and (for each image) plot the χ2 statistic as a function of N using the function you implemented above.

Dr Hans Georg Schaathun Statistics and Steganalysis Spring 2007 50 / 52

Generalised χ2 test

Generalised χ2 test

suggested first by Niels Provos Fridrich et al (2003) suggested details for an implementation No rigid statistics or hypothesis test . . . but works in practice Designed to work for embedding in random pixels

Dr Hans Georg Schaathun Statistics and Steganalysis Spring 2007 51 / 52

Generalised χ2 test

Generalised χ2 test

suggested first by Niels Provos Fridrich et al (2003) suggested details for an implementation No rigid statistics or hypothesis test . . . but works in practice Designed to work for embedding in random pixels

Dr Hans Georg Schaathun Statistics and Steganalysis Spring 2007 51 / 52

Generalised χ2 test

Generalised χ2 test

suggested first by Niels Provos Fridrich et al (2003) suggested details for an implementation No rigid statistics or hypothesis test . . . but works in practice Designed to work for embedding in random pixels

Dr Hans Georg Schaathun Statistics and Steganalysis Spring 2007 51 / 52

Generalised χ2 test

Generalised χ2 test

suggested first by Niels Provos Fridrich et al (2003) suggested details for an implementation No rigid statistics or hypothesis test . . . but works in practice Designed to work for embedding in random pixels

Dr Hans Georg Schaathun Statistics and Steganalysis Spring 2007 51 / 52

Generalised χ2 test

Generalised χ2 test

suggested first by Niels Provos Fridrich et al (2003) suggested details for an implementation No rigid statistics or hypothesis test . . . but works in practice Designed to work for embedding in random pixels

Dr Hans Georg Schaathun Statistics and Steganalysis Spring 2007 51 / 52

Generalised χ2 test

The idea

Generalised χ2 test

The basic idea in a window of s pixels

starting at first pixel counts pixel values in the window

The generalised test

Uses a sliding window s counting pairs in s pixels starting at position i averaging over i

Fridrich et al suggests to find a threshold experimentally

test database of ‘sufficient richness and diversity’

Dr Hans Georg Schaathun Statistics and Steganalysis Spring 2007 52 / 52

Generalised χ2 test

The idea

Generalised χ2 test

The basic idea in a window of s pixels

starting at first pixel counts pixel values in the window

The generalised test

Uses a sliding window s counting pairs in s pixels starting at position i averaging over i

Fridrich et al suggests to find a threshold experimentally

test database of ‘sufficient richness and diversity’

Dr Hans Georg Schaathun Statistics and Steganalysis Spring 2007 52 / 52

Generalised χ2 test

The idea

Generalised χ2 test

The basic idea in a window of s pixels

starting at first pixel counts pixel values in the window

The generalised test

Uses a sliding window s counting pairs in s pixels starting at position i averaging over i

Fridrich et al suggests to find a threshold experimentally

test database of ‘sufficient richness and diversity’

Dr Hans Georg Schaathun Statistics and Steganalysis Spring 2007 52 / 52

Generalised χ2 test

The idea

Generalised χ2 test

The basic idea in a window of s pixels

starting at first pixel counts pixel values in the window

The generalised test

Uses a sliding window s counting pairs in s pixels starting at position i averaging over i

Fridrich et al suggests to find a threshold experimentally

test database of ‘sufficient richness and diversity’

Dr Hans Georg Schaathun Statistics and Steganalysis Spring 2007 52 / 52

Generalised χ2 test

The idea

Generalised χ2 test

The basic idea in a window of s pixels

starting at first pixel counts pixel values in the window

The generalised test

Uses a sliding window s counting pairs in s pixels starting at position i averaging over i

Fridrich et al suggests to find a threshold experimentally

test database of ‘sufficient richness and diversity’

Dr Hans Georg Schaathun Statistics and Steganalysis Spring 2007 52 / 52

Generalised χ2 test

The idea

Generalised χ2 test

The basic idea in a window of s pixels

starting at first pixel counts pixel values in the window

The generalised test

Uses a sliding window s counting pairs in s pixels starting at position i averaging over i

Fridrich et al suggests to find a threshold experimentally

test database of ‘sufficient richness and diversity’

Dr Hans Georg Schaathun Statistics and Steganalysis Spring 2007 52 / 52

Generalised χ2 test

The idea

Generalised χ2 test

The basic idea in a window of s pixels

starting at first pixel counts pixel values in the window

The generalised test

Uses a sliding window s counting pairs in s pixels starting at position i averaging over i

Fridrich et al suggests to find a threshold experimentally

test database of ‘sufficient richness and diversity’

Dr Hans Georg Schaathun Statistics and Steganalysis Spring 2007 52 / 52

Generalised χ2 test

The idea

Generalised χ2 test

The basic idea in a window of s pixels

starting at first pixel counts pixel values in the window

The generalised test

Uses a sliding window s counting pairs in s pixels starting at position i averaging over i

Fridrich et al suggests to find a threshold experimentally

test database of ‘sufficient richness and diversity’

Dr Hans Georg Schaathun Statistics and Steganalysis Spring 2007 52 / 52

Generalised χ2 test

The idea

Generalised χ2 test

The basic idea in a window of s pixels

starting at first pixel counts pixel values in the window

The generalised test

Uses a sliding window s counting pairs in s pixels starting at position i averaging over i

Fridrich et al suggests to find a threshold experimentally

test database of ‘sufficient richness and diversity’

Dr Hans Georg Schaathun Statistics and Steganalysis Spring 2007 52 / 52