A Statistical Framework to Enlarge the Potential of Digital TV - - PowerPoint PPT Presentation

3rd International Symposium on Image and Signal Processing and Analysis ISPA 2003 September 18-20, 2003, Rome, Italy

A Statistical Framework to Enlarge the Potential of Digital TV Broadcasting

Maria Teresa Andrade, Artur Pimenta Alves INESC Porto/FEUP Porto, Portugal

3rd International Symposium on Image and Signal Processing and Analysis ISPA 2003 September 18-20, 2003, Rome, Italy

Aims of the work

• use statistical multiplexing for multi-channel TV

systems with the ability of anticipating bit rate behaviour of individual channels to obtain:

– Efficient use of bandwidth

• anticipate amount of unused bandwidth and re-allocate to other existing services
• include additional datacasting or video services using the same resources
• Reduce costs

– Higher picture quality – Increased programming/services choice

3rd International Symposium on Image and Signal Processing and Analysis ISPA 2003 September 18-20, 2003, Rome, Italy

The context

• DTV scenario

– moving to all digital world with seamless integration with the Internet – interactivity (local and remote) – possibility of datacasting additional services – demand for higher quality and more choice – demand for HDTV programmes

• Still remaining problem

– bandwidth is still an expensive resource – high-quality still requires high-bandwidth – CBR encoding not efficient in the trade-off quality/bandwidth – transmission channels usually of fixed bit rate – broadband connections introduced rather slowly – existing stat mux not very efficient

3rd International Symposium on Image and Signal Processing and Analysis ISPA 2003 September 18-20, 2003, Rome, Italy

Statistical multiplexing – present situation versus advanced solution

• Encoders co-located to the

multiplexer

• Restricted VBR mode with

feedback loop

• No guarantees of seamless

constant quality

• Released bandwidth

redistributed within the existing services in the multiplex

• Remote encoder operation
• True VBR operation
• Know in advance expected bit

rate requirements

• Define a-priori maximum and

minimum expected bandwidth gains

• Include on-the-fly new services

within the multiplex

3rd International Symposium on Image and Signal Processing and Analysis ISPA 2003 September 18-20, 2003, Rome, Italy

Possible approaches regarding bit rate

• constant bit rate with number of bits spent in each

GOP controlled to a mean value - varying picture quality or distortion

• variable bit rate controlling the distortion -

constant quality sequences

• constant bit rate using always the highest bit rate

needed to satisfy at all times a minimum level of distortion

3rd International Symposium on Image and Signal Processing and Analysis ISPA 2003 September 18-20, 2003, Rome, Italy

The probabilistic framework

• Statistical framework using bayesian inference to

describe occurrence of valleys in the bit rate

– likelihood of valleys

• Statistical framework using bayesian Weibull

survival model to analyse duration of valleys

– probability that the duration of valleys exceeds a certain time t

• Numerical sampling methods (MCMC) to obtain

posteriori distributions

– predict occurrence and duration of valleys

3rd International Symposium on Image and Signal Processing and Analysis ISPA 2003 September 18-20, 2003, Rome, Italy

Statistical framework

• Basis

– Video sources with almost-constant and high picture quality – VBR operation – Sources analysed on a GOP and scene-basis – Analysis of a comprehensive number of sources – Build data base with characteristics extracted from sources:

• Degree of criticality (difficulty to encode) and variability
• Type of content
• Parameters of a family of probabilistic models

– Assign and use posteriori pdfs for real sources in a statistical mux; anticipate released bandwidth to include extra services

3rd International Symposium on Image and Signal Processing and Analysis ISPA 2003 September 18-20, 2003, Rome, Italy

Core of the work

• build classification matrix (genre, profile, quality)
• build statistical framework with Bayesian inference

techniques

– obtain statistical characterisation of VBR video sources – infer family of probabilistic models capable of adequately describing the video sources, incorporating prior knowledge obtained from sets of training data – predict the amount of unused bandwidth in respect to a specified mean value using the posteriori probabilities (predict occurrence and duration of “valleys”)

3rd International Symposium on Image and Signal Processing and Analysis ISPA 2003 September 18-20, 2003, Rome, Italy

Classifying sources

• Cover whole spectrum of TV programming

– Use DVB category classification for genres

• Encode with two different quantisation step

sizes obtaining two different quality levels

– Fair and Good

• Obtain statistics (mean and variance) per

quality level

– mouvement + detail (activity) and criticality

3rd International Symposium on Image and Signal Processing and Analysis ISPA 2003 September 18-20, 2003, Rome, Italy

Characterising the traffic

• Granularity limited to GOPs (0,5 s) and to

scenes:

– Human eye can’t perceive variations in quality with durations less than 1 s – Mechanisms that (re) allocate bandwidth are not able to react faster – Inside a scene the type of content remains essentially the same, therefore little variation in bit rate – New scenes always start with a new GOP

3rd International Symposium on Image and Signal Processing and Analysis ISPA 2003 September 18-20, 2003, Rome, Italy

Influence of content

• bit rate follows a pdf (best fit Weibull, Gamma or Beta)

– at instante ti there is a given probability θ that the GOP bit rate be above or bellow a certain threshold.

• generation of bits per GOP varies with content

– probability of ocurrence of valleys will also differ

Encoder (generation

• f bits)

Video source Same type of content θ 1-θ above bellow Encoder (generation

• f bits)

Video sources Different type

• f content

θi 1-θi above bellow

Influence (type of content)

3rd International Symposium on Image and Signal Processing and Analysis ISPA 2003 September 18-20, 2003, Rome, Italy

Problem formulation

• Knowing that bitstream X has a number of

characteristics, which is the probability of expecting a given bit rate behaviour, or the occurrence of valleys, throughout its duration?

• valley -> a random variable with two possible states
• bservations -> specific characteristics -> prior knowledge as a pdf

model selection (maximizing a pdf) -> posteriori pdf -> state of random variable

3rd International Symposium on Image and Signal Processing and Analysis ISPA 2003 September 18-20, 2003, Rome, Italy

Bayes analyses

• Unknown parameter: GOPdev
• Fix a prior on the unknown parameter: p(GOPdev)
• Collect and observe the data: D = {X1, X2, ..., XN}
• Calculate the posteriori distribution p(valleys)

knowing the data X

3rd International Symposium on Image and Signal Processing and Analysis ISPA 2003 September 18-20, 2003, Rome, Italy

Statistical model

• an unknown parameter:

– GOPdev, the deviation of the GOP bit rate in respect to the expected mean bit rate

and a random variable:

– Occurrence of a valley in the bit rate, with 2 possible states:

• S = 1

⇒ a valley has occurred (GOP bit rate less than a certain threshold)

• S = 0

⇒ a valley did not occurred

3rd International Symposium on Image and Signal Processing and Analysis ISPA 2003 September 18-20, 2003, Rome, Italy

Bayes analyses

Sequence of scenes stream of GOPs θ 1-θ Above (red light) Bellow (green light)

• Uncertain variable or parameter, θ = GOPdev (variability of GOP bit rate)
• Prior distribution of parameter, p(θ | ξ)
• Set of observations D = {X1, X2, ..., XN} = stream of GOPs
• Having observed N GOPS, how to predict the value of occurrence N+1?

Will it be a valley in the GOP bit rate (green light) or not (red light)? p(xN+1 | θ, ξ) = ? knowing p(xN+1 | θ, ξ) and p(θ | ξ) Use Bayes rule, average over the possible values of θ and use the expansion probability rule.

3rd International Symposium on Image and Signal Processing and Analysis ISPA 2003 September 18-20, 2003, Rome, Italy

Bayes analysis

• Bayes rule to determine posteriori distribution of θ given the D set of

GOPs and background knowledge

• average over the possible values of θ, using the expansion rule:

p(xN+1= green | D, ξ) = ∫ p(xN+1= green | θ, ξ) . p(θ | D, ξ) dθ = ∫ θ . p(θ | D, ξ) dθ E p(θ | D, ξ)(θ)

      × = − × = × =

∫

= + → →

θ ξ θ ξ θ ξ θ θ ξ θ ξ ξ θ ξ θ ξ θ d p D p D p a b D p D p D p p D p

N b a b a

) | ( ) , | ( ) | ( ) 1 ( ) , | ( and , ) | ( ) , | ( ) | ( ) , | (

; lights red

• f

number ; lights green

• f

number

3rd International Symposium on Image and Signal Processing and Analysis ISPA 2003 September 18-20, 2003, Rome, Italy

Bayes analyses - remarks

• prior p(GOPdev) is only an approximation

– obtained posteriori p(valleys) will also be approximation

• Important to carefully analyse the data, select the

priors and test/calculate the posterioris for a great number of priors

• Choosing the prior:

– Kolmogorov method (minimum distance) – Maximum likelihood – Bayes

• Test/calculate the posterioris:

– Simulation methods through Markov Chain (MC Monte Carlo)

3rd International Symposium on Image and Signal Processing and Analysis ISPA 2003 September 18-20, 2003, Rome, Italy

Building the framewok

• Select diferent types of content covering major typical TV

programming – build complexity matrix with classes

• Encode in VBR / near-constant quality
• Analyse and extract characteristics on a GOP and scene-

basis

– ocurrence of scene changes, length of scenes – bit rate per image, GOP and scene – criticality (number of bits per pixel for a given quality) – variability (peak-to-mean ratio, coeficient of variation, auto- correlation) – “valleys” in bit rate within GOPs and scenes (intensity, variance, distance between valleys, length of valleys)

• Obtain the best priors
• Conduct calculations (using MCMC) to obtain posterioris

3rd International Symposium on Image and Signal Processing and Analysis ISPA 2003 September 18-20, 2003, Rome, Italy

Building blocks of the framework

• VBR encoder
• Video analyser (objective measures and

subjective classification)

• Classes and characteristics database
• Bayesian inference
• Probabilistic distributions database

3rd International Symposium on Image and Signal Processing and Analysis ISPA 2003 September 18-20, 2003, Rome, Italy

Functional blocks

Good / high quality video sources (MPEG-2/4 CBR

• r VBR)

VBR transcoder Extract statistics Database Statistics, classes analyser Subjective classifier Viewer Database VBR /constant quality sequences Posteriori calculations prior selection Database pdf families, parameters, classes Advanced statistical mux Aditional services (datacasting, scalable video coders, etc) Increased-value multiplex Statistical Bayesian framework

3rd International Symposium on Image and Signal Processing and Analysis ISPA 2003 September 18-20, 2003, Rome, Italy

Using the framework

• Analyse VBR video source (at start and

periodically every 5 s) -> classify / update classification and select model

• Calculate the amount of probable bit rate

that will be available

• Estimate occurrence and duration of next

valley

3rd International Symposium on Image and Signal Processing and Analysis ISPA 2003 September 18-20, 2003, Rome, Italy

Initial measures

• ccurrences per % of mean GOP rate - film "Gladiator"

20 40 60 80 100 120 140 160 180 200 1,15 1,1 1,05 1 0,95 0,9 0,85 0,8 0,75 0,5 % of mean number of occurrences

• 80% of GOP ocurrences within +/- 5% of mean
• 16% of GOP occurrences are bellow –5% mean (green light)
• 4 % of GOP occurrences are above mean (red light)

3rd International Symposium on Image and Signal Processing and Analysis ISPA 2003 September 18-20, 2003, Rome, Italy

Initial measures

• currences per % of mean value - film on DVD

10 20 30 40 50 60 70 1,35 1,30 1,25 1,20 1,15 1,10 1,05 1,00 0,95 0,90 0,85 0,80 0,75 0,70 0,65 % of mean GOP rate number of GOPs

• 18% of GOP ocurrences within +/- 5% of mean
• 50% of GOP occurrences are bellow –5% mean (green light)
• 32 % of GOP occurrences are above mean (red light)

3rd International Symposium on Image and Signal Processing and Analysis ISPA 2003 September 18-20, 2003, Rome, Italy

Initial measures

• currences per % of mean value - film on DVD

20 40 60 80 100 120 140 160 1,30 1,25 1,20 1,15 1,10 1,05 1,00 0,95 0,90 0,80 0,75 % of mean GOP rate number of GOPs

• 70% of GOP ocurrences within +/- 5% of mean
• 20% of GOP occurrences are bellow –5% mean (green light)
• 10 % of GOP occurrences are above mean (red light)

3rd International Symposium on Image and Signal Processing and Analysis ISPA 2003 September 18-20, 2003, Rome, Italy

Initial measures

• currences per % of mean value - film on DVD

10 20 30 40 50 60 1,40 1,35 1,30 1,25 1,20 1,15 1,10 1,05 1,00 0,95 0,90 0,85 0,80 % of mean GOP rate number of GOPs

• 24% of GOP ocurrences within +/- 5% of mean
• 52% of GOP occurrences are bellow –5% mean (green light)
• 24% of GOP occurrences are above mean (red light)

3rd International Symposium on Image and Signal Processing and Analysis ISPA 2003 September 18-20, 2003, Rome, Italy

Initial measures

• currences per % of mean value - film on DVD

20 40 60 80 100 120 1,25 1,20 1,15 1,10 1,05 1,00 0,95 0,90 0,85 0,80 % of mean GOP rate number of GOPs

• 60% of GOP ocurrences within +/- 5% of mean
• 28% of GOP occurrences are bellow –5% mean (green light)
• 12% of GOP occurrences are above mean (red light)

3rd International Symposium on Image and Signal Processing and Analysis ISPA 2003 September 18-20, 2003, Rome, Italy

Statistical measures

3rd International Symposium on Image and Signal Processing and Analysis ISPA 2003 September 18-20, 2003, Rome, Italy

Statistical measures

3rd International Symposium on Image and Signal Processing and Analysis ISPA 2003 September 18-20, 2003, Rome, Italy

Statistical measures

3rd International Symposium on Image and Signal Processing and Analysis ISPA 2003 September 18-20, 2003, Rome, Italy

Statistical measures

3rd International Symposium on Image and Signal Processing and Analysis ISPA 2003 September 18-20, 2003, Rome, Italy

Statistical measures

3rd International Symposium on Image and Signal Processing and Analysis ISPA 2003 September 18-20, 2003, Rome, Italy

Statistical measures

3rd International Symposium on Image and Signal Processing and Analysis ISPA 2003 September 18-20, 2003, Rome, Italy

3rd International Symposium on Image and Signal Processing and Analysis ISPA 2003 September 18-20, 2003, Rome, Italy

Statistical measures

3rd International Symposium on Image and Signal Processing and Analysis ISPA 2003 September 18-20, 2003, Rome, Italy

Thank you so much for your attention!