Randomized Algorithms Course Info yitong.yin@gmail.com - PowerPoint PPT Presentation

Randomized Algorithms 南京大学 尹一通

Course Info • 尹一通 yitong.yin@gmail.com yinyt@nju.edu.cn • office hour: 804, Thursday 2-4pm • course homepage: http://tcs.nju.edu.cn/wiki

Textbooks Rajeev Motwani and Prabhakar Raghavan. Randomized Algorithms. Cambridge University Press, 1995. Michael Mitzenmacher and Eli Upfal. Probability and Computing: Randomized Algorithms and Probabilistic Analysis. Cambridge University Press, 2005.

References CLRS Feller An Introduction to Probability Theory and Its Applications Aldous and Fill Reversible Markov Chains and Random Walks on Graphs Alon and Spencer The Probabilistic Method

Randomized Algorithms “algorithms which use randomness in computation” Turing random Machine coin Why? • Simpler. • Faster. • Can do impossibles. • Can give us clever deterministic algorithms. • Random input. • Deterministic problem with random nature. • ... ...

Checking Matrix Multiplication three n × n matrices A , B , C : ? = × A B C best matrix multiplication algorithm: O ( n 2.373 )

Checking Matrix Multiplication three n × n matrices A , B , C : ? = × A B C O ( n ω ) best matrix multiplication algorithm: O ( n 2.373 )

Checking Matrix Multiplication three n × n matrices A , B , C : ? = × A B C best matrix multiplication algorithm: O ( n 2.373 ) Freivald’s Algorithm pick a uniform random r ∈ {0,1} n ; check whether A ( Br ) = Cr ; time: O( n 2 ) if AB = C , always correct

Freivald’s Algorithm pick a uniform random r ∈ {0,1} n ; check whether A ( Br ) = Cr ; if AB = C , always correct let if AB ≠ C , D = AB � C � = 0 n × n say D ij � = 0 2 n − 1 = 1 Pr[ ABr = Cr ] = Pr[ Dr = 0 ] ≤ 2 n 2 n � ( Dr ) i = D ik r k =0 D r k =1 n r j = − 1 � i D ik r k D ij k � = j

Freivald’s Algorithm pick a uniform random r ∈ {0,1} n ; check whether A ( Br ) = Cr ; if AB = C , always correct Theorem (Freivald, 1979) If AB ⇥ = C , for a uniformly random r � { 0 , 1 } n , Pr[ ABr = Cr ] ≤ 1 2 . repeat independently for 100 times time: O( n 2 ) if AB ≠ C , error probability ≤ 2 − 100

Monte Carlo vs Las Vegas Two types of randomized algorithms: Las Vegas Monte Carlo running time is fixed always correct correctness is random running time is random

Polynomial Identity Testing (PIT) Input : two polynomials f, g ∈ F [ x ] of degree d Output : f ≡ g ? d X a i x i of degree d : for f ( x ) = a i ∈ F f ∈ F [ x ] i =0 Input : a polynomial of degree d f ∈ F [ x ] Output : f ≡ 0? f is given as black-box

Input : a polynomial of degree d f ∈ F [ x ] Output : f ≡ 0? simple deterministic algorithm: check whether f ( x )=0 for all x ∈ { 1 , 2 , . . . , d + 1 } Fundamental Theorem of Algebra: A degree d polynomial has at most d roots. Randomized Algorithm pick a uniform random r ; S ⊆ F ∈ S check whether f ( r ) = 0 ;

Randomized Algorithm pick a uniform random r ; S ⊆ F ∈ S check whether f ( r ) = 0 ; | S | = 2 d if f �� 0 d = 1 Pr[ f ( r ) = 0] ≤ | S | 2 Fundamental Theorem of Algebra: A degree d polynomial has at most d roots.

Checking Identity 北京 database 1 Are they identical? 南京 database 2

Communication Complexity (Yao 1979) f ( a, b ) # of bits communicated a b Han Meimei Li Lei EQ : { 0 , 1 } n × { 0 , 1 } n → { 0 , 1 } Theorem (Yao, 1979) � 1 a = b EQ( a, b ) = There is no deterministic communication protocol 0 a ̸ = b solving EQ with less than n bits in the worst-case.

Communication Complexity n − 1 n − 1 f ( r )= g ( r ) ? � X b i x i a i x i f = g = i =0 i =0 r , g ( r ) ∈ { 0 , 1 } n a b ∈ { 0 , 1 } n Han Meimei Li Lei pick uniform by PIT: random r ∈ [2 n ] one-sided error ≤ 1 2 # of bit communicated: too large!

Communication Complexity n − 1 n − 1 f ( r )= g ( r ) ? � X b i x i a i x i f = g = i =0 i =0 r , g ( r ) ∈ { 0 , 1 } n a b ∈ { 0 , 1 } n O(log n ) bits Han Meimei Li Lei pick uniform random r ∈ [ p ] k = � log 2 (2 n ) � let choose a prime p ∈ [2 k , 2 k +1 ] f, g ∈ Z p [ x ]

Polynomial Identity Testing (PIT) Input : of degree d f, g ∈ F [ x 1 , x 2 , . . . , x n ] Output : f ≡ g ? F [ x 1 , x 2 , . . . , x n ] : ring of n -variate polynomials over field F f ∈ F [ x 1 , x 2 , . . . , x n ] : X a i 1 ,i 2 ,...,i n x i 1 1 x i 2 2 · · · x i n f ( x 1 , x 2 , . . . , x n ) = n i 1 ,i 2 ,...,i n ≥ 0 degree of f : maximum i 1 + i 2 + · · · + i n with a i 1 ,i 2 ,...,i n 6 = 0

Input : of degree d f, g ∈ F [ x 1 , x 2 , . . . , x n ] Output : f ≡ g ? equivalently: Input : of degree d f ∈ F [ x 1 , x 2 , . . . , x n ] Output : f ≡ 0? X a i 1 ,i 2 ,...,i n x i 1 1 x i 2 2 · · · x i n f ( x 1 , x 2 , . . . , x n ) = n i 1 ,i 2 ,...,in ≥ 0 i 1+ i 2+ ··· + in ≤ d

Input : of degree d f, g ∈ F [ x 1 , x 2 , . . . , x n ] Output : f ≡ g ? equivalently: Input : of degree d f ∈ F [ x 1 , x 2 , . . . , x n ] Output : f ≡ 0? f is given as block-box: given any ~ x = ( x 1 , x 2 , . . . , x n ) returns f ( ~ x ) e.g. or as product from: Vandermonde determinant x n − 1 x 2   1 x 1 . . . 1 1 x n − 1 x 2 1 x 2 . . . Y f ( ~ x ) = det( M ) = ( x i − x j )   2 2 M =   . . . . ... . . . .   . . . .   j<i x 2 x n − 1 1 x n . . . n n

PIT: Polynomial Identity Testing Input : of degree d f ∈ F [ x 1 , x 2 , . . . , x n ] Output : f ≡ 0? f is given as block-box or product from if ∃ a poly-time deterministic algorithm for PIT: either: NEXP ≠ P/poly or: #P ≠ FP

Input : of degree d f ∈ F [ x 1 , x 2 , . . . , x n ] Output : f ≡ 0? fix an arbitrary S ⊆ F pick random r 1 , r 2 , ... , r n ∈ S uniformly and independently at random; check whether f ( r 1 , r 2 , ... , r n ) = 0 ; f ( r 1 , r 2 , . . . , r n ) = 0 f ≡ 0

Input : a polynomial of degree d f ∈ F [ x ] Output : f ≡ 0? fix an arbitrary S ⊆ F pick a uniform random r ∈ S ; check whether f ( r ) = 0 ; f ≡ 0 f ( r ) = 0 Fundamental Theorem of Algebra: A degree d polynomial has at most d roots. Pr[ f ( r ) = 0] ≤ d f 6⌘ 0 | S |

Input : of degree d f ∈ F [ x 1 , x 2 , . . . , x n ] Output : f ≡ 0? fix an arbitrary S ⊆ F pick random r 1 , r 2 , ... , r n ∈ S uniformly and independently at random; check whether f ( r 1 , r 2 , ... , r n ) = 0 ; f ( r 1 , r 2 , . . . , r n ) = 0 f ≡ 0 Schwartz-Zippel Theorem Pr[ f ( r 1 , r 2 , . . . , r n ) = 0] ≤ d f 6⌘ 0 | S |

Schwartz-Zippel Theorem Pr[ f ( r 1 , r 2 , . . . , r n ) = 0] ≤ d f 6⌘ 0 | S | X a i 1 ,i 2 ,...,i n x i 1 1 x i 2 2 · · · x i n f ( x 1 , x 2 , . . . , x n ) = n i 1 ,i 2 ,...,in ≥ 0 i 1+ i 2+ ··· + in ≤ d f can be treated as a single-variate polynomial of x n : d X x i f ( x 1 , x 2 , . . . , x n ) = n f i ( x 1 , x 2 , . . . , x n − 1 ) i =0 = g x 1 ,x 2 ,...,x n − 1 ( x n ) Pr[ f ( r 1 , r 2 , . . . , r n ) = 0] = Pr[ g r 1 ,r 2 ,...,r n − 1 ( r n ) = 0] done? g r 1 ,r 2 ,...,r n − 1 6⌘ 0?

Schwartz-Zippel Theorem Pr[ f ( r 1 , r 2 , . . . , r n ) = 0] ≤ d f 6⌘ 0 | S | induction on n : basis: single-variate case, proved by n =1 the fundamental Theorem of algebra I.H.: Schwartz-Zippel Thm is true for all smaller n

Schwartz-Zippel Theorem Pr[ f ( r 1 , r 2 , . . . , r n ) = 0] ≤ d f 6⌘ 0 | S | induction step: f k 6⌘ 0 n k : highest power of x n in f degree of f k ≤ d − k k X x i f ( x 1 , x 2 , . . . , x n ) = n f i ( x 1 , x 2 , . . . , x n − 1 ) i =0 n f k ( x 1 , x 2 , . . . , x n − 1 ) + ¯ = x k f ( x 1 , x 2 , . . . , x n ) k − 1 ¯ X x i where f ( x 1 , x 2 , . . . , x n ) = n f i ( x 1 , x 2 , . . . , x n − 1 ) i =0 highest power of x n in ¯ f < k

Schwartz-Zippel Theorem Pr[ f ( r 1 , r 2 , . . . , r n ) = 0] ≤ d f 6⌘ 0 | S | n f k ( x 1 , x 2 , . . . , x n − 1 ) + ¯ = x k f ( x 1 , x 2 , . . . , x n ) f ( x 1 , x 2 , . . . , x n ) f k 6⌘ 0 n highest power of x n in ¯ f < k degree of f k ≤ d − k law of total probability: ≤ d − k I.H. Pr[ f ( r 1 , r 2 , . . . , r n ) = 0] | S | = Pr[ f ( ~ r ) = 0 | f k ( r 1 , . . . , r n − 1 ) = 0] · Pr[ f k ( r 1 , . . . , r n − 1 ) = 0] + Pr[ f ( ~ r ) = 0 | f k ( r 1 , . . . , r n − 1 ) 6 = 0] · Pr[ f k ( r 1 , . . . , r n − 1 ) 6 = 0] ≤ k = Pr[ g r 1 ,...,r n − 1 ( r n ) = 0 | f k ( r 1 , . . . , r n − 1 ) 6 = 0] | S | where g x 1 ,...,x n − 1 ( x n ) = f ( x 1 , . . . , x n )

Schwartz-Zippel Theorem Pr[ f ( r 1 , r 2 , . . . , r n ) = 0] ≤ d f 6⌘ 0 | S | Pr[ f ( r 1 , r 2 , . . . , r n ) = 0] ≤ d − k + k = d | S | | S | | S |

Input : of degree d f ∈ F [ x 1 , x 2 , . . . , x n ] Output : f ≡ 0? fix an arbitrary S ⊆ F pick random r 1 , r 2 , ... , r n ∈ S uniformly and independently at random; check whether f ( r 1 , r 2 , ... , r n ) = 0 ; f ( r 1 , r 2 , . . . , r n ) = 0 f ≡ 0 Schwartz-Zippel Theorem Pr[ f ( r 1 , r 2 , . . . , r n ) = 0] ≤ d f 6⌘ 0 | S |