Polynomial tuning of multiparametric combinatorial samplers Maciej - PowerPoint PPT Presentation

Polynomial tuning of multiparametric combinatorial samplers Maciej Bendkowski 3 Olivier Bodini 1 Sergey Dovgal 1 , 2 , 4 1 Université Paris-13, 2 Université Paris-Diderot 3 Jagiellonian University in Kraków 4 Moscow Institute of Physics and Technology Computational Logic and Applications, Paris, May 25th 2018 1/29

Outline Combinatorial sampling techniques Multivariate tuning Applications 2/29

Warm-up — combinatorial samplers for binary trees Z = Z + B B B B ( z ) = z + zB 2 ( z ) Recursive method Boltzmann samplers exact-size sampling approximate-size sampling (Nijenhuis and Wilf, 1978) (Duchon et al., 2004) ◮ if n = 1 return Z ,  z ◮ otherwise, sample k from Z P Z = B ( z ) ,   probability distribution    Z Γ B := P ( k ) = b k b n − k − 1 P Γ B = zB 2 ( z )  b n   B ( z ) .  Γ B Γ B  and recurse accordingly. 3/29

Univariate Boltzmann samplers — outcome distribution ◮ P (Γ B outputs tree τ ) = z | τ | B ( z ) (uniform conditioned on size); ◮ Expected output size is z B ′ ( z ) B ( z ) (increasing function of z ). How long do we wait for an object of size in [ n ( 1 − ε ) , n ( 1 + ε )] ? 0.25 0.20 0.15 probability 0.10 0.05 0.00 0 2 4 6 8 10 12 14 16 18 20 size of generated object O ε ( n ) for binary trees. 4/29

Combinatorial samplers — summary For general, admissible combinatorial specifications, sampling objects of size n takes, respectively: ◮ O ( n 2 ) using the recursive method; ◮ O ( n 2 ) using exact-size Boltzmann sampling; ◮ O ( n ) using approximate-size Boltzmann sampling. 5/29

Multiparametric sampling — example Problem Generate uniformly at random plane trees with ◮ n nodes in total, ◮ j leaves in total, ◮ and k nodes with 3 children. z z = z + + T . . . T T T T T � �� � 0 , 1 , 2 , 4 , 5 ,... = N \{ 3 } 6/29

Multiparametric sampling — state of the art Problem Given an admissible, algebraic 1 combinatorial specification S and an expectation vector X controlling k parameters of associated objects, generate a uniformly random object with expectation profile X . Status (exact-parameter case): ◮ Dynamic programming using the recursive method O ( n k + 1 ) (essentially Nijenhuis and Wilf, 1978); ◮ Boltzmann sampling using multidimensional oracles O ( n 1 + k / 2 ) (Bodini and Ponty, 2010); Our contributions: ◮ (target parameter expectations) n · Poly ( k ) ; ◮ (exact-size, approximate-size parameters for strongly-connected rational languages) n + O ( 1 ) . 1 extensions are also available 7/29

Outline Combinatorial sampling techniques Multivariate tuning Applications 8/29

Multivariate Boltzmann model Probability model Let C ( z ) = � p ≥ 0 c p z p be a multivariate generating function associated with the class C . Then, the probability P z ( ω ) of generating object ω of (composite-)size p takes the form z p 1 1 · · · z p k z p k P z ( ω ) = C ( z ) = C ( z 1 , . . . , z k ) . In consequence, the expectation of E z ( N i ) of parameter z i takes the form ∂ C ( z ) ∂ z i E z ( N i ) = z i . C ( z ) 9/29

Multivariate Boltzmann model — example Q: How to quickly sample Motzkin trees, uniformly at random conditioned on size, with roughly 30 % of unary nodes? 30 % z z = z + + M M M M M ( z , u ) = z + uzM ( z , u ) + zM ( z , u ) 2 ∂ ∂ uM ( z , u ) E z , u ( N u ) = u . M ( z , u ) 10/29

Tuning as an optimisation problem � ∂ � ⊤ f ( z ) , . . . , ∂ Let ∇ z f ( z ) = f ( z ) denote the gradient ∂ z 1 ∂ z k vector with respect to z = ( z 1 , . . . , z k ) . Then, solving E z ( N ) = p is equivalent to � � log C ( e z ) − z ⊤ p ∇ z = 0 . Theorem (B., Bodini, Dovgal) Fix the expectations E z ( N ) = p . Then, the associated tuning vector z is equal to e ξ where ξ comes from the following minimisation problem: log C ( e ξ ) − p ⊤ ξ → min . ξ 11/29

Log-exp transform for algebraic systems Theorem (B., Bodini, Dovgal) Let C = Φ ( C , Z ) be an algebraic system. Fix the expectations E z ( N ) = p of objects sampled from C 1 . Then, the corresponding tuning problem is equivalent to the following log-exp transformed convex minimisation problem: � c 1 − p ⊤ ξ → min ξ , c , log Φ ( e c , e ξ ) − c ≤ 0 . 12/29

Log-exp transform — example   A ≥ zxB 2 + zyC α ≥ e ζ e ξ e 2 β + e ζ e η e γ     β ≥ e ζ e ξ e β + e ζ e α B ≥ zxB + zA ⇒   γ ≥ e ζ + e ζ e α   C ≥ z + zA Associated convex minimisation problem:  α − E z ζ − E x ξ − E y η → min ζ,ξ,η,α    � e ζ e ξ e 2 β + e ζ e η e γ �   α ≥ log � e ζ e ξ e β + e ζ e α � β ≥ log    � e ζ + e ζ e α �   γ ≥ log 13/29

Convex optimisation complexity Theorem (Nesterov and Nemirovskii, 1994) Convex optimisation programs can be effectively optimised using polynomial-time interior-point methods. As a consequence, for multiparametric combinatorial systems with description length L , the tuning problem can be solved with precision ε in time � � L 3 . 5 log 1 O . ε 14/29

Singular samplers for algebraic systems Theorem (B., Bodini, Dovgal) Let C = Φ ( C , Z , U ) be an algebraic system with Z marking the object size. Fix the expectations p of atoms U . Set z = e ξ and u = e η . Then, ( z , u ) from the following convex program tune the corresponding singular Boltzmann sampler: � ξ + p ⊤ η → max ξ, η , c log Φ ( e c , e ξ , e η ) − c ≤ 0 . � � z → max , ζ → max , ⇒ b ≥ log ( e ζ + e ζ e 2 b ) T ≥ z + zT 2 10 10.0 7.5 8 5.0 2.5 6 0.0 4 −2.5 −5.0 2 −7.5 0 −10.0 0.0 0.1 0.2 0.3 0.4 0.5 −10 −8 −6 −4 −2 15/29

Outline Combinatorial sampling techniques Multivariate tuning Applications 16/29

Open source implementation — Boltzmann Brain Example – boolean formulae over {¬ , ∨ , ∧} : 30 % ¬ ∨ ∧ z = + + + seq ≥ 1 ( ) F F F F F F z F ( z ) = uzF ( z ) + 2 zF ( z ) 2 + 1 − z Fun = Neg Fun [0.3] 1 | And Fun Fun 2 | Or Fun Fun 3 | Variable Index (0) 4 5 Index = Zero 6 | Succ Index 7 Code available at https://github.com/maciej-bendkowski/boltzmann-brain https://github.com/maciej-bendkowski/multiparametric-combinatorial-samplers 17/29

Boltzmann Brain — example output sampler 18/29

Toy example — random tilings Problem Tile a stripe 7 × n with 126 different types of tiles such that the area covered by each tile is (approximately) uniform. Note: The associated automaton has ◮ over 2 , 000 states, and ◮ over 28 , 000 transitions. 19/29

Random tilings — example 20/29

Simply-generated trees with node degree constraints Simple varieties of plane trees with fixed sets of admissible node degrees, satisfying the general equation y ( z ) = z φ ( y ( z )) φ : C → C . for some polynomial For instance, consider plane trees φ ( y ( z )) = a 0 + a 1 y ( z ) + a 2 y ( z ) 2 + · · · + a 9 y ( z ) 9 . We tune the corresponding algebraic specification so to achieve a target frequency of 1 % for all nodes of degrees d ≥ 2. Frequencies of nodes with degrees d ≤ 1 are lef undistorted. Node degree 0 1 2 3 4 5 6 7 8 9 1 . 00 % 1 . 00 % 1 . 00 % 1 . 00 % 1 . 00 % 1 . 00 % 1 . 00 % 1 . 00 % Tuned frequency - - - - - - Observed frequency 35 . 925 % 56 . 168 % 0 . 928 % 0 . 898 % 1 . 098 % 0 . 818 % 1 . 247 % 0 . 938 % 1 . 058 % 0 . 918 % Default frequency 50 . 004 % 24 . 952 % 12 . 356 % 6 . 322 % 2 . 882 % 1 . 984 % 0 . 877 % 0 . 378 % 0 . 169 % 0 . 069 % Table: Empirical frequencies of the node degree distribution. 21/29

Simply-generated trees with node degree constraints – example 22/29

λ -terms with given de Bruijn index distribution z 9 L = zL + zL 2 + u 1 z + u 2 z 2 + . . . + u 8 z 8 + 1 − z λ @ = + seq ≥ 1 ( z + ) L L L L 23/29

λ -terms with given de Bruijn index distribution – example 24/29

Closed λ -terms for property-based sofware testing Some interesting parameters: ◮ number of variables, λ ◮ de Bruijn index distribution, λ ◮ Number of head lambdas, @ ◮ Number of redexes, i.e. · · λ λ ◮ . . . @ Motivation Generate closed lambda terms @ @ (programs) with skewed distribution to find bugs in functional language 2 0 1 0 compilers such as Haskell’s GHC. 25/29

Integer partitions and the Bose–Einstein condensate Example: 16 = 1 + 1 + 3 + 4 + 7 P = mset ( mset ≥ 1 ( Z )) Generalisation from statistical physics (Bose–Einstein) In the model of d -dimensional harmonic trap the number of states � d + λ − 1 � for particle with energy λ is . Each state is represented as a λ multiset of λ elements having d different colours. E = mset ( mset ≥ 1 ( Z 1 + Z 2 + · · · + Z d )) 26/29

d-dimensional quantum harmonic oscillator Weighted partition Random particle assembly Sum of numbers Total energy Number of colours Dimension ( d ) Row of Young table Particle Number of rows Number of particles Number of squares in the row Energy of a particle ( λ ) Problem Generate random assemblies with given (expected) numbers ( n 1 , n 2 , . . . , n d ) of colours (dimension). 27/29

Weighted integer partitions (5,8, and 9 colours, respectively) 28/29

Thank you! Thank you for your atention! 29/29