Improving Implementation of SAT Competitions 2017-2019 Winners Stepan Kochemazov veinamond@gmail.com Matrosov Institute for System Dynamics and Control Theory SB RAS Irkutsk, Russia SAT2020 1/18
Motivation The solvers that enter recent SAT competitions are biased towards the winner of the previous competition. 2/18
Motivation The solvers that enter recent SAT competitions are biased towards the winner of the previous competition. • Each of the winners of SAT competitions 2017-2019 is based on the winner of the previous competition. • SC2016 main track winner: MapleCOMSPS . • In 2017, 2 solvers out of 29 participants were based on MapleCOMSPS . • SC2017 main track winner: Maple_LCM_Dist . • In 2018, about 13 out of 41 participants were based on Maple_LCM_Dist . • SC2018 main track winner: MapleLCMDistChronoBT . • In 2019, about 24 out of 55 participants were based on Maple_LCM_Dist , 15 of them – on MapleLCMDistChronoBT . • SR2019 winner: MapleLCMDistChronoBT-DL-v3 . 2/18
Motivation The solvers that enter recent SAT competitions are biased towards the winner of the previous competition. • Each of the winners of SAT competitions 2017-2019 is based on the winner of the previous competition. • SC2016 main track winner: MapleCOMSPS . • In 2017, 2 solvers out of 29 participants were based on MapleCOMSPS . • SC2017 main track winner: Maple_LCM_Dist . • In 2018, about 13 out of 41 participants were based on Maple_LCM_Dist . • SC2018 main track winner: MapleLCMDistChronoBT . • In 2019, about 24 out of 55 participants were based on Maple_LCM_Dist , 15 of them – on MapleLCMDistChronoBT . • SR2019 winner: MapleLCMDistChronoBT-DL-v3 . • On the one hand, when implementing a new heuristic it is easier to work with the existing implementation as is. • On the other hand, improving common implementation aspects of winners of several years may result in improving the effectiveness of a major part of potential SAT competition participants. 2/18
Goal The goal of the study was to make deterministic implementations of SAT Competitions 2017-2019 winners with performance comparable or better to the original solvers. 3/18
Goal The goal of the study was to make deterministic implementations of SAT Competitions 2017-2019 winners with performance comparable or better to the original solvers. • MapleCOMSPS , Maple_LCM_Dist and MapleLCMDistChronoBT switch between branching heuristics after 2500 seconds. • On different systems / launches this switch happens at different stages in terms of number of conflicts / number of propagations. 3/18
Goal The goal of the study was to make deterministic implementations of SAT Competitions 2017-2019 winners with performance comparable or better to the original solvers. • MapleCOMSPS , Maple_LCM_Dist and MapleLCMDistChronoBT switch between branching heuristics after 2500 seconds. • On different systems / launches this switch happens at different stages in terms of number of conflicts / number of propagations. • The solving of all problems with runtime > 2500 seconds is very hard to explicitly reproduce. • Hinders the development of SAT solver modifications since the same solver in different launches over the same benchmark set may solve +/- 3 instances. 3/18
Goal The goal of the study was to make deterministic implementations of SAT Competitions 2017-2019 winners with performance comparable or better to the original solvers. • MapleCOMSPS , Maple_LCM_Dist and MapleLCMDistChronoBT switch between branching heuristics after 2500 seconds. • On different systems / launches this switch happens at different stages in terms of number of conflicts / number of propagations. • The solving of all problems with runtime > 2500 seconds is very hard to explicitly reproduce. • Hinders the development of SAT solver modifications since the same solver in different launches over the same benchmark set may solve +/- 3 instances. • The authors of MapleCOMSPS realized their oversight and submitted to later competitions only deterministic variants of their solver. 3/18
Experimental setup All experiments were performed using a single node of the “Academician V.M. Matrosov” cluster. 1 • Two 18-core Intel Xeon E5-2695 (Broadwell) CPUs with 128 GB DDR4 RAM. • The solvers were launched in 36 simultaneous threads with time limit of 5000 seconds. • All logs of experiments are available online. 2 To test the proposed implementation changes the modifications of MapleLCMDistChronoBT were launched on the instances from SAT Race 2019. 1 http://hpc.icc.ru/en/ 2 https://github.com/veinamond/SAT2020 4/18
Experimental setup All experiments were performed using a single node of the “Academician V.M. Matrosov” cluster. 1 • Two 18-core Intel Xeon E5-2695 (Broadwell) CPUs with 128 GB DDR4 RAM. • The solvers were launched in 36 simultaneous threads with time limit of 5000 seconds. • All logs of experiments are available online. 2 To test the proposed implementation changes the modifications of MapleLCMDistChronoBT were launched on the instances from SAT Race 2019. Main contributions • Implementing the deterministic strategy for switching between branching heuristics. 1 http://hpc.icc.ru/en/ 2 https://github.com/veinamond/SAT2020 4/18
Experimental setup All experiments were performed using a single node of the “Academician V.M. Matrosov” cluster. 1 • Two 18-core Intel Xeon E5-2695 (Broadwell) CPUs with 128 GB DDR4 RAM. • The solvers were launched in 36 simultaneous threads with time limit of 5000 seconds. • All logs of experiments are available online. 2 To test the proposed implementation changes the modifications of MapleLCMDistChronoBT were launched on the instances from SAT Race 2019. Main contributions • Implementing the deterministic strategy for switching between branching heuristics. • Improving the handling of Tier2 and Core learnts. 1 http://hpc.icc.ru/en/ 2 https://github.com/veinamond/SAT2020 4/18
On switching between branching heuristics • In COMiniSatPS it was proposed to use distinct two sets of VSIDS scores: one with Minisat -like Luby restarts and another with glucose restarts. COMiniSatPS frequently switches between these two phases. 5/18
On switching between branching heuristics • In COMiniSatPS it was proposed to use distinct two sets of VSIDS scores: one with Minisat -like Luby restarts and another with glucose restarts. COMiniSatPS frequently switches between these two phases. • In MapleCOMSPS the first set of VSIDS scores was replaced by LRB scores. 5/18
On switching between branching heuristics • In COMiniSatPS it was proposed to use distinct two sets of VSIDS scores: one with Minisat -like Luby restarts and another with glucose restarts. COMiniSatPS frequently switches between these two phases. • In MapleCOMSPS the first set of VSIDS scores was replaced by LRB scores. • In MapleCOMSPS the switch happens once at 2500 seconds: the solver starts with LRB+Luby restarts and then switches to VSIDS+glucose restarts. • The MapleCOMSPS implementation of the switching between branching heuristics was inherited by Maple_LCM_Dist and MapleLCMDistChronoBT , the winners of SAT Competitions 2017 and 2018. • The MapleLCMDistChronoBT-DL-v3 solver that won at SAT Race 2019 uses a deterministic strategy for switching between LRB+Luby restarts and VSIDS+glucose restarts. 5/18
Cactus plot for MapleLCMDistChronoBT over benchhmarks from SC2017-SR2019 5000 MapleLCMDistChronoBT 4000 3000 Time 2000 1000 0 0 100 200 300 400 500 600 Instances 6/18
Deterministic strategies for switching between branching heuristics Abstracted part of the solve_ procedure VSIDS = false; int phase_allotment = initial_value ; int c_r = 0; for (;;){ int weighted = VSIDS ? phase_allotment * VSIDS_mult : phase_allotment ; while (status == l_Undef && weighted > 0) if (VSIDS) status = search(weighted ); else{ int nof_conflicts = luby(r_inc , c_r) * r_first; c_r ++; weighted -= nof_conflicts ; status = search( nof_conflicts ); } if (status != l_Undef) break; VSIDS = !VSIDS; if (! VSIDS) phase_allotment *= mult1; else phase_allotment *= mult2; } Initial value VSIDS_mult mult1 / mult2 Source fcm 100 conflicts 2 1.1/1 COMiniSatPS 100 conflicts 1 1.1/1 f1 MapleCOMSPS_LRB f2 10K conflicts 1 2/1 MapleCOMSPS_LRB_VSIDS_2 30M propagations 1 1.1/1.1 v3 MapleLCMDistChronoBT-DL-v3 7/18
On switching between branching heuristics:Evaluation 5000 MapleLCMDistChronoBT MapleLCMDistChronoBT-f2 MapleLCMDistChronoBT-v3 MapleLCMDistChronoBT-f1 MapleLCMDistChronoBT-fcm 4000 3000 Solver SCR:S:U PAR-2 236:138:98 4799 Chrono Time f1 231:135:96 4882 f2 235: 140 :95 4716 v3 232:137:95 4787 2000 fcm 228:133:95 4985 1000 0 100 120 140 160 180 200 220 240 Instances 8/18
On the importance of separate branching strategies:Evaluation 5000 MapleLCMDistChronoBT MapleLCMDistChronoBT-f2 MapleLCMDistChronoBT-LRB-f2 MapleLCMDistChronoBT-VSIDS-f2 4000 3000 Solver SCR:S:U PAR-2 Chrono 236:138:98 4799 Time f2 235:140:95 4716 223:134:89 4996 LRB-f2 203:115:88 5589 VSIDS-f2 2000 1000 0 100 120 140 160 180 200 220 240 Instances 9/18
On the classification of learnt clauses in COMiniSatPS -based solvers • In COMiniSatPS -based solvers all learnt clauses are split into three tiers: 10/18
Recommend
More recommend
