Interoperability-Guided Testing of QUIC Implementations using Symbolic Execution Felix Rath , Daniel Schemmel, Klaus Wehrle https://comsys.rwth-aachen.de EPIQ Workshop, Heraklion, Greece, 2018-12-04
QUANT ? Mozquic ? mvfst ? picoquic ? AppleQUIC ? Are the implementations interoperable? Motivation 2 Felix Rath
QUANT ? Mozquic ? mvfst ? picoquic ? AppleQUIC ? Are the implementations interoperable? Motivation 2 Felix Rath
QUANT ? Mozquic ? mvfst ? picoquic ? AppleQUIC ? Are the implementations interoperable? Motivation 2 Felix Rath
QUANT ? Mozquic ? mvfst ? picoquic ? AppleQUIC ? Are the implementations interoperable? Motivation 2 Felix Rath
QUANT ? Mozquic ? mvfst ? picoquic ? AppleQUIC ? Are the implementations interoperable? Motivation 2 Felix Rath
QUANT ? Mozquic ? mvfst ? picoquic ? AppleQUIC ? Are the implementations interoperable? Motivation 2 Felix Rath
QUANT ? Mozquic ? mvfst ? picoquic ? AppleQUIC ? Are the implementations interoperable? Motivation 2 Felix Rath
Can analyze complex communications Uncovered subtle, hard-to-detect bugs But: Deep interoperability testing requires more insight into implementations Our idea: Comparing belief states of endpoints Based on a common defjnition Provided by implementations We would like to propose the development of a comon way to query implementations for their current belief state. Results Summary • State-of-the-Art software testing approaches work 3 Felix Rath
Uncovered subtle, hard-to-detect bugs But: Deep interoperability testing requires more insight into implementations Our idea: Comparing belief states of endpoints Based on a common defjnition Provided by implementations We would like to propose the development of a comon way to query implementations for their current belief state. Results Summary • State-of-the-Art software testing approaches work ▶ Can analyze complex communications 3 Felix Rath
But: Deep interoperability testing requires more insight into implementations Our idea: Comparing belief states of endpoints Based on a common defjnition Provided by implementations We would like to propose the development of a comon way to query implementations for their current belief state. Results Summary • State-of-the-Art software testing approaches work ▶ Can analyze complex communications ▶ Uncovered subtle, hard-to-detect bugs 3 Felix Rath
Our idea: Comparing belief states of endpoints implementations Based on a common defjnition Provided by implementations We would like to propose the development of a comon way to query implementations for their current belief state. Results Summary • State-of-the-Art software testing approaches work ▶ Can analyze complex communications ▶ Uncovered subtle, hard-to-detect bugs • But: Deep interoperability testing requires more insight into 3 Felix Rath
implementations Based on a common defjnition Provided by implementations We would like to propose the development of a comon way to query implementations for their current belief state. Results Summary • State-of-the-Art software testing approaches work ▶ Can analyze complex communications ▶ Uncovered subtle, hard-to-detect bugs • But: Deep interoperability testing requires more insight into • Our idea: Comparing belief states of endpoints 3 Felix Rath
Provided by implementations implementations We would like to propose the development of a comon way to query implementations for their current belief state. Results Summary • State-of-the-Art software testing approaches work ▶ Can analyze complex communications ▶ Uncovered subtle, hard-to-detect bugs • But: Deep interoperability testing requires more insight into • Our idea: Comparing belief states of endpoints ▶ Based on a common defjnition 3 Felix Rath
implementations We would like to propose the development of a comon way to query implementations for their current belief state. Results Summary • State-of-the-Art software testing approaches work ▶ Can analyze complex communications ▶ Uncovered subtle, hard-to-detect bugs • But: Deep interoperability testing requires more insight into • Our idea: Comparing belief states of endpoints ▶ Based on a common defjnition ▶ Provided by implementations 3 Felix Rath
implementations We would like to propose the development of a comon way to query implementations for their current belief state. Results Summary • State-of-the-Art software testing approaches work ▶ Can analyze complex communications ▶ Uncovered subtle, hard-to-detect bugs • But: Deep interoperability testing requires more insight into • Our idea: Comparing belief states of endpoints ▶ Based on a common defjnition ▶ Provided by implementations 3 Felix Rath
Observations: Requires a new “implementation” in the form of a specifjcation Even then: Verifjcation of implementations almost impossible Many implementations already available Test multiple implementations against each other Interoperability-issues occur when “things go wrong” Result: Either non-compliance or bug in standard Testing Interoperability • Checking an implementation for standard compliance is problematic 4 Felix Rath
Observations: Even then: Verifjcation of implementations almost impossible Many implementations already available Test multiple implementations against each other Interoperability-issues occur when “things go wrong” Result: Either non-compliance or bug in standard Testing Interoperability • Checking an implementation for standard compliance is problematic • Requires a new “implementation” in the form of a specifjcation 4 Felix Rath
Observations: Many implementations already available Test multiple implementations against each other Interoperability-issues occur when “things go wrong” Result: Either non-compliance or bug in standard Testing Interoperability • Checking an implementation for standard compliance is problematic • Requires a new “implementation” in the form of a specifjcation • Even then: Verifjcation of implementations almost impossible 4 Felix Rath
Test multiple implementations against each other Observations: Interoperability-issues occur when “things go wrong” Result: Either non-compliance or bug in standard Testing Interoperability • Checking an implementation for standard compliance is problematic • Requires a new “implementation” in the form of a specifjcation • Even then: Verifjcation of implementations almost impossible • Many implementations already available 4 Felix Rath
Observations: Interoperability-issues occur when “things go wrong” Result: Either non-compliance or bug in standard Testing Interoperability • Checking an implementation for standard compliance is problematic • Requires a new “implementation” in the form of a specifjcation • Even then: Verifjcation of implementations almost impossible • Many implementations already available • → Test multiple implementations against each other 4 Felix Rath
Result: Either non-compliance or bug in standard Observations: Testing Interoperability • Checking an implementation for standard compliance is problematic • Requires a new “implementation” in the form of a specifjcation • Even then: Verifjcation of implementations almost impossible • Many implementations already available • → Test multiple implementations against each other • Interoperability-issues occur when “things go wrong” 4 Felix Rath
Observations: Testing Interoperability • Checking an implementation for standard compliance is problematic • Requires a new “implementation” in the form of a specifjcation • Even then: Verifjcation of implementations almost impossible • Many implementations already available • → Test multiple implementations against each other • Interoperability-issues occur when “things go wrong” • Result: Either non-compliance or bug in standard 4 Felix Rath
Example 1: 4 open streams + 1 closed stream 5 open streams Example 2: malformed (?) packet “Things Going Wrong” 5 Felix Rath
Example 1: 4 open streams + 1 closed stream 5 open streams Example 2: malformed (?) packet “Things Going Wrong” 5 Felix Rath
Example 1: 4 open streams + 1 closed stream 5 open streams Example 2: malformed (?) packet “Things Going Wrong” 5 Felix Rath
Example 1: 4 open streams + 1 closed stream 5 open streams Example 2: malformed (?) packet “Things Going Wrong” 5 Felix Rath
Example 1: 4 open streams + 1 closed stream 5 open streams Example 2: malformed (?) packet “Things Going Wrong” 5 Felix Rath
Example 1: 4 open streams + 1 closed stream 5 open streams Example 2: malformed (?) packet “Things Going Wrong” 5 Felix Rath
Example 1: 4 open streams + 1 closed stream 5 open streams Example 2: malformed (?) packet “Things Going Wrong” 5 Felix Rath
Example 1: 4 open streams + 1 closed stream 5 open streams Example 2: malformed (?) packet “Things Going Wrong” 5 Felix Rath
Recommend
More recommend
