Testing/Debugging CS 1111 OCTOBER 21, 2019 Warm-up problem Write a - PowerPoint PPT Presentation

Testing/Debugging CS 1111 – OCTOBER 21, 2019

Warm-up problem  Write a function that can take in a nested list of ints and return the sum of all numbers in the nested list.  Example: [[1, 2, 3], [4, 5], [6], [7, 8, 9 10]] -> 55

Upcoming deadlines  PA due dates  PA 12: This morning (now “late”)  PA 13: Due Friday  PA 14: Due next Monday  PA 15: Due next Wednesday  Exam 2: November 6  Lab this Wednesday in class

Consider the max function to the right  Is there anything wrong with this code?

Consider the max function to the right  Is there anything wrong with this code?  Yes!!!!

What about this max function?

What about this max function?  This one works fine!

The point:  It’s impossible to, at a glance, debug most code  With debugging, we are trying to fix logical errors  This is, find the problem with our logic  What strategy can we use to help us find bugs to fix them?  TESTING

Testing  Testing is the process of searching for bugs  We test to FIND bugs, NOT to prove our software is bug free:  Design tests as the following  Input  Expected output  What SHOULD return  Actual output  What DOES return  Test PASSES if Expected == actual, FAILS if it does not

The Hidden Cost of Not Testing  While most PAs so far have been small, they are getting larger  In CS 2110, you will write a single program across multiple files and hundreds of lines of code  In later classes and industry, you will write thousands, tens of thousands, and more lines of code  Bugs are costly:

NATS Air Traffic Failure  Caused by one bug in one line of code  “buried” among 4 million of lines of code  Fault had existed, but not manifested, since at least the 1990s, possibly earlier

Designing a test case  Consider the following test case for get_max_3:  Input  3, 2, 1  Expected output (calculate by hand)  3  Actual output (run the code)

Designing a test case  Consider the following test case for get_max_3:  Test Case 1  Input  3, 2, 1  Expected output (calculate by hand)  3  Actual output (run the code)  3  Result: PASS  wait, didn’t we have a bug

Multiple Tests  We almost always want multiple tests  This is especially true in functions with conditionals and loops  One test passing may have no bearing on whether or not another test passes  Example Test Case 2:  Input: 1, 2, 3  Expected: 3  Actual: 1  Result: Fail

Why do tests pass  Test Case 1:  Input: 3,2,1 Expected: 3 Actual: 3 Result: PASS  Test Case 2:  Input: 1,2,3 Expected: 3 Actual: 1 Result: FAIL  Why does the first test fail, and the other pass?

Why do tests pass  Test Case 1:  Input: 3,2,1 Expected: 3 Actual: 3 Result: PASS  Test Case 2:  Input: 1,2,3 Expected: 3 Actual: 1 Result: FAIL  Why does the first test fail, and the other pass?  Because test case 2 executes the “fault” that causes the bug  Test case 1 does not execute the fault

True or False  If a test executes a bug, that test will fail?  Thus, we only need to look at lines of code the failing test executes to find a bug, right?

True or False  If a test executes a bug, that test will fail?  Thus, we only need to look at lines of code the failing test executes to find a bug, right?  False!!!  Consider this test case (Test Case 3):  Input: 3, 3, 3  Expected Output: 3  Actual Output: 3  Result: PASS  But wait, that executes the fault!

Sound Tests  We want to make sure our tests are sound  That is, they are correct with the specification  Example: Test Case 4  Input: 4, 5, 6  Expected Output: 4 – this is erroneous  Actual Output: 4  Result: PASS – False Negative  Example: Test Case 5  Input: 9, 8, 7  Expected Output: 7 – this is erroneous  Actual Output: 9  Result: FAIL – False Positive

Importance of Sound Testing  We want all our tests to be sound  Unsound tests will mislead us to think  There are no bugs when there are  There are bugs where there are none  Sounds Tests required a detailed, well-understood specification  Example:  Mars Climate Orbiter  One system expected Metric  Another system produced English

Debugging: Tracing  If we have a failing, sound test, we now want to trace  To assist in tracing, we can print values as we trace to ensure they are correctly updated

Debugging Example:  Consider the code below:  Test case 0 and 1 work fine  Test Case 0: “python”, “o” Expected: 4 Test Case 1: “python”, “z” Expected: -1  Test case 2 fails  Test Case 2: “python”, “p” Expected: 1 def get_index_of(string, letter): index = -1 for i in range(1, len(string)): # check if we get the right character if string[i] == letter: index = i return index

Adding Print Statements to Help  This is a temporary solution: We don’t want to do this long term to leave this print statements in, but they can help us debug. Try this:  These print statements can help us find the reason Test case 2 fails.

We’re not done  There’s still one more bug we haven’t found.  With a partner:  Design test case(s) to look for bugs  When you have a failing test, use print statements to find why the bug occurs  Fix it, and remove the print statements

Parting Thoughts  Designing test cases is not easy  There are a number of “testing strategies”, but none will ever be perfect  Print statements are awkward  Adding and removing print statements may not be the most efficient way to debug code  Debug Mode, Automatic Testing  Print statements are a starting point while you are still learning the basics

Next Time: Dictionaries  For a given key, find a given value  willDictionary  FirstName: Paul  LastName: McBurney  Example: uh…well…dictionaries  Age: 31  For a given word, find a definition  Weight: More than I’m comfortable with  Phone book:  Hair: None;  For a given name, find a phone number  willDictionary[ “Age”] -> 31  Define multiple fields of a single entity