@extends('template')

@section('title')
    Lab 8: Part 2. Testing with Optimism
@stop

@section('content')

# Lab 8: Part 2. Testing with Optimism

A **test case** is a piece of code that we will run while observing what happens (including what it prints and/or what it evaluates to). Often, this will be a single function call, and we'll be interested in the result value.

An **expectation** is an idea about what the correct result value (or printed output) should be when we run a test case. We can simply have expectations in our head, and check them by observing a test case ourselves, but we can also dictate them to the computer and have it check them for us.

## Using `optimism`

Back in [lab 2](/labs/lab02) we covered the use of `optimism` ([the
`optimism` reference is here](/reference/optimism)), but that was before
we had even covered custom functions. We're now revisiting this library
because it offers us a way to do automatic testing which can improve the
quality of our code as well as reducing the time it takes us to solve
problems.

**The provided file `tracing.py`**, which you've just used to test out
the debugger, includes three broken definitions of `hasCGBlock`, in
addition to the correct one. Your job in this part of the lab is to
define test cases that distinguish the correct version from the broken
versions: the correct version should pass all of your tests, while the
broken versions should fail at least one of them.

To define a test case in `optimism`, we use the `testCase` function.
First, we need to import the library, like this:

```py
from optimism import *
```

Now, we can do something like:

```py
testCase(hasCGBlock('CGAGGGCCUG'))
```

For every test case we establish, we also need one or more expectations
about what it should do. For this, if we're interested in a function's
return value (or the value of a test expression) we can use
`expectResult`. If we're interested in the printed output instead, we can
use `expectOutputContains`, although we'd also have to use
`captureOutput` before the test, and we'd eventually use `restoreOutput`
after the test. For this lab, only `expectResult` is needed. We can use
it like this:

```py
expectResult(True)
```

Note that when we define test cases, if they crash, or forget to restore
output, or cause other problems, this will affect the entire file's
correctness. To isolate them, we can put them in a `test` function, and
just call that function from the shell. So we can test when we want to,
but any issues with the tests won't cause problems with the normal
operation of other things in the file. The `tracing.py` file already has
a test function in it, and that's where you should add your tests.
Putting that all together, we should have a test function that looks like
this (the import could also go at the top of your file if you prefer):

```py
from optimism import *

def test():
    """
    This function is designed to be used to set up and run tests. If you
    put them here, they won't interfere with anything else you might want
    to do in the rest of the file, until you call this function.
    """
    # Put your test cases and expectations here

    testCase(hasCGBlock('CGAGGGCCUG'))
    expectResult(True)
```

If you run the file, nothing should happen, although your functions will
be defined. If you then call `test` in the shell, the results should look
like this:

<pre>
>>> <span style="color:gray">%Run tracing.py</span>
>>> test()
<span style="color:red">✓ tracing.py:100</span>
</pre>

The check-mark indicates that your test passed, and it reports the line
number where the expectation was established. The red color is because
it's part of the error log rather than normal printed output (although in
this case, it's not actually an error).

### More testing

<div class="partnerB">Partner B</div>

Now that you've got one test set up, create copies of it for the
`hasCGBlock1`, `hasCGBlock2`, and `hasCGBlock3` functions, with the same
expectation for each test. Run your file, and check which of the three
extra functions pass or fail this test. If things are set up correctly,
`hasCGBlock1` should fail the test, but `hasCGBlock2` and `hasCGBlock3`
should pass it. One test is certainly not enough to catch all possible
problems, after all.

With your partner, brainstorm at least 2 more tests to try, and apply
them to each of the four functions.


### Mass testing

<div class="partnerA">Partner A</div>

Copy-pasting these test cases can easily get to be a lot of typing, so
let's use a loop to minimize that. First, we'll define a `testCases` variable holding a list of tuples, like this:

```py
def test():
    ...
    testCases = [
        ('CGAGGGCCUG', True),
        ('CG', False)
    ]
```

Each tuple should contain an argument in the first position, and the
expected value for the result of `hasCGBlock` in the second position.
Now define a loop which goes through those test cases, and uses the
`testCase` and `expectResult` functions to define **four tests for each
iteration**, one for each of the variants of `hasCGBlock` we've provided.
For the smoothest solution, use multiple loop variables to deal with the
tuples in the list of test cases.

Refer back to the [lecture notes on testing and
debugging](/lectures/lec_testing_debugging) for an example of this
list-of-tuples strategy.

Once this is set up, adding a new test case and testing it with all four functions is as simple as adding a new row to our `testCases` variable.

### Finding counterexamples

<div class="partnerB">Partner B</div>

See if you can come up with a set of tests such that the correct
`hasCGBlock` passes all of them, while each of the incorrect versions
fails at least one of them. Any test where one of the incorrect versions
fails is a **counterexample** for that version.

You should start this process without reading the details of the code for
the broken functions, but if you're having trouble finding a
counterexample for one of them, try looking at their code or using the
debugger to step through it to figure out what's wrong, and design your
counterexample based on that.

Once you've found at least one counterexample for each broken version of
`hasCGBlock`, you're ready to move on to the next part of the lab.

@include('/labs/lab08/_toc')

@stop