Testing: General Principles

All functions must have tests that exercise non-trivial cases. We require this to encourage good software development skills, but, more importantly, to force you to show us that you know what your code is supposed to do and that you’re not getting a correct answer by guessing. We will not give full credit to untested functionality, even if it is correct!

Good testing:

TestFest

Test-fest is a collaborative testing system. You upload both programs and tests. Your tests test others' programs, and vice versa.

You upload your tests in the form of a test suite. A test suite is a listing of test and test/exn expressions. For example:

(test (calc (interp '{+ 2 3})) 5)
(test/exn (calc (interp '{with {x 6} y})) "unbound identifier")

When you upload a test suite, it is run against our TA-written solution. If it passes, we add it to a global pool of test suites. (If it fails we notify you that it is buggy; you may contest our claim, claiming that your test suite is correct and that the TAs have a buggy solution.)

Before the handin deadline, you may update your test suite as often as you like.

You also submit solutions to the homework. When you do, the system tests your solution against the global pool of test suites and reports the results back to you.

How does test-fest affect your grade?

While the test-fest is in progress, it doesn't. You may freely update your test suite and your solution, without penalty. We encourage you to submit test suites to help your classmates. In return, you can run your solution against their test suites.

After the assignment is due, we will run your handin-solution against the final global test suite in the test-fest system. If the last test suite that you submit catches bugs in peoples' final solution, you will gain points for testing. Conversely, if your handin-solution fails on the final test suite, you will lose points for testing.

We will also review your work manually. That way, if you write a good test suite but it does not catch any errors (because everyone turned in perfect solutions), you will still receive credit for testing.