Write the acceptance tests (meaning any tests needed beyond unit and integration tests: system, functional, end-to-end, performance, security, whatever) during the planning phases of the project, rather than after development is complete. More importantly, if you’re trying an agile approach, write executable acceptance tests.
Executable acceptance tests serve two purposes. First of all, they’re a mechanism to involve your ‘customer’ or business expert in defining the tests which will verify that the level of quality and functionality he has specified is met. Secondly, they are a part of your automated acceptance test framework. If you can, avoid having to re-work your acceptance test cases into input to test automation tools.
There are quite a few test tool frameworks which allow this. The concept of data-driven testing isn’t new to XP, and there are plenty of tools which let you define tests in an Excel spreadsheet and then use these as input to your automated tests. XP-style test frameworks abound, as well. Here are a couple of sample JUnit tests:
The first test proves that logging in with a valid userid and password works correctly. The second proves that logging in without a valid password fails. If you have fairly savvy business users, you can train them to read this syntax. If not, it’s not hard for your programmers to write a tool to suck data out of an Excel test case and input it to JUnit.
But wait a minute, you aren’t on an XP team where your programmers are totally on board to help you with acceptance testing. That might rule out Fit (http://fit.c2.com) and FitNesse (http://www.fitnesse.org) as well. Fit allows you to define acceptance tests as HTML tables, which are input to fixtures written in Java, Ruby or some other language that executes the tests. FitNesse goes one better by letting you define the Fit tests as a Wiki page, a sort of modifiable HTML page that’s even easier to handle than HTML. It will also let you define tests in an Excel spreadsheet and paste them onto the Wiki page.
If you really can’t get any programmer support to write the simple fixtures that drive these ‘behind the GUI’ tests, check out Canoo WebTest http://webtest.canoo.com/webtest/manual/WebTestHome.html., This allows you to define test cases in XML format. Again, users might not be totally comfortable with XML format, so you might have to do the spreadsheet-to-test tool conversion.
It doesn’t matter what tool you’re going to use. Defining acceptance test cases up front helps your programmers write the correct code. It gives you a huge jump on your testing too.
A word about test plans. Nobody ever reads them, so try not to write them. They’re cumbersome to write and impossible to maintain. Just write test cases. If your organization requires you to deliver a test plan, try to include information that will be useful to you, such as a release plan, and refer to the test cases as much as possible.