The purpose of testing is twofold:
1. Testing facilitates communication between the developer and the user.
The developer learns the user's goals, perceptions and expectations. Users
are able to have input early in the design process.
2. Testing evaluates the product. Potential problems can be identified early
in the design and development process. Testing can also allow the comparison
of alternative prototypes.
Developers and users possess different models.
Developers and users will (usually) have different levels of knowledge and different expectations. For example, developers use terminology that does not always match the terminology used by the target user group.
It is impossible to predict usability from appearance.
Developers intuitions are not always correct.
Intuition is error prone. Don't guess...Test !
Informal feedback is inadequate.
Informal feedback is hit and miss. It is important to test within an organised framework. Just asking the cleaner, your brother and three work mates what they think of the interface is not enough. Parts of the system may never be looked at, and problems will not be documented.
There is no average user. All users differ. Users have differing levels of ability in all areas - motor skills, intellect, learning abilities, device preferences, speed and motivation. For example, an experienced data entry operator will probably be twice as fast as an inexperienced operator and make 10 times fewer errors. The design must allow users - of all levels - to satisfactorily learn to use the system.
If the product does not work or does not do what its supposed to then: significant amounts of money and time have been wasted and the client may not get the return for investment. Furthermore, problems found late are more difficult to fix (and much more expensive) as opposed to problems found early --so test early and test often!
The above diagram illustrates the relationship between authoring and evaluation. Formative evaluation is in-process evaluation that makes sure the product does what is was ordered and supposed to do. It starts from the moment storyboards and requirements activities begin.
Summative evaluation happens after the authoring (at stages through production- not just at the end). All requirements are checked and storyboards are compared to the product to make sure that the product satisfies the specified design.
Alpha testing - the product is evaluated relatively early in the development phase - some content or functionality might be missing. The main focus in alpha testing is to review the application concept, format, user interface and page layouts.
Beta testing - the product is evaluated just prior to final release; product shoudl be relatively bug free and complete. The main interest in beta testing is to find bugs and content mistakes.
( extracted from Hillman,1998, Multimedia Technology and Applications).
Test participants are vital to the development process.
Test success largely depends on the cooperation, abilities
and insights of test participants. Therefore, it is important to assemble
test participants who closely match the target user profile.
If the product is designed to be used by a specific group of users - it
is logical to ensure that the prototype is tested on people representative
of that target user group.
Recruit strangers not friends. Friends may be inclined to withhold information
that may 'hurt the tester's feelings'.
Be sensitive to the test participant's feelings:
Testing should commence at the first stages of product development. Testing should continue during all stages of development, and should include as many user tasks and product components as possible.
Testing examples:
Competitive analysis: occurs during feasibility study.- explore user reactions to existing similar products
Exploratory testing: done early in the design stage - check if the user conceptual model is effective; does the user understand the objects and relationship between them? General look and feel - usually done either with paper designs or prototypes.
Macromedia provides some good guidelines for usability testing.
You can read them locally here
or online at http://www.macromedia.com/devnet/articles/usability_testing.html
--more specific questions found at http://www.macromedia.com/devnet/articles/usability_testing/definegoals.html
or
here is a local copy of some of the main parts of the link