While we have always provided usability testing feedback as “part and parcel” of our everyday QA testing, we’ve recently begun to branch out and provide more specific feedback in this area; as part of this, we’ve started employing some methodologies that differ from our usual QA testing practices:
These are meant to address everything that the respondents (testers) are expected to both accomplish, and to know about the project. Usually our instructions are vehicles to discussion and collaboration; for usability testing, these are meant to cover all the tasks to perform, including admonitions not to interact with other team members so as to keep feedback as unbiased as possible.
Exit questionnaires, to be completed after all usability tasks are completed (or in phases during multi-phase projects)
We default to a seven-point scale (to allow for a true “middle ground” response point) with a breakout option (“don’t know/can’t rate”) to allow for elective non-responses where merited, but we can easily incorporate any desired rating scale. Questionnaire items are written with an eye toward reducing any implicit bias in wording or phraseology (for example, we try to avoid asking things like “Tell us how much you liked X feature”, since it leads the respondent towards positive responses instead of unbiased ones).
Demographic data collection, if requested
For the moment, this isn’t a very big item for us since we’re providing only small-group feedback, but it is something that we’re prepared to provide if it is of interest. Given that we work in a lab environment, we don’t anonymize our responses by default, but we’re prepared to do so if asked, or if requested demographic data would be reasonably considered sensitive.
Screen recordings of software use, if requested
We can capture videos of how testers interact with the software being tested for later review, either without audio, or with a running commentary by each respondent. We prefer to keep screen recording short and use them to illustrate activities that may merit further scrutiny and consideration (such as using a creative tool for a self-directed project) rather than recording potentially multiple-hours sessions, as a way to keep recordings to a manageable size and length.
One thing we don’t do is contract our usability testing out – we’re experts in software quality assurance testing, not broader market research, so while we’re happy to take a more scientific approach to usability testing, our sample groups still tend to be rather small and homogeneous. That said, the methodologies and even underlying work products (instructions, questionnaires, etc.) can be adapted to use by larger and more diverse groups of respondents in a straightforward manner.
As always, please contact us if you have any questions, are interested in more information regarding our usability testing methodologies, or are interested in engaging our testing services for one of your own projects.