Notes for Reading Seminar 2

One very important part of developing a product is evaluation, which the book summarizes into four points. The first is called “quick and dirty” which means that designers informally get feedback from users or consultants to confirm that their ideas are in line with users’ needs. The second is called usability testing and gives information that is used to calculate performance times, identify errors and help explain why the users did what they did. The third is field studies which helps increase understanding about what users do naturally and how technology impacts them.

Another important part when developing a product is to use the right methods, i.e. the combinations of techniques used to develop a product. There are five categories. The first is observing users, which is commonly done by notes, audio, video, and interaction. However, there is a challenge for evaluators in how to observe without disturbing the people being observed. The second is asking users – an obvious way of getting feedback. Interviews and questionnaires are the main techniques for doing this. The third is asking experts, which is relatively inexpensive and quick to perform. In addition, experts frequently suggest solutions to problems. The fourth is user testing, where data is collected so that performance can be analyzed. The fifth and final is modeling users' task performance.

The book also bring up a term called the DECIDE framework, which is a list of things to think about during evaluation. The list consists of the following points:

• Determine the overall goals of the evaluation.
• Explore the questions that need to be answered to satisfy the goals.
• Choose the evaluation paradigm and techniques to answer the questions.
• Identify the practical issues that need to be considered.
• Decide on the ethical issues and how to ensure high ethical standards.
• Evaluate, interpret, and present the data.

A final thing which is good to think about is to draw up a schedule for your evaluation study in advance. You should also do one or several pilot studies. This helps to ensure that the study is well designed and likely to be successful.

My question: if you find it difficult to find access to the user group, how do you get feedback on your prototype?