TY - JOUR

T1 - What do thinking-aloud participants say? A comparison of moderated and unmoderated usability sessions

AU - Hertzum, Morten

AU - Borlund, Pia

AU - Kristoffersen, Kristina Bonde

PY - 2015

Y1 - 2015

N2 - The value of thinking aloud in usability tests depends on the content of the users’ verbalizations. We investigated moderated and unmoderated users’ verbalizations during relaxed thinking aloud (i.e., verbalization at levels 1 through 3). Verbalizations of user experience were frequent and mostly relevant to the identification of usability issues. Explanations and redesign proposals were also mostly relevant, but infrequent. The relevance of verbalizations of user experience, explanations, and redesign proposals showed the value of relaxed thinking aloud but did not clarify the tradeoff between rich verbalizations and test reactivity. Action descriptions and system observations – two verbalization categories consistent with both relaxed and classic thinking aloud – were frequent but mainly of low relevance. Across all verbalizations, the positive or negative verbalizations were more often relevant than those without valence. Finally, moderated and unmoderated users made largely similar verbalizations, the main difference being a higher percentage of high-relevance verbalizations by unmoderated users

AB - The value of thinking aloud in usability tests depends on the content of the users’ verbalizations. We investigated moderated and unmoderated users’ verbalizations during relaxed thinking aloud (i.e., verbalization at levels 1 through 3). Verbalizations of user experience were frequent and mostly relevant to the identification of usability issues. Explanations and redesign proposals were also mostly relevant, but infrequent. The relevance of verbalizations of user experience, explanations, and redesign proposals showed the value of relaxed thinking aloud but did not clarify the tradeoff between rich verbalizations and test reactivity. Action descriptions and system observations – two verbalization categories consistent with both relaxed and classic thinking aloud – were frequent but mainly of low relevance. Across all verbalizations, the positive or negative verbalizations were more often relevant than those without valence. Finally, moderated and unmoderated users made largely similar verbalizations, the main difference being a higher percentage of high-relevance verbalizations by unmoderated users

KW - Faculty of Humanities

KW - Thinking aloud

KW - Verbalization

KW - Usability evaluation method

KW - User test

KW - Usability

KW - User experience

U2 - 10.1080/10447318.2015.1065691

DO - 10.1080/10447318.2015.1065691

M3 - Journal article

VL - 31

SP - 557

EP - 570

JO - International Journal of Human-Computer Interaction

JF - International Journal of Human-Computer Interaction

SN - 1044-7318

IS - 9

ER -