Week 2: Conducting Research with Usability Studies
Conducting Research with Usability Studies INTRODUCTION
The module equips the learners with knowledge about how to conduct usability studies that are a requirement for assessing how well a user can actually complete tasks in your design.
This part of the training will primarily focus on the minimization of bias and encouragement of inclusivity in usability studies, and the crucial importance of developing the skill of note-making while you observe participants during a usability study. When integrated into the workflow of the learner, the skills opened up the possibility of the learner designing impassioned and effective user experiences that are appropriate for a wide array of audiences.
Learning Objectives:
- To take accurate, neat, and efficient notes of usability studies.
- Finding and discussing advice about minimizing bias in interviewing.
- Replicate and elucidate the pitfalls of moderation bias during interviews.
- Raising rapport build quality with participants.
- Can conduct both moderated and unmoderated usability testing.
- Students should understand and articulate the purpose and methods of usability: .
TEST YOUR KNOWLEDGE ON THE IMPORTANCE OF USABILITY STUDIES
1. Why are usability studies important for UX research?
- They allow designers a final opportunity to get user feedback before production.
- They solicit positive feedback from users.
- They provide user feedback that helps the design team make improvements to the user experience. (CORRECT)
Correct: Usability studies assess the difficulty of completing core tasks in a design.
2. In unmoderated usability studies, participants test out the prototypes without human guidance. What are some benefits of an unmoderated usability study? Select all that apply.
- Participants remain fully engaged as they complete the activities.
- Participants operate the product in a real-world environment. (CORRECT)
- Participants complete the tasks on their own time and in their own space. (CORRECT)
- Participants may feel more comfortable giving feedback without others around. (CORRECT)
Correct: Researchers observe how users interact with a product on their terms of doing it freely without any kind of input or mediation from a moderator.
Correct: Unmoderated usability studies provide more freedom for participants to do a task. This can make it easier for them to give feedback in a very relaxed and natural way. Therefore, they use the product in real-world settings.
Correct: This is because in unmoderated usability studies, people know that what they write will not appear anywhere else until the final results are ready since there is no one sitting next to them. In addition, such solutions allow participants complete tasks at a time most convenient to them, so they seem more spontaneous in giving feedback.
3. Which of the following is a limitation of a moderated usability study?
- The moderator can build rapport with participants.
- The moderator guides the participant through the study.
- The moderator can influence participants. (CORRECT)
- The moderator can follow up in real time.
Correct: The limitation for this kind of study type considering the possibility of the moderators’ influence or bias affecting the participants.
TEST YOUR KNOWLEDGE ON USABILITY STUDY BEST PRACTICES
1. When conducting a usability study for a UX design prototype, what are some components that should be included? Select all that apply.
- Assumptions of participant reactions
- Sufficient sample of participants (CORRECT)
- Follow-up prompts or questions for participants (CORRECT)
- List of tasks for participants to complete (CORRECT)
Correct: Usability studies should have a participant sample size large enough to identify major user issues while remaining cost-effective. Participants should be provided with a set of tasks to complete, followed by prompts or questions for other clarifications.
Correct: Usability studies should maintain an adequate sample size and provide participants with a list of tasks to complete, along with follow-up prompts or questions for feedback.
Correct: Usability studies require a significant sample size samples should be provided with a set of tasks to complete, followed by prompts or questions to further teach.
2. A UX team is building an app for a pharmacy pick-up service. They want to test a prototype in an unmoderated usability study. What should the researcher pay attention to while viewing the video of the participants?
- How each user responds to the tasks they are asked to perform (CORRECT)
- How the app’s code was written
- How well the app performed for the users who successfully completed each task
- How the app looks in the video
Correct: It is severely vital to collect feedback as they engage with the prototype. It will help both the researchers as well as users to know that how easy or quick is the accomplishment of most tasks; also being able to discover any usability hurdles.
3. During a moderated usability study, what techniques should you employ to effectively communicate with participants? Select all that apply.
- Ask leading questions
- Ask open-ended questions (CORRECT)
- Expand participant feedback with follow-up questions (CORRECT)
- Repeat participant feedback for clarity (CORRECT)
Right: A moderated usability study will be started by asking open-ended questions and present the participants, follow-up with clarifying feedback to deepen the discussion further, and record an exact conversation with the participants for the sake of review or clarification.
Correct: Confirm unclear feedback by repeating the subjects’ responses clearly and then begin with open-ended questions for optimal response. Subsequent questions would better open conversation to a potentially bigger and more detailed evaluation of the subjects’ feedback, as the latter could otherwise become stimuli for subjective responses.
TEST YOUR KNOWLEDGE ON POTENTIAL BIASES DURING INTERVIEW MODERATION
1. A UX researcher is conducting a usability study for a banking app. Some of the participants include people over the age of 60. The researcher assumes the participants will have difficulty using some of the app’s features due to their age. What type of bias is the researcher displaying?
- Social desirability bias
- Friendliness bias
- Implicit bias (CORRECT)
- Serial position effect
Correct: That is an example of implicit bias because this comes from an already determined stereotype on the part of the researcher about individuals belonging to age groups so identified.
2. At the beginning of a usability study, the moderator establishes a very strong rapport with the participants. As a result, the participants provide only positive feedback on the design prototype. What type of bias are the study participants displaying?
- Friendliness bias (CORRECT)
- Serial position effect
- Implicit bias
Correct: Friendship bias refers simply to the fact that when a person agrees with a friend of his or hers in order to keep the conversation non confrontational. They may shy from disagreeing with the moderator in this case.
3. A designer conducts a usability study to get feedback on a map feature for a delivery app. One of the interview questions asks, “Did you like using the map feature to track your deliveries?” Which type of bias may occur if participants provide only favorable feedback?
- Implicit bias
- Serial position effect
- Social desirability bias (CORRECT)
Correct: The study question only asks participants to reflect on the positive aspects of their experience with a product. This may result in social desirability bias.
TEST YOUR KNOWLEDGE ON BIAS DURING INTERVIEWS
1. Which of the following is a good example of an open-ended interview question?
- Would you recommend this product to a friend?
- What tasks were difficult to understand or accomplish, and why? (CORRECT)
- Was the product easy to use?
- Did you enjoy the designs?
Correct: Open-ended questions like this one allow participants to give honest feedback.
2. Implicit bias is the tendency for people to answer questions in a way that will be viewed favorably by others.
- True
- False (CORRECT)
Correct: Internalized oppression refers to the oppressed group’s endorsement of the dominant group’s ideology. Stereotypes against the dominant culture reinforce beliefs of one’s inferior or inadequacy. Overt behaviour denotes obvious discriminatory hire that tends to have an adverse effect on whosoever might be its victim, which simultaneously promotes effects that force the victim to behave as though both parties were of equal status.
3. Fill in the blank: To reduce the chance of designer biases leading you down the wrong path, you should define the _____ criteria before conducting interviews.
- digital
- design
- research (CORRECT)
- fidelity
Correct: Without clear delimitation of research criteria, the impact on certain research content can be biased by formatting from research design.
4. At what point in the design process can you conduct usability studies? Select all that apply.
- When you identify the design problem
- When you have an interactive prototype (CORRECT)
- When you have a low-fidelity prototype (CORRECT)
- When the product is complete (CORRECT)
Correct: This is the most common time to conduct a usability study because it provides the design team with valuable insights on what needs to be revised or added before the product launch. Usability studies can also be performed once a low-fidelity prototype is available or when the product is complete.
Correct: Often referred to as concept testing, usability studies can be conducted at this stage of the design process. They can also be performed once there is an interactive prototype or when the product is finished.
Correct: Once the product is complete, conducting a usability study can help refine a feature or test it with a specific group of people. Usability studies can also be carried out when there is an interactive or low-fidelity prototype.
5. What is a benefit of moderated usability studies?
- They allow the moderator to ask specific questions and follow up in real time. (CORRECT)
- They allow participants to complete tasks on their own time and in their own space.
- They make it easier to see how a participant uses the product in the real world.
- They may allow participants to feel more comfortable giving feedback without others around.
Correct: This is an advantage of having a supervisor; by asking focused questions to gather detailed information, which can be later fed into the design and having real time feedback can improve the design.
6. What are the benefits of conducting unmoderated usability studies? Select all that apply.
- They allow real-time feedback.
- They encourage more honest feedback. (CORRECT)
- They give participants time and space to complete tasks on their own. (CORRECT)
- They provide more accurate user experiences to analyze. (CORRECT)
To avoid a meticulously planned usability test, people work in a world where they are intrusive in a way that raises the whole of their own ego concerns on behalf of others. Hence, in usability tests a participant feels better if he is contributing by suggesting comments in terms of comfort than as interpreted by a group of listeners.
A usability study without any moderator, more so than traditional usability research, provides users with enough time and opportunity to cover tasks independently. Research of this kind offers really discreet insights on real-user experiences and invites feedback to a greater extent.
The observation of product use in real-world environments is easier without a net that is drained pool, orderly or differently taken apart by a moderator. The slack available, minutes or even hours, given for participants to perform tasks individually, can lead to very honest, straight forward observations.
7. How can you reduce the possibility of influencing participants during a usability study interview?
- Remind participants they’re not being tested
- Keep emotions neutral (CORRECT)
- Build a rapport
- Remind participants to be open and honest
Correct: Whether or how much their temperament influences their input is beside the point. Preparing their mental state before starting the usability experience will be helpful to them in not mingling their emotions with all the things occurring and in connection to the study.
8. What is the first step you should take to lessen the influence of bias when interviewing?
- Let participants express themselves fully and in their own words
- Phrase interview questions thoughtfully
- Identify and admit your own biases (CORRECT)
- Be mindful of your body language
Correct: The first thing to do in minimizing any interview bias is to try to identify and admit your own bias. By acknowledging your biases and knowing them, you can help maintain a slightly impartial stance during a usability study. Such activities could require assistance, so actively asking for input from others can indeed help you see more in a different light.
9. In a moderated study, how can the moderator avoid making biased assumptions about how a participant feels from their body language or tone of voice?
- Repeat the participant’s actions to make them feel comfortable.
- Ask the participant how they felt at the point in the study where they appeared to show an emotion. (CORRECT)
- Encourage the participant to stop reacting to the question in that way.
- Make assumptions based on other participants’ reactions.
Correct: Asking the users how they are feeling when they are expressing their emotion, it allows avoiding the biases in the study, as the users get the opportunity to express themselves openly. People prefer giving their focus feelings as it ensures clearer communication concerning how they feel rather than just guessing their feelings or responses. This whole process encourages accurate feedback and reduces the designer’s biases.
10. You are considering when you should conduct a usability study on the app you are creating. When are common times in a product’s lifecycle for usability studies to be conducted? Select all that apply.
- After empathy interviews have been conducted and stakeholders have a better idea of the app.
- When you have an interactive prototype, which will give the design team insight on what needs to be revised or added before the product launches (CORRECT)
- When the product is complete, which allows you to change a feature or test if the product is usable with a specific group of people (CORRECT)
- For concept testing, when you have an early idea, like a low-fidelity prototype that is only somewhat interactive (CORRECT)
11. You are considering using a moderated usability test for the prototype. Your client inquires about the limitations that a moderated approach might have. What limitation could you identify for your client?
- The moderator could influence or bias the participants by accidentally letting their own thoughts or feelings come into the study. (CORRECT)
- The moderator could address any technical issues that participants may encounter.
- The moderator could connect with the participants and make them feel at ease in the prototype they are testing.
- The moderator could ask follow up questions to the participants in real time.
12. You are ready to moderate the usability study for the prototype. When you interact with the participants, what should your mood and demeanor be?
- Cheery, so participants are more likely to provide positive feedback about the prototype
- Pessimistic, so you will be pleasantly surprised if participants provide positive feedback
- Humorous, so you put the study participants at ease
- Neutral, so you do not influence or bias participants (CORRECT)
13. You are moderating the usability study for your prototype. One of the participants makes an observation about the navigation, and you are unsure of what they are referring to. What would you do to better understand their comment?
- Ask another participant to summarize what was said.
- Summarize what the participant said and move on to the next topic.
- Request that the participant use different words so you can understand what they are talking about.
- Repeat back what you think the participant meant and allow them to correct or confirm the statement. (CORRECT)
14. You are ready to moderate a usability study for your prototype. Your UX design mentor reminds you to be mindful of any biases you may carry that could impact the study participants and the study itself. What biases might you have that could impact the usability study? Select all that apply.
- Self-serving bias
- Serial position effect (CORRECT)
- Implicit or unconscious bias (CORRECT)
- Friendliness bias (CORRECT)
15. During the usability study you are moderating, a participant makes an observation that you agree with. You nod your head and smile in agreement. What type of bias does this convey?
- Confirmation bias
- Friendliness bias (CORRECT)
- Observation bias
- Self-serving bias
16. The usability study on your prototype is being recorded. You are taking notes as the study is going on. A junior UX designer asks you why you are taking notes if the study is being recorded. What would you tell them? Select all that apply.
- Note-taking allows a UX designer to be in real time and saves time overall as they do not have to go back and watch the recording.
- Note-taking allows a UX designer to write down participant body language during a particular question or interaction. (CORRECT)
- Note-taking allows a UX designer to capture additional thoughts during the usability study. (CORRECT)
- Note-taking allows a UX designer to identify parts of the study at which participants have differing tones and emotions. (CORRECT)
17. During the usability study, a participant states, “The icons are confusing and I do not find them helpful at all.” You are taking notes in the spreadsheet you have created. In which category would you place this information from the participant?
- Task completion
- Task
- Click path
- Observation (CORRECT)
18. You are planning to conduct an unmoderated usability study for your prototype. Your client asks you what the benefits are of an unmoderated usability study. What would you tell them? Select all that apply.
- An unmoderated usability study allows participants to be guided through specific tasks.
- An unmoderated usability study will help you see how the participant would experience your product without being guided for each step. (CORRECT)
- An unmoderated usability study allows participants to complete tasks on their own time and in their own space. (CORRECT)
- An unmoderated usability study may allow participants to feel more comfortable giving feedback without others around. (CORRECT)
19. You are taking notes in your spreadsheet during the usability study for your prototype. You have asked participants to follow a specific prompt and then, when observing, you assign numbers in one of your spreadsheet categories based on how the users followed the prompt. Which spreadsheet category would you use number values for?
- Task
- Task completion (CORRECT)
- Click path
- Observations
20. You let your client know that you are ready to conduct a usability study on the app you are creating for them. Your client is worried about the risks involved with a usability study. What would you tell them?
- Usability studies are just a checkbox that is marked in the design process. The user feedback does not really matter so there is low to no risk.
- Usability studies give UX designers a way to test their designs and get feedback without much risk and with a lot of reward. (CORRECT)
- Usability studies have a high degree of risk because the user feedback may not be positive and the designs may need to drastically change.
- Usability studies have no risk whatsoever. User feedback is gathered and the UX designer can implement it or not.
21. You have decided that it is best to conduct an unmoderated usability study for the app you are creating. Your client asks you to explain what unmoderated means in this context. What would you say? Select all that apply.
- A guide takes participants through the prototype and points them toward specific actions.
- A guide is present but remains silent as participants explore the prototype.
- The study is usually recorded on video, and the UX team reviews the video footage after the study. (CORRECT)
- Participants test out the prototypes without human guidance. (CORRECT)
22. You are ready to start a moderated usability study for your prototype. The participants have arrived and are ready to go. What would you say to participants before the study begins? Select all that apply.
- Tell them that you are looking for positive feedback and comments about the prototype.
- Let them know that their feedback is valuable and you appreciate their constructive criticism. (CORRECT)
- Ask general questions about how they are and what their day has been like to build rapport. (CORRECT)
- Thank them for coming and participating in the study. (CORRECT)
23. In the midst of moderating the usability study, a participant makes a statement that is unclear to you. You ask a follow-up question to clarify their answer. What are some examples of follow-up questions? Select all that apply.
- Was it difficult?
- Can you expand on that? (CORRECT)
- Could you tell me more about that? (CORRECT)
- What about that did you like? (CORRECT)
24. You are moderating the usability study for your prototype. One of the participants, who is a good friend of yours, states that they love the images used on a specific page. You agree and tell your friend how happy you are to hear that. What type of bias are you exhibiting?
- Social desirability bias
- Implicit bias
- Friendliness bias (CORRECT)
- Confirmation bias
25. Your UX design team is conducting a usability study on a prototype. The study participants have been chosen, and the UX design team is ready to conduct the study. Which members of the UX design team are expected to take notes during the study? Select all that apply.
- The entry-level UX designers
- The UX designers (CORRECT)
- The UX researchers (CORRECT)
- The UX writers (CORRECT)
26. During the usability study, you note how the study participants are navigating through the prototype. You want to document your observations in the spreadsheet you have created. Which category in the spreadsheet would you place this navigation insight under?
- Click path (CORRECT)
- Task completions
- Observation
- Task
27. You are preparing to moderate a usability study for your prototype. What can you do to help reduce bias during the study? Select all that apply.
- Ask participants to write down any biases they may carry that will impact the usability study.
- Ensure the usability study participants represent your key user group as well as user groups that are often marginalized. (CORRECT)
- Phrase questions in a way that allows for a diversity of responses. (CORRECT)
- Define the research criteria before the study begins. (CORRECT)
28. A participant in the usability study answers one of your questions with a sharp tone. You are convinced that this participant does not like the screen you are asking questions about. How would you confirm the participant’s response?
- Ask the participant how they felt at the point in the study where they appeared to show an emotion. (CORRECT)
- Ask the participant if they liked the screen or not.
- Ask the other study participants if they heard the response the way you did.
- Ask the participant why their tone changed.
29. You are planning a usability study for the app you are creating. You need to decide which type of usability study you will conduct. What are the types of usability studies?
- Moderated and unmoderated (CORRECT)
- Conventional and unconventional
- Independent and dependent
- Common and uncommon
30. As you are moderating the usability study for your prototype, you notice that participants are focusing on the positive aspects of their experience with the prototype and are minimizing the negative aspects. You are worried they are exhibiting social desirability bias. What can you do to negate this bias?
- Provide participants with a series of statements that came from other users, ask them which statement they relate to, and emphasize that there is no correct answer.
- Remind participants that you actually need honest feedback in order to improve the product. (CORRECT)
- Ask the participants to write down their feedback and submit it to you so other participants are not influenced.
- Provide a list of questions to the users in different orders so they do not put more emphasis on one part of the prototype over another part.
31. Your UX design mentor has recommended that you use a spreadsheet to record notes from the usability study for your prototype. They have advised you to create categories so you can organize your notes. In which category would you record the prompts that the participant will be guided to complete during the study?
- Observations
- Tasks (CORRECT)
- Click path
- Task completion
32. You are creating a spreadsheet to organize your notes from the usability study you are conducting. You have created categories in the spreadsheet and want to input the prompts that the participant will be guided to complete during the study. Which category should this information go in?
- Observations
- Task completion
- Click path
- Task (CORRECT)
33. Your client has asked that the participants in the usability study be allowed to explore the prototype on their own. They want to use an unmoderated usability study. You are worried that this type of usability study will not provide the feedback you are looking for. What would you tell your client about the limitations of an unmoderated usability study? Select all that apply.
- Participants are oftentimes disrupted from their tasks by follow-up questions and other participants.
- Certain participants, like single parents or night shift workers, will not be able to participate in the usability study.
- Participants get a list of fixed questions and tasks, but if they have issues or need technical assistance, there’s often no one to help them. (CORRECT)
- There is little to no control over the environment to ensure the participant’s full attention is focused on the key activities in the study. (CORRECT)
34. You are taking notes in your spreadsheet during the usability study of your prototype. Before the study, you added categories to your spreadsheet, including an observation category. What type of information would you populate in this category? Select all that apply.
- The participant’s name and each of the prompts that the participant will be guided to complete during the study
- The route the participants take to complete a task during the study
- The participant’s tone of voice or attitude while completing the tasks in the prompt (CORRECT)
- Information that may provide an indirect understanding of the behaviors and opinions of potential users (CORRECT)
35. You tell your client you are ready to start the usability study you have been preparing for. They ask you what the goal of the usability study is. How would you respond?
- It assesses how many participants visit the site.
- It assesses participants’ eye-patterning and how they view the pages in the design.
- It assesses how easy it is for participants to complete core tasks in a design. (CORRECT)
- It assesses the participants’ feelings on the images used and the coloring in the design.
36. You are ready to moderate the usability study for the prototype. You have prepared questions that are open-ended. What is an example of an open-ended question?
- Did the navigation make sense to you?
- What are your thoughts on the navigation? (CORRECT)
- Did you like it?
- Was it easy to navigate?
37. The usability study on your prototype is being recorded. You are taking notes as the study is going on. A junior UX designer asks you why you are taking notes if the study is being recorded. What would you tell them? Select all that apply.
- Note-taking allows a UX designer to be in real time and saves time overall as they do not have to go back and watch the recording.
- Note-taking allows a UX designer to write down participant body language during a particular question or interaction. (CORRECT)
- Note-taking allows a UX designer to capture additional thoughts during the usability study. (CORRECT)
- Note-taking allows a UX designer to identify parts of the study at which participants have differing tones and emotions. (CORRECT)
Conducting Research with Usability Studies CONCLUSION
With all those said, the research with Participants of UX is a necessity as feedback is most valuable when it comes to design. That is why usability studies are an integral part of research and know its right kind of conduct is the main objective of this course. Minimizing biases while designing for inclusivity and usability testing is one of the vital steps to incorporate testing methods. It is also worth mentioning that taking notes during studies is essential in recording the observations for later review.
At the end of this course, you will possess the tools and knowledge to progress from the conception of design choices to their realization through quality usability studies. So, why wait? Enroll in Coursera today and enhance your designs using extensive feedback from real users!