Which Usability Test is Right for You?

Usability testing lays the foundation for building a successful product. Conducting a user test is a great way to obtain feedback, validate ideas, and/or confirm the need to iterate, improve, or start over with a new approach. This article will define usability testing, three different types of user testing, their pros and cons, and the best time to use each type of test. 

Usability Tests 

Discovering issues within a product’s design can help companies save time and money. Usability testing is a means to evaluate the user interface (UI) and user experience (UX) of a product with representative users before launching the product. Identifying problems, collecting qualitative and quantitative data, and gauging user satisfaction are all key objectives for those conducting usability tests. With a better understanding of participant reactions, UX design teams can discern changes that need to be made to better meet customer needs. 

Different types of usability testing 

There are many different types of usability tests including moderated, unmoderated, and guerilla. Each type has unique characteristics making them ideal for certain scenarios. Here’s a further look into all three: 

Moderated Usability Testing  

Moderated testing allows moderators to observe and converse with the user directly. These sessions can be held in-person or remotely via video. In-person sessions can be held in a corporate setting or a more casual environment, like a user’s home or place of employment. The moderators should exhibit expert knowledge of the product and its interface. Moderators are responsible for administration, assisting participants, answering questions, and recording feedback.  

Moderated pros: 

• Moderated testing is best for situations when the UX moderators and participants can interact at a high level  
• Because moderated testing is visual, observations can be made through behavior, responses, and body language. This can give great insight into the UX of the product 
• Witnessing unexpected behavioral outcomes, UX moderators can follow-up with needed questions and have unplanned discussions, providing further insight 

Moderated cons: 

• With moderate testing, travel costs and time normally factor into the equation, proving more costly than unmoderated testing. One way to diminish the cost factor is to opt for remote, rather than in-person testing  
• Another reason for the higher cost relates to participant compensation 
• Testing length can lead to lack of data because of decreased sample size 
• UX moderator needs to be cognizant of injecting any bias that can skew results 

Moderated testing activities 

• Typical moderated tests include (but are not limited to): 
• Remote Usability Testing 
• Lab Study 
• Interviews 
• Focus Groups 
• Click Testing 
• Ethnographic Study 

When to use moderated usability testing

Moderated testing is recommended for early development stage use cases, including competitive analysis, usability tests, prototype testing and discovery interviews. 

Unmoderated Usability Testing 

Unmoderated tests can be described as unobserved tests. Participants are not in the presence of moderators when tasks are completed. At the time of the study, the only one present is the participant. The participants can complete tasks and answer questions on their own time in a location of their choice without the assistance of a guide. 

Unmoderated pros: 

• Can be done anywhere at any time 
• Unmoderated testing is more cost-efficient than moderated testing 
• The ability to run a greater number of tests with a wider contributor pool  
• If tasks or questions need to be repeated once testing has ceased, unmoderated usability testing offers more flexibility than moderated testing  
• Without the presence of a moderator, participants do not experience the observer-expectancy effect, also known as expectancy bias (meaning they’re not reactive to a researcher’s bias). Participants can interact with products more naturally 

Unmoderated cons: 

• Without a moderator there to answer questions, tasks and questionnaires need to be very explicit 
• Inability to guide participants if issues arise  
• Less control over testing, which can lead to skewed data or bias if a participant seeks help completing a research activity 
• Unlike moderated testing, the results cannot be immediately questioned as there is a lack of behavioral observation  

Unmoderated testing activities: 

• Surveys 
• True Intent Studies 
• Card Sorting 
• Click Testing 
• Diary Study 
• Tree Testing 

When to use unmoderated usability testing

Unmoderated testing works well if a large sample size is required, and feedback is needed quickly. Specific test questions can be answered as contributors interact in their natural environment. Concept and design validation is quickly realized with a diverse group of contributors. 

Guerilla Usability Testing 

Guerrilla usability testing, also referred to as hallway usability testing, is a fast, cost-effective, and informal way to evaluate how effective an interface is by testing out a prototype on its intended audience and capturing their responses. Guerrilla test participants are selected at random, typically during live interceptions on the street, in a café, or a library. Often, participants are compensated for their time with a small gift (such as a free coffee). Researchers can get sufficient feedback to help discover problems with user experiences. Undetailed, guerilla testing should not be a replacement for formal testing. 

Guerilla pros: 

• Speed 
• Cost-effective 
• It’s better than no testing at all  
• Uses the public; Can interact with a large audience 
• Helps a company to avoid paralysis with decision-making 
• Can identify other areas needing further research 
• Validating/invalidating early assumptions 

Guerilla cons: 

• Participants may not be a part of the overall user group of the product 
• Insights garnered from this type of testing may not be enough to convince stakeholders of the results  
• Lack of time spent with participants  

Guerilla testing methods: 

• Approaching people at a public location and asking them to test your site, app, or prototype 
• It is ideal to target persona users, but a quick usability test with co-workers and colleagues can yield immediate feedback 

When to use guerilla usability testing

For those with time and budgetary concerns, guerilla testing is a popular option. Without doing a lot of research, valuable product feedback can be gathered. It also proves advantageous for small-iteration type work 

Moving Forward with Usability Testing 

Usability testing is the design research method of testing the functionality of a website, app, or other digital product by observing real users as they attempt to complete tasks. Depending on the situation, there are several different types of tests to garner user feedback; among them are moderated, unmoderated, and guerilla. When used in the proper scenarios, each can unlock unique benefits to design teams. Combining the usability tests is also an option to gather different types of feedback. No matter your strategy, usability testing is a great way to make informed decisions about your digital products and increase the chances of its success when officially introduced to the market.  

Apexon offers a full suite of experience services designed to turn all your digital touchpoints into competitive advantages. Our UX strategy and design services include prototyping and usability testing to give you confidence with your next product launch. To learn more, check out Apexon’s UX strategy and design services or get in touch using the form below.