Leave no one behind: Digital user research in rare disease
- User research
When measuring the usability of a digital product or service, there are different metrics that can be used to clearly benchmark its success.
Here, we’ll take a look at System Usability Scale (SUS) scores — what they are, when they can be used, and how they can help you measure the success of your digital healthcare or pharma solutions.
There are different metrics that can be used to clearly benchmark the usability and success of a digital product or service.
They can provide a clear snapshot of the usability of your solution in a given moment, and comparing scores over time will help demonstrate whether the changes you’re making are having an impact and improving the digital experience for your customers.
System Usability Scale (SUS) scores are a commonly used metric that allow you to test your usability by looking at effectiveness, efficiency, and overall ease of use. Here, we explain what SUS scores are and how they can be used to help you measure the success of your digital healthcare or pharma solutions.
The System Usability Scale (SUS) is a quick and easy way to evaluate the usability of a product. It was created by John Brooke in 1986 and is often referred to as a "quick and dirty usability scale".
The SUS is used to measure things such as the perceived ease of use and is employed across various digital products and services to help UX practitioners determine if there is an overall problem with a design solution. However, it’s important to note that the SUS is not a diagnostic tool, but rather a means of providing an overall assessment measurement.
The SUS is made up of 3 key characteristics:
Effectiveness - whether users are able to successfully achieve their objectives
Efficiency - refers to how much effort and resource is used in order to achieve those objectives
Satisfaction - an understanding of whether the experience was satisfactory for the user
Users are presented with a 10-question SUS questionnaire after a usability testing session.
The aim of the questionnaire is to be able to get quick and unfiltered feedback from the user in an unmoderated environment. When carrying this out, you need to make sure that the project has been described well to avoid any confusion with users.
When using this questionnaire to gain feedback, we always aim to carry out this process with a minimum of 5 users to ensure reliability in your results. If you can speak to a larger sample size, this is even better to obtain more representative results. You should allow 1 – 2 minutes for users to complete the questionnaire, with only ranking scores being collected as feedback.
Below are the 10 System Usability Scale questions. Statements are answered using a 5-point Likert scale — participants rank each question from 1 – 5 based on how much they agree with the statements, with 5 being complete agreement.
I think that I would like to use this [project] frequently.
I found the [project] unnecessarily complex.
I thought the [project] was easy to use.
I think that I would need the support of a technical person to be able to use this [project].
I found the various functions in this [project] were well integrated.
I thought there was too much inconsistency in this [project].
I imagine that most people would learn to use this [project] very quickly.
I found the [project] very cumbersome to use.
I felt very confident using the [project].
I needed to learn a lot of things before I could get going with this [project].
To generate the final SUS scores, your UX researcher will follow a predetermined scoring tabulation methodology to get a SUS score out of 100.
The average SUS score is 68, so scoring above or below this figure will provide some immediate insight into the usability of your digital product. A score below 50 indicates a particularly poor user experience, and anything above an 85 would be considered excellent with minimal need for improvements.
The SUS scoring system can sometimes be seen as complicated. As results come out on a scale of 0 – 100, they are often interpreted as a percentage. This is not the case — they are percentiles, therefore it’s necessary to be aware of this when clearly communicating these findings with stakeholders.
SUS scores provide a quick and reliable method of assessing the usability of your digital design solutions. They have a variety of use cases in the context of digital product design in pharma and healthcare, such as:
Test the success of design changes: by comparing SUS scores taken before and after design changes have been implemented across your product or service, you can tangibly demonstrate the positive impact to product stakeholders — e.g. Average SUS scores improved by 35% in 6 months due to implementing UX or UI design changes.
See a real-life example from our work with healthcare provider, Vhi, here.
A/B testing between prototype designs: by giving half of your users one design and half another, you can clearly compare how users respond to different design concepts in order to make informed decisions when finalising designs and moving forward into development.
Evaluate the usability of prototypes: as SUS scores are relatively quick and low effort to obtain, they can be used regularly to evaluate the usability of your digital product and keep you on track — for example, at the end of each design sprint. They can help uncover any issues so that they can be resolved early and avoid having to backtrack or waste effort and resource on developing solutions that your customers find difficult to use.
Easy to create and scale: SUS scoring processes are quick and easy to create and build into your design process as there is a ready-made format to follow in regards to questions, but also how to analyse and interpret results. They can be carried out with small or large sample groups of users easily.
Valid: SUS scores allow you to clearly differentiate between usable and unusable digital solutions, ascertaining the validity of your product design decisions.
Reliable: Using SUS scores has been shown to be more reliable than custom questionnaires made case-by-case by researchers and product teams. They are able to detect differences and come to conclusions using a smaller sample size.
Affordable and fast: The short set of questions means that it will not require too much time and effort for your UX researcher to administer SUS testing, which is particularly useful if you’re on a tight budget.
SUS scores are a useful way of quickly demonstrating the usability of your digital products and services and generating numerical data to show the impact of product design changes over time.
However, they should not be solely relied upon to provide a full picture of how your user interacts with your product. SUS testing is not diagnostic, so responses are unlikely to indicate specific problems that need to be addressed.
Use SUS scores alongside methods such as usability testing, user interviews and UX auditing to provide a holistic view of user experience and allow you to make meaningful changes and improvements.
If you’re looking for an experienced partner to support you with your digital product challenges, get in touch.