Concept testing is one of the main pillars of the UX research process. You can prepare a design, which heeds all the new and cool best practices – but until you pit it against your real users, there is no guarantee that it will actually perform well.
In other words, concept testing makes sure that your design is not only done right, but also fits users’ expectations.
Types of Concept Testing
-
Monadic testing – testing a single concept
A test where you prepare one design/concept and you test it with multiple respondents/focus groups. Ideal tools for this type of testing, based on the complexity of the concept, are Survey, Tree Testing, First Click Test, Five Second Test and Prototype Testing.
-
Sequential monadic testing – multiple concept testing
Includes multiple respondents/focus groups. You test each concept with a different group of respondents and then compare the results. The tools we suggest for this type of testing remain the same as for the Monadic testing, however, you would run separate tests for every alternative.
-
Comparative testing – comparing multiple concepts
You present several alternatives to one respondent/focus group and you compare their performances against each other. The perfect tool for this would be the Preference test. Thanks to it you can directly compare the alternatives of your image, video or audio designs. The other option for this would be a Survey containing mainly radio button questions, to show the preference of your users.
-
Protomonadic testing – when just simple Monadic testing is not enough
Adds a comparative part to your monadic testing. We suggest using tools fit for monadic testing, while attaching a post-study questionnaire, to ask respondents to give their opinion on alternative options of some of the design pieces, such as images or copywriting text.
We suggest this type of testing for later stages of the development process. Asking additional questions without a proper aim could lead you off the right course. Therefore we suggest adding these questions after you are done with the initial testing.
As you might have noticed, Survey proved to be a multifunctional tool that can be used for any type of concept testing. If you want to learn more about the use of Survey in concept testing, check out our guide on writing Concept Testing Questions.
When to test?
In the ideal world where you had an unlimited budget and time, we’d say that you should do concept testing all the time and at every opportunity. However, since we live in a harsh reality, it’s not always possible to test as much as we would like.
When planning your testing, it’s good to cover a few key points altogether so you don’t waste your resources. In short, you want to test your key decisions and additions or modifications to your concept. You don’t need to test every little change of a font or a button, but if you add an atomic part of your design, it should be tested properly.
So what are the stages of the testing process?
The genesis of your concept
Let’s say your concept has been barely defined yet. You are in the discovery phase. Now is a great time to gather as much relevant information and as many relevant ideas as you can.
Ask your future respondents about their habits, their pains, even their emotions. Fill your Surveys with open-ended questions. Get to know your users and their preferences early on to make sure your concept stands on a strong foundation.
Use the information you gather in the early stages to narrow down your concept and to make sure it fits your target audience, what appeals to them and the problems they need solved.
Once the designing starts
This stage begins once you already have some part of your concept designed and ready to show off to provoke some reactions.
No matter if it’s something that you’ve scribbled on a napkin in half a minute (low-fidelity) or a realistic take on what the final thing should look like (high-fidelity). Even a verbal description of your proposed solution or copywriting texts can be tested for whether your solution addresses the right problems and whether you’re getting your ideas across.
Load a Preference Test or a Survey study with your concepts and compare them to learn which version is the best for your customers. Once you know that you’re going in the right direction, further test the qualities of your concepts with First Click, Five Second or Prototype Testing to make sure it speaks the message you want it to speak and that using it is a fine experience.
The design phase is meant to be the one where you fail and fail often. Don’t worry about this, it’s all part of the process.
Be ready to abandon what isn’t working.
With the launch on the horizon
Finish line in sight, your concept is almost done, but you have one last test to do before you trust it completely.
Prepare a fully clickable prototype that completely simulates the main flows of your concept. Make sure that you test the cases which you deem as the
most complicated. However, don’t necessarily chase the most complicated scenario possible. Stick to the most important ones. Once your concept passes this test, you should be more or less safe. Even if you fail, it’s okay. It’s much better and much cheaper to fail sooner than after the launch.
Running a Concept test – what to do and when
1. Know your objective
Your goal has to be clear. Focus on what’s important for you based on the development stage you are in. Only if your goal is clear can you pick the correct method for the test you are about to conduct.
For example, open-ended questions aren’t the ideal solution if your primary goal is to statistically compare the performance of two concepts.
2. Target your audience
As we mentioned in the general guide, it is crucial to recruit participants who are actually part of your target audience.
Finding the right participants for your study may be difficult, but do believe us, it’s worth it. Pay close attention to picking the correct demographic attributes based on the personas you have defined during the analysis. Don’t waste time asking good questions of people who don’t have the answers you seek.
Also, if you think you could use some additional screening by asking a question, there’s a high probability you need it. It will lower the percentage of people who are qualified for your recruiting and so will most likely result in a higher cost per respondent.
But if you think about it, what’s more worth it, paying a tad more and listening to authentic testimonies of what troubles your target group, or saving some money at the beginning just to end up basing important decisions on wrong presumptions?
3. Make sure your concepts are ready
Don’t rush your tests. Remember your goal is to save money, not waste it on wrong designs and misspelled copywriting text. If your concepts contain glaring errors, participant attention will be drawn to those instead of what you actually need to validate.
Take your time to make sure the concepts are indeed what you are aiming to test.
4. Plan out your test and prepare your questions
Timing is an important ingredient for your test. Do you want the user’s first impressions or do you want them to ponder over your concept more immersively? Note there is no correct answer for this – both types of tests have something to offer, but you should know what exactly you’re aiming for.
Once that’s been decided, try to make an estimate of how long each task is going to take and sum it up to get the expected duration of the test. With a moderated test, we suggest not crossing the 1-hour mark, unless you have a good reason behind it. Longer than that and people tend to lose focus and will need to take breaks.
For unmoderated tests, the threshold is closer to the 30-minute point. The further down the line from the beginning of the test, the harder it becomes for the user to concentrate and their responses deteriorate in quality.
When you are done with estimating the duration of your test, it’s time to decide on the details of the questions you want to ask. To make sure you ask the correct questions, check out our Concept Testing Questions guide or watch the video below.
5. Don’t leave out the quantitative data
When testing a concept, one is likely to focus on more natural expressions such as like/dislike, good/bad, etc. Using a numbered scale allows you to more clearly understand the opinions of your respondents. It’s also much easier to analyze results in this format when you have tens or even hundreds of responses.
6. Don’t be afraid to test even your test
With enough time before you launch your study, create a copy of your study and conduct a pilot run to make sure all your tasks and questions are perceived as you designed them. It doesn’t matter what type of test you are about to run, even unmoderated studies can benefit from a small pilot.
There’s nothing worse than finding out you’ve collected tens or hundreds of results and because of unexpected holes and mistakes in your study design, people didn’t understand what you were asking of them.
7. Dive deep
Once your test is done, go and explore the results your chosen tool provides you. Aggregate the responses to your questions and look for similarities and differences. Compare the scores that your respondents gave to the alternative concepts.
Don’t forget to analyze the implicit data as well. Which task took them the longest? Why? Was it faster on a different version of your concept? Why? Make sure you don’t omit anything, even the smallest detail can be a game changer.
If you think you’ve asked “Why?” too many times, you are halfway there.
Quick recap
Done with this guide? Nice. What would be the most important takeaway?
There is always an opportunity to test your concepts, however, you most likely don’t have unlimited resources and time. Therefore always thoroughly think through what and when you should test, based on the phase of the development process you currently find yourself in.
If you decide to run your own concept test, remember that you have to know your objective. Even if you manage to target your audience precisely, don’t forget to make sure your concepts are ready and refined, you don’t want to show a flawed concept to your respondents.
Showing is one thing you can do, but don’t limit yourself and while planning your test, prepare some questions as well. Make sure you get the most out of your respondents. Qualitative data should be your main focus, but don’t leave out the quantitative data either. They are able to provide you with unique insights.
Even the best data from a flawed test are flawed, therefore don’t be afraid to test even your test. While on the note of not being afraid, dive deep, roam free through your data, look at every angle, and retrieve as much information as possible.
