The approach to user research has shifted quite a lot lately. Traditional in-person usability testing used to the be ultimate technique to understand how users go about using a product or service. There is no denying that this kind of research is comprehensive and — if conducted well — also very valuable. However, it is also a fact that in-person user testing takes a lot of time, manpower, and money — in other words, resources that many companies don’t have.
We live in a time of technological advancement, high competition, and fast development cycles. Even if money and manpower are not an issue, there hardly is enough time to dive into extensive in-person user research — at least not repeatedly throughout the development process of a new product or design.
At the same time, the industry becomes more and more aware of the importance of user testing. It does not only help to detect issues that need fixing, it also creates an increased awareness of what people want, what they use a product for, when they use it, and how they use it. These context-related aspects are essential for both the usability and he user experience of a product.
Therefore it’s not an option to skip user research altogether, but it needs to adapt to the high speed of our time. It needs to be quick, simple, and seamlessly fit into an already optimized development process. The solution is simple: Automated remote user research.
Automated means there is no moderator involved and participation is completely time independent. Remote means the participants are not bound to any physical location, but participate remotely in an online research. This kind of user research is extremely flexible. However, it is important to consider a couple of key concepts to ensure optimal test results.
1. Start with a solid research plan
What we see a lot when discussing a remote research set up with our clients is, that people think they don’t need a research plan when conducting remote research. It seems that the simplicity of the approach makes people lazy and lets them believe good results will come to them naturally. This is wrong. No matter what kind of research you conduct, you need to start with a solid research plan.
Here is what your research plan should at least comprise:
- Main research questions (What is it that you want to find out?)
- Sub-research questions (What aspects are particularly important?)
- Define participants (Do people need to be familiar with your product or design?)
- Define a number of participants (Are you looking for qualitative or quantitative insights?)
- Define the context of your research (Do you need user scenarios? Do you need to offer any additional information? Etc.)
- Define a time frame for the research (When do you need the results?)
2. Be specific
The next important thing to consider when conducting remote research is to be specific. This is even more important that it might be for in-person research. The reason for that is the lack of interaction during remote research. You cannot ask follow-up questions or jump in with additional information if people get stuck during the test.
You have to be sure to offer sufficient background information and ask the right questions before you actually invite people to participate in your test. The most effective way to make sure of that is to test your test before it goes live. Sounds complicated, but really it is only a matter of minutes. Just ask a colleague or friend to take the test and report anything that needs clarification. Make sure you specifically state what you do and don’t expect from your test participants.
3. Keep the language simple
In line with being specific about your research is the advice to keep it simple. You don’t want people to get frustrated because they don’t understand your questions or don’t know what to do. It is important that you avoid jargon and choose a simple language that anyone can understand. Remember, participants cannot ask questions during the test and if they get stuck, they will leave.
Try to talk to your participants in their own language. If you target a group of people that you are not familiar with yourself, get someone to help, you who is. For example, if you need to conduct research with teenagers, why not ask your teenage nephew for his opinion.
4. Keep it short
One of the great things about remote research is that people can participate on their own initiative, during their own time. This makes them highly motivated and valid test participants for your research — if you decide to recruit participants on your website, or from an existing user base. Don’t spoil the quality of your results by making your research too extensive.
People don’t mind taking a quick survey. However, if you need more time that you initially asked for, or if people get bored, they will either stop the test, or run through it without giving you quality answers. Here is what you can do to keep people on board:
- Keep your questions short
- Keep any additional information to the point
- Offer an indication at the beginning of how long it will take to participate
- Add a little joke halfway through the test, or make use of some innocent irony to keep people entertained
- Split up long tests into multiple short ones
5. Define relevant participants
The question about who should participate in your remote research is quite controversial. Some people argue that for remote research it is just as important to test real users as it is when conducting in-person usability tests. Others say that the idea of remote research is rather quantitative and therefore it’s not important who participates.
In reality, we see a lot of different scenarios and we are convinced there is not one single right answer to this question. Basically, it depends on the goal of your research, so on your initial research questions. For example, you are planning a redesign of your website and you want to know if you should stick to your existing content structure, if people are missing any features, or if there are any other issues you should consider, then you should certainly ask current visitors. How could anyone, who has never seen your site before tell you what you should change?
However, if you want to find out if your redesign is clear and intuitive to new customers who are not yet familiar with your site, then you should ask strange people. Then again, whether the people you ask should be potential users or not, depends on your research questions. Do you need people to have a certain level of foreknowledge, a certain interest, or come from a certain area? Then focus on those people. If you are looking for more general feedback, you can focus on the numbers.
6. Make it personal
One of the biggest advantages of in-person user testing is that you get to be in the same room with your participants. You can observe them, talk to them, and make sure they feel comfortable in your presence. Still, there is the obstacle of an artificial test setting, which is believed to have tremendous impact on the test results.
In remote user research, people participate on their own agenda, in the familiar setting of their own living room or office. This natural test environment offers you a great advantage regarding the honesty of their feedback. Still, there are a couple of things you should do to make people feel appreciated:
- Use a personal tone of voice
- Introduce yourself to create a sense of interpersonal communication
- Offer contact details and be open for a conversation
- Express your appreciation by saying thank you
7. Be clear about the nature of your research
Another very important aspect of remote research is to be open about the nature of the research. The mediated nature requires additional information on who is behind the research and why it is conducted. Try to answer questions like:
- Who is the initiator of the research?
- Why is this research being conducted?
- What happens with with the collected data?
Just like in any form of user research, it is important to make clear that the test is not about the abilities, or the performance of the participants. Explaining what you test and what you are going to do with the results helps to take away uncertainty from you participants, helps them to relax and to be more honest.
8. Offer an incentive
While participants of an in-person user test usually get paid money for their time and efforts, online your have more diverse ways to compensate and motivate participants. For example, if you recruit participants on your site, beware of the persuasion principle of reciprocity, which describes that ”if you give something away for free, most people feel compelled to return the favor.” If you want visitors to your site to participate in an online research, make sure you have something to offer them in return.
For example, you can provide excellent information on your site, a free white paper download, great customer service, or simply the promise that you will actually listen to the feedback and make improvements in line with what your users really want.
If you decide to recruit participants through a mailing list, or social media, you can consider to offer an incentive in form of a 1 in 20 chance for a gift voucher.
9. Test continuously
With remote research, you can easily test little things, while already working on something else. And that’s exactly what you should do. Make user testing a consistent part of your development process — not something that can be considered in case there is still money or time left at the end. And trust me, if you don’t introduce continuous user research to the standard procedure of your development process — and if you don’t plan in the necessary resources from the very beginning — there will be neither time nor money left at the end.
Yet, without comprehensive user research, you will never know if you are building the right product in the right way, for the right people. Basically, this means you are on a blind flight until you eventually release your product. Any changes that need to be made to your final product will be expensive — both for your budget and your brand.
10. Combine different research methods
Last but not least, I’d like to acknowledge the possibility to combine different research methods for optimal results. Every research method has its advantages and limitations and the most comprehensive results certainly come from combining different methods.
For example, when looking at in-person and remote user research, here are three suggestions of how to combine them in different phases of the development process:
In the inspiration phase of a project, broad ideas can be gathered with in-person user research. These ideas can then be verified in with remote user research.
In the design phase, flaws in the design can be detected by doing in-person user research. These flaws should then be verified and prioritized with remote research.
In the final evaluation phase, in-person user testing can be used to verify the design and collect concrete feedback. A final remote research session can be conducted to gather more general and representative feedback.
Your thoughts
What are your thoughts on remote user research? Any experiences you would like to share? Please let us know in the comments below.