This article presents a 5-step approach to developing a quantitative questionnaire that can be implemented in several countries. This type of questionnaire is particularly useful for multi-country surveys and market research.
Let us face it: this will not be the article that gets me the most views. But that does not matter. Conducting market research in several countries is such a complicated task that explaining how to conduct a multi-country questionnaire properly would be interesting. I will start by explaining the situations in which multi-country market research can be useful before detailing the 5 steps in preparing the questionnaire.
Contact IntoTheMinds Market Research Agency
When should you conduct a multi-country questionnaire? Our 2 most common use cases.
Many applications exist for a multi-country questionnaire (i.e., in several languages). We often receive such requests from customers. Here are the most frequent cases.
Measuring customer satisfaction in several countries
Customer satisfaction barometers are the most common use case we encounter at IntoTheMinds. You will need to conduct a multi-country questionnaire if you have customers in different countries. The mistake (often made) is to entrust the task of measuring satisfaction to the various subsidiaries, each creating a different questionnaire.
If you want to do things properly, you need to create a single questionnaire for each country, and that’s where things get complicated. If you want to measure concepts such as service quality, you had better read the entire article because service quality does not mean the same thing in Japan as in Germany, for example.
Measuring the appeal of an idea/product in several countries
Multi-country market research is the other major use case we cover. Start-ups, scale-ups, and more mature companies consult us to assess their chances of success in different countries and prioritize their efforts.
As part of this process, we conduct surveys that consider each country’s specificities. These specificities exist in B2B as well as in B2C and relate to the measurement of “breadcrumbs,” i.e., the problems that marketing targets want to solve.
How do I prepare a multi-country quantitative questionnaire?
Good market research is complex. Quantitative questionnaires are one of the best-known market research techniques and are no exception to this rule. The difficulty is greater when the questionnaire must be administered in several countries simultaneously.
One of the keys to success is to consider each market’s linguistic and cultural specificities to ensure reliable results. To explain the various steps involved, I’ve drawn on the work of Douglas and Craig (2007). They suggest a 5-step process.
To explain the process, I assume that the questionnaire has already been developed in the source language. In international market research, this will often be English.
Initial translation
The first step in the process is to translate the source questionnaire into all target languages. As we try to eliminate bias, it is usual to have the translation conducted by 2 independent translators. In this way, you can compare the results.
One of the difficulties is to ensure that the “items” are understood in the same way in all languages. This means that:
- the question
- the measurement scale
are correctly translated.
Here are a few concrete points to bear in mind:
- Attitudinal items. For example, “happiness” or “grief” may have different connotations in diverse cultures.
- Measures containing both positive and negative statements. Research shows that such formulations can lose internal coherence and dimensional structure in international contexts.
- Answers to questions concerning professional practices. When using “multi-choice” answers, ensuring that each choice applies to the target country is important.
Review
The second step is for the Research Manager to meet with the translators to check that;
- each translated version of the questionnaire consistently reflects the meaning of the source questionnaire
- is comprehensible to local respondents.
At this stage, the focus is on checking for cultural and linguistic nuances. The aim is to avoid any misinterpretation.
Correcting questionnaire inconsistencies
Once the preliminary version of the questionnaire has been defined, the next step is to resolve any inconsistencies between the different translations and check that each question measures the same thing in each language.
Pretesting the questionnaire
The questionnaire is then pretested with native respondents in each language. The pretest identifies any problems of comprehension and relevance. Aside from being a superfluous step, pretesting is necessary, whatever the context. It allows us to pinpoint obvious problems that may have escaped the attention of the person in charge of the research.
Administration and continuous iteration
Finally, for longitudinal research, where data is collected over several years, the idea is to integrate the lessons learned from each wave of data collection to improve the questionnaires. Here, the challenge is to preserve the comparability of data from one year to the next. We often encounter this problem outside the international context in customer satisfaction barometers.
Recommendations and conclusions
In conclusion, creating a reliable questionnaire is generally complicated. It becomes even more so when several countries must be surveyed. The difficulty lies in cultural differences, which influence how questions are perceived and responses.
To correctly measure the important constructs, you must follow a strict procedure for eliminating countries. It is the one I detailed above.
The criteria for success in this procedure are as follows:
- Hire professional translators who are also skilled in questionnaire design.
- Involve local references to validate the cultural relevance of questions
- Always favor simple formulations in questions, and avoid the use of negative forms
- Conduct several pretest cycles with local respondents to identify sources of confusion.
