FROM THE LITERATURE. Harzing’s article ‘Response styles in Cross-national research’ (2006) builds on the fact that survey responses are influenced by people’s response style. This enlightening article (and later additions) not only adds ‘culture’ as a determinant of response styles, it also described how response styles differ between countries. When culture has an impact on how people respond, we need to ask ourselves ‘Do surveys measure the same things across cultures?’.

Business scenario

In the late nineties, I worked as assistant Project Director in Singapore for a US international research company, who specialised in measuring attitudes of employees. Multi-National Corporate (MNC) clients engaged the company to conduct surveys to measure and compare staff satisfaction across subsidiaries in different countries.

How did we measure: Likert-Scale in surveys

The research company used Likert-scales (1 – 5 scales) to measure attitudes directly, and explore variations in agreement, frequency, quality, importance, and likelihood, etc. The scale assumes that the strength and intensity of an attitude is linear: on a continuum from ‘strongly agree’ (e.g. 1 on a 5-point scale) to ‘strongly disagree’ (e.g. 5 on a 5-point scale), with a middle ‘neutral’ position (e.g. 3 on a 5-point scale), and rely on a comparison of aggregated mean scores.

What did we measure: Management & Work Practices

The staff satisfaction survey looked at e.g., communication, building trust, leadership, empowerment, decision-making, accountability, feedback, resolving conflict etc. and the results of the surveys were then linked to the performance of each manager.

However, if response styles differ across cultures, is it appropriate to link staff satisfaction results from culturally diverse staff in teams, business units or subsidiaries to management performance indicators? Harzings’ article about her research in 26-countries comes in as a great help.

Do surveys measure the same things across cultures?

We need to look more closely at response styles.

The most cited examples of response styles are the tendency to Agree/Affirmative Response Style (ARS) or Disagree Response Style (DRS) with an item regardless of the content, and Extreme Response styles (ERS) versus Middle Response styles (MRS); the tendency to use the extreme or middle response categories on ratings scales.

According to Harzing, country-level characteristics (as researched by Hofstede’s 6 D Model) significantly influence response styles such as Agreement and Extreme Response Styles.

Furthermore, Harzings’ 26-country study presents great insight into how culture impact on participants’ response style. It shows that cultural differences, language of questionnaires, and language competence of participants provide insights into the direction of responses (ARS or DRS and EMS or MRS).

Cultural Differences as Determinants

(1) Determinant: Power Distance
First, Power Distance: the higher the level of a country’s power distance, the higher the agreement to people in positions of higher status. Overall, respondents in countries with high power distance values seem to prefer positive extreme response styles (e.g., 1 ‘strongly agree’ on a 5-point scale) to Middle response styles (e.g., 3 or ‘neutral’ on a 5-point scale).

(2) Determinant: Collectivism
Another determinant is Collectivism: collectivistic countries are characterised by harmony, avoidance of confrontations and more conformity behaviour. Overall, respondents from collectivistic countries appears to lead to a preference for agree and middle response styles (e.g., 2 ‘agree’ or 3 ‘neutral’ on a 5-point scale).

(3) Determinant: Strong Uncertainty Avoidance
In countries with Strong Uncertainty Avoidance, people experience higher levels of stress and anxiety, and have a need for clarity and structure. This determinant is associated with a higher level of agreement and associated with extreme positive answers (e.g., 1 ‘strongly agree’ on a 5-point scale): respondents are likely to agree with what they think the investigator sees as the ‘right answer’, rather than questioning this by disagreeing.

(4) Determinant: Extraversion
Finally, Extraversion, one of the big-five personality characteristics (Hofstede & McCrae 2004), bears a strong positive relationship to the level of expressiveness and exaggeration in communication styles across cultures: the higher the country-level extraversion, the higher the positive extreme response style (e.g., 1 ‘strongly agree’ on a 5-point scale).

language of questionnaires and language competence as Determinants

(5) English-language questionnaires Determinant
Harzing also found that for non-native English speakers, English-language questionnaires are shown to produce a higher level of middle (thus neutral) responses, while questionnaires in a respondent’s native language result in more extreme response styles.

(6) English competence Determinant
Also, English-language competence is positively related to extreme response styles and negatively related to middle response styles: when respondents feel they might not understand a question properly, they would be more likely to choose a safe (middle) response (e.g., 3 on a 5-point scale).

Interpret your survey results against response bias due to cultural determinants

Survey results might reflect differences in the way culturally diverse people respond to surveys, rather than picking up real differences in e.g., management phenomena. Therefore, culture response bias could be a real eye opener for international survey companies and their MNC clients: it warns against linking the wrong information.

For instance, how many respondents in the teams, units, or subsidiary are from high power distance, collectivistic, strong uncertainty avoidance or extraversion cultures? You might need to correct the overall results for cultural response bias before you start punishing or rewarding managers.

Let’s return to the question: do surveys measure the same things across cultures? They might not. When you link culturally diverse staff satisfaction survey results to manager performance, you have to be culturally knowledgeable to be able to design the appropriate questionnaire. Furthermore, you have to be able to interpret the answers and to make a fair call on the use of management performance indicators.

Harzings’ article provides invaluable insight about major differences in response styles between countries: it argues that a first step towards finding a solution to response bias is acknowledging that this can be a serious threat to valid comparisons across countries. This obviously impacts all kinds of surveys, not only your staff surveys.

References

Harzing, AW., (2006). ‘Response styles in cross-national research. A 26-Country study’. International Journal of Cross Cultural Management. Vol 6(2): 243–266 

More reading by Haring AW. on the topic of response styles in cross-national research https://link.springer.com/article/10.1007/s11575-011-0111-2.

_________________________________________
Joost Thissen, Partner & Interculturalist
joost@cultureresourcecentre.com.au
from the literature [email_link]