Why back translation is inadequate to assess quality in translated surveys

by Steve Dept, cApStAn co-founder

When translating a survey, a linguistically correct, fluent translation does not ensure that same constructs are measured, that the survey questions are understood the same way or, for that matter, that comparable data will be collected with the translated survey questionnaire. This explains why more sophisticated translation designs are required to obtain robust translated surveys. It also explains why back translation in surveys is not sufficient to evaluate translation quality. Richard Brislin described the back translation method in the 1970s (Brislin 1970, 1976), but one often forgets that he also described the shortcomings of back translation.

So, why isn’t back translation sufficient?

It is known that literal translations of survey questions don’t work well. In one question about how successful the police usually are in catching people who commit house burglaries, the literal translation of “house burglaries” into Slovak excluded burglaries in flats, because “house burglaries” only concerned houses. This would result in flawed data. The back translation would have shown a correct translation. Similarly, it may be necessary to adapt the wording in a translated survey question when a literal translation is likely to increase the social desirability factor. In such a case, the back translation might reveal a discrepancy that is in fact intentional and desirable. As a rule, more literal translations score well on the back translation scale, while they often result in DIF (differential item functioning).

Likewise, a back translation will not give you enough information about fluency and appropriateness of register in the translated survey. Whether a form of address is formal or informal in the translated survey, it is likely to be back translated to a uniform “you” if the source language is English. Lost grammatical matches between question stem and response categories may not be detected by back translation. The Cross-cultural Survey Guidelines say: “Comparisons of an original source text and a back-translated source text provide only limited and potentially misleading insight into the quality of the target language text.” (§)

What steps to follow to ensure quality of the translated survey:

Indeed, if the survey translation project is well prepared, and clear, user-friendly question-by-question translation and adaptation notes are developed, then one can task the translator with commenting on how s/he addressed each of these notes; and one can task the reviewer with formally confirming that each translation and adaptation note is satisfactorily addressed in the translated survey. The reviewer can use back translation to document potential equivalence issues, so that the questionnaire authors are in a position to understand the issue. The reviewer can also implement proposed corrections directly in the translated survey.

I believe that the enduring, overstated success of the back translation method comes from the fact that it gives questionnaire authors and principal investigators an illusory sense of control. While back translation will help detect mistranslations in the translated survey, which the questionnaire authors can then ask to correct (and then request another round of back translation), it does not always ensure that equivalence flaws are identified. It is a more effective and efficient approach to combine robust question-by-question translation and adaption notes with a high level of confidence in the reviewer’s ability to adequately report potential issues in the translated survey and fix them as needed, possibly with additional guidance from the questionnaire authors.

Check out our webinar on back translations here

Footnotes

• Cross-cultural Survey Guidelines: https://bit.ly/35aNeUQ