The challenge of translating scales in multilingual surveys and the value of leveraging “legacy materials”

by Pisana Ferrari – cApStAn Ambassador to the Global Village

To quote scholars Barbara Byrne and Fons van de Vijver (2010), the goal of translating scales in multilingual and cross-cultural comparative survey is to produce an instrument that “measures the same construct(s) in exactly the same way” across all language versions. We know of course that word-by-word translation of all the formal or semantic elements of a scale from one language to another does not necessarily lead to functional equivalence. By functional equivalence we mean that the target version “works” or “functions” in the same way as the source.

Different types of linguistic and cultural aspects come into play. Once it is established that the construct can be measured in different cultures, it remains a challenge to strike the best possible balance between accuracy and fluency, because one needs to take into account not only possible measurement effects such as social desirability or acquiescence, but also linguistic and cultural effects such as differences in conversational norms.

At the 2018 annual conference of WAPOR, the World Association for Public Opinion Research, we were honoured to be given the opportunity to present in the course of a session on challenges in comparative surveys on the value of “Using a Searchable Repository of Scale Translations in Multilingual Surveys”. The paper was co-authored by Steve Dept and Elica Krajčeva from the cApStAn Brussels headquarters and as the topic of translating scales in multilingual surveys is still very relevant in the survey translation world, we prepared a blog post to summarize the paper for you.

Validated translations of widely used response scales are a precious resource

Empirical evidence of the impact of linguistic and cultural parameters on the reliability and validity of translated or adapted scales exists. At cApStAn we have developed a searchable repository of translations of widely used response scales. These are linked to language/item interactions observed during the analysis of results, whenever such documentation is available. New survey questions can be automatically matched with this repository, with a view to flagging known problems, and possibly prevent their replication.

In this way, a number of challenges and biases related to questionnaire scales that are already known can be identified in the earliest stages of survey design, using a combination of human expertise and automated recognition of previously observed patterns. This identification process can be embedded in questionnaire development and can help taking informed decisions aiming at comparability of data in multi-lingual and cross-cultural surveys.

Addressing known translation difficulties: cApStAn’s two-fold approach

“Upstream” (before translation starts): known difficulties reported before translation are discussed with survey designers; guidelines for translators are developed, workarounds are suggested whenever possible and according to the intended construct — read more about our linguistic quality assurance work.

“Downstream” (after translation, before final version): discrepancies reported at translation/verification are discussed with survey designers to find the most appropriate approach according to the construct — read more about our linguistic quality control work.

“MemoryLn”, cApStAn’s in-house searchable repository of “legacy materials”

MemoryLn is a web-based search engine for translation memories and glossaries allowing to find expressions and patterns in segments from completed translation projects as well as in term glossaries. It contains scales and their validated translations from past surveys. It has been developed by our internal R&D experts and is a “work in progress”, constantly updated.

The largest project/client repository in it has scale translations in 54 languages. It covers surveys from 99 countries. A credentials system grants selective access, essential for confidentiality. Other cApStAn in-house tools include Inky, VeryFire™, and a number of applications we like to refer to as cApps.

Making informed decisions when translating scales in multilingual surveys

To conclude, we believe that to maximize the reliability and validity of the translated scales, one must keep the following steps in mind:

• Multidisciplinary discussions at different stages of questionnaire development along with a Translatability Assessment workshop.

• A well-maintained repository of existing issues, such as the one we have developed in-house

• Systematic documentation and robust knowledge management to enable researchers to follow-up on any issue that may show at data collection or analysis of results.

Photo credit: Graphic elaboration by Graphillus/Milan, background photo by Pawel Czernwinski at Unsplash