“Translation is Not Enough: Preparing Your Exam for a Global Environment”

There is a growing interest and awareness of the need for a robust translation/adaptation design when administering tests in multiple languages and that this was reflected in the recent ATP Innovations in Testing 2019 conference program.

cApStAn LQC is proud to be gold sponsor of this exceptional forum and we were delighted to present with partners Prometric and PMI at a joint session on March 17, where we brought two decades of experience into best practices to follow, and pitfalls to avoid, when planning to go international with tests and assessments. Our warmest thanks go to Sierra Hampton-Simmons, from PMI, and Ibidun Layi-Ojo and Ashley Willnecker, from Prometric, for their valuable input into the session. cApStAn was represented by Musab Hayatli, Managing Director of the cApStAn Philadelphia office.

cApStAn has a holistic approach to multilingual projects which ranges from translatability assessments pre-translation, to delivering premium translations fit for use, and providing linguistic quality control post-translation. What follows is a brief summary of the ATP presentation, with a particular focus on the pre-and post-translation processes.

Mitigating translation issues

When tests are administered in different languages they must measure the same thing. Straightforward translation mostly does not work well. Meaning shifts (which are language-driven) and perception shifts (which are culture-driven) may affect the psychometric properties of items and compromise comparability across languages.

During the joint session Musab made the case for having linguists involved from the very early stages of test design. Tests earmarked for translation may contain idioms and colloquialisms that cannot be rendered adequately in target languages. There can be potential ambiguities (wording that can be interpreted in more than one way), cultural issues (text may difficult to adapt to a particular culture or language group), register adjustments to make (text not suitable for the target population, too formal/too informal), etc.. (an example of a cultural issue is illustrated in the photo below). Adaptation” (or “localisation”) comes in whenever direct translation could put respondents from the target group at a disadvantage or at an advantage. Fairness is key to comparability. Take the question “What kind of a doctor is a dentist”? In some languages the answer may be in the name, giving an unfair advantage… In German – Zahnarzt, Finnish – hammaslääkäri, Hungarian – fogorvos, all mean “tooth doctor”…

Before: Write a test suitable for translation

Time spent optimising the source version “upstream” drives quality of translated texts and saves time “downstream”. The cApStAn translatability assessment process checks each segment against standardised translatability categories. All issues are documented in a centralised monitoring tool, and, where deemed necessary, alternative text is proposed. A translatability assessment will validate the assumption that a source questionnaire is ready for translation into the intended target languages. It will ensure that the multiple language versions of a document will meet the highest standards of linguistic, cultural and functional equivalence.

After: Check the quality of the translation

As said above, translatablity assessment will save time further downstrean at the post translation stage. If this is not the procedure of choice, once a test is translated, an independent quality check of the translation is always desirable in order to maximise comparability. Translation verification should compare source and target versions segment by segment, check compliance with translation and adaptation notes, with a focus on sematic equivalence. At cApStAn we use a set of standardised intervention categories to report issues detected by the verifiers, and all issues are reported in a centralised monitoring tool. The intervention categories include missing or added information, adaptation issues, inconsistencies, wording issues, register, grammar, syntax, mistranslation, words left in source language, as well as format and layout.

Conclusion

cApStAn has organized, coordinated and supervised the translation/adaptation process in major international large-scale assessments (ILSAs) for two decades. Today cApStAn applies this expertise in multilingual psychological tests and talent management instruments. We have made what we believe is a modest but tangible contribution to this field by continuously shaping, testing and streamlining best practices in test and questionnaire localization, and by promoting them at conferences such as the ATP, E-ATP, I-ATP and ITC. 

For more information about cApStAn visit our website or contact us for more details.

cApStAn Linguistic Quality Assurance method recap:

Main photo: Musab Hayatli, Managing Director of the Philadelphia office, and Devasmita Ghosh, Enterprise Growth manager, Brussels office, at the cApStAn booth at ATP 2019.