Why Documentation of the Survey Translation Process is Crucial and How to Do It

by Elica Krajceva – Senior Project Manager, Cross-cultural Survey Specialist

Is it necessary and worth your time to keep a record of all important steps in a survey translation process? The answer is – yes, it is absolutely essential! Read on to find out the benefits of documentation.

Survey documentation is an important part of the survey design and implementation process. In the thriving multilingual survey research world, this also includes documenting the translation process; which has become an integral part of best practice for more than two decades. It is regarded as an essential methodological aspect in relation with comparability across languages and across cultures.

Documentation serves two main purposes and this is described extensively in Harkness et al. [1].  

We distinguish the overall process documentation (e.g., translation method applied, staff involved), the across-round documentation (e.g., tracking changes in translations in a continuing survey), the input documentation, and the output documentation as shown in the image below:

What is Input Documentation?

The input documentation is everything that is fed into the process by survey developers or translation managers to support the process of translation. It can include various types of instructions that specify product and process requirements, such as translation goal, translation process to be applied, required qualifications of translators, background information on concepts, or previous translations. This type of documentation comes in written form and in the form of trainings.

This is a necessary layer of additional information already emphasized in translation industry standards [2-4], it does not significantly increase a project’s budget, while the benefits in terms of translation quality are significant, and it gives an indication of the level of effort devoted to translation design and preparation, thus it is valued by the survey methodologists.

What is Output Documentation?

Output documentation is everything that originates from translators, adjudicators, or other parties involved in the actual translation: the final translated survey, comments on problems or adaptations, or draft translations. Data analysts find it useful when the final translation includes indications of and reasons for difficult-to-translate items (e.g., grammar, idioms, difficult-to-find equivalents), indications of cultural adaptations, indications of which items were changed from a positive to a negative wording, or indications of doubt (e.g., relevance of a concept) [5].

What are the Key Features of a High-Quality Documentation?

Detailed. Clear. Economical. Accessible.

Documentation should be adequate in terms of content, detail, and clarity. At the same time, it should be economical and only provided or requested where necessary. Ease of accessibility and cross-references are also important [6].

Monitoring forms, communication tools, or documentation systems all have received attention in recent years (e.g., ISSP monitoring reports, Survey Metadata Documentation System) [6]. The format evolved from Word to Excel, which allowed to align source question elements, question-specific translation guidelines, and multiple versions of translation with comments. Some translation tools integrate the specific needs of survey translation and documentation, too.

The Chapter Documenting the Survey Translation and Monitoring Process in Advances in Comparative Survey Methods [7] describes documentation in the Second European Survey of Enterprises on New and Emerging Risks (ESENER‐2)

It gives insights into many and diverse decision criteria that are applied in questionnaire translation. And it shows how concepts that trigger most of the discussions can be identified per country and across countries; truly problematic items (misunderstanding, ambiguous or difficult design, etc.) can be recognised and followed for possible effect on the survey data. 

So, documentation serves two main purposes: it provides the basis for quality assurance and monitoring and informs data users on the design and implementation of the survey.

If you’d like to learn more about how cApStAn documents the translation process of high-stakes surveys, do fill the form below, and we’ll get back to you as soon as we can.

References:

1 Harkness, J.A., Pennell, B.‐E., and Schoua‐Glusberg, A. (2004). Questionnaire translation and assessment. In: Methods for Testing and Evaluating Survey Questionnaires (ed. S. Presser, J. Rothgeb, M. Couper, et al.), 453–473. Hoboken, NJ: Wiley.

2 DIN (2006). DIN EN 15038 Translation Services – Service Requirements. Berlin: DIN Deutsches Institut fur Normung/Beuth Verlag.

3 ASTM International (2006). F 2575 ‐ 06: Standard Guide for Quality Assurance in Translation. West Conshohoken, PA: ASTM.

4 ISO (2015). ISO 17100 Translation Services – Requirements for Translation Services. Geneva: ISO.

5 Brislin, R.W. (1986). The wording and translation of research instruments. In: Field Methods in Cross‐cultural Research (ed. W.J. Lonner and J. Berry), 136–164. Beverly Hills, CA: Sage.

6 Mohler, P.P., Pennell, B.‐E., and Hubbard, F. (2008). Survey documentation: toward professional knowledge management in sample surveys. In: International Handbook of Survey Methodology (ed. E.D. de Leeuw, J.J. Hox and D.A. Dillman), 403–420. New York/London: Lawrence Erlbaum Associates/Taylor & Francis Group.

7 Behr, D., Dept, S., Krajčeva, E., (2018). Documenting the Survey Translation and Monitoring Process. In: Johnson, T. P., Pennell, B.-E, Stoop, I. A. & Dorer, B. (eds.) Advances in Comparative Survey Methods: Multinational, Multiregional and Multicultural Contexts (3MC), Wiley-Blackwell, 10/2018, 341-354