Setting up a robust Linguistic Quality Assurance process for your tests and assessments on a low budget

Andrea Ferrari, cApStAn co-founder

You are going to administer a test or questionnaire in multiple countries, cultures, and languages. We don’t need to convince you that the reliable validation of translated data collection instruments requires­ a robust Linguistic Quality Assurance (LQA) design. A design that focuses on equivalence, comparability as well as appropriateness in each target culture.

cApStAn has over 25 years of world-class experience in providing language solutions for linguistic and cultural comparability. We do this for ‘mammoth’, well-funded projects like the OECD PISA programme, IEA’s TIMSS and PIRLS studies, the EU’s European Social Survey ̶ but not only! We also have quite a track record in providing scaled-down but effective support for more modest ventures, where budget is an important consideration.

Indeed, a modest investment can go a long way in terms of improving the quality of the instruments for your tests, assessments or questionnaires, and ensuring the fairness and equity of the instruments as well as reliability of the data that you will collect through these instruments.

Let us then review the ‘cost-savvy’ choices that can be made (or better not made) in relation to the 4 key stages of a test or assessment localization project (see figure 1 below)

Figure 1: the 4 key moments in an assessment localization project in cApStAn’s Modular Approach

Consultancy Services            

For a project with a limited budget, we would certainly forego ‘luxury items’ such as a workshop for item writers (where we train your item writers to write content geared for localization) or a full-blown DEI-BR consultancy (where we help to maximize Diversity, Equity, Inclusion, and Bias Reduction in your content).

The only item we would retain here, for a very modest price, is a workflow consultancy and content profiling (where we figure out together which combination of our building blocks you need).

Preparation

Our service offer for this stage of a project includes 5 modules:

B1 Translatability assessment
B2 Legacy content transfer & management
B3 Translation & Adaptation notes
B4 Bilingual glossaries, style guides
B5 Training & support to translation teams

While all of these contribute to LQA robustness, they can be foregone when cost needs to be contained except for two relatively inexpensive must-haves:
– B2 File and project preparation: a little effort from our tech team will ensure that we can get the most out of CAT tools and leverage language assets in the next stage.(?)
– B3 Translation & Adaptation notes: this is a keystone in a LQA design, simply put because linguists work better with guidance than if they have to guess what you mean. The production/validation of these notes is a collaborative process between cApStAn and the client, and savings can be achieved here if the client is willing and able to shoulder more of the (anyway modest) effort involved. Note also that these Notes play a role later, in stage D.

Translation and adaptation process

These are the 6 modules to choose from at this stage:

C1 (Single) translation
C2 Machine translation + post-editing solutions
C3 (Double) translation + reconciliation
C4 (Team) translation + adjudication
C5 Adaptation to local context and usage
C6 Domain expert /SME review

Projects with sensitive high-stakes content often require some variation of the TRAPD (Translation, Review, Adjudication, Pretesting and Documentation) procedure, pointing to either C3 or C4 which at least triple the cost versus a single translation (C1).

Depending on which source and target languages are involved, one way to reduce cost if C3 or C4 must be retained is to replace one of the two initial human translations by a neural machine translation (the reconciler or adjudicator will nonetheless be able to use a second translation to make choices and refine the final product). (?)

But when cost really needs to be contained and C3 or C4 is not a requirement, a more efficient solution is C1 single translation followed by a revision in the next stage (D1), both involving carefully selected professionals (the reviser could e.g., have domain or subject matter expertise), who will both consult the Notes coming from B3 in the previous stage.

Lastly, the C6 Proofreading step is not to be skipped even in a low-cost approach, but it could be merged with the D4 step in the next stage. (?)

Linguistic Quality Control process

Here too we have 6 modules:

D1 Revision or proofreading (of a cApStAn translation)
D2 Translation Verification (of a 3rd-party translation)
D3 Error and update management
D4 Final checks, PDF or web reviews
D5 Coding of responses and evaluation of interviews

For a project with a limited budget, we would suggest opting for a D1 revision process following a single translation from the previous stage. Note that in our parlance, a reviser compares source and target, checks compliance with all translation & adaptation notes, and is instructed to aim for best balance between faithfulness to the source and fluency in the target. (?)

Because some defects can only be observed in the delivery mode (PDF document with final print layout, web page, tablet screen, CAPI or CATI interviewer’s script…), it is prudent to include the D4 final check step. In a low-cost approach, it can be combined with the C6 proofreading step. (?)

Concluding remarks

Every project is different, with its particularities, and it is not easy to state upfront, in a general way, where the resources available for LQA are best spent. This post provides some indications. In any case, it is always possible to design a LQA process that fits almost any given budget. Perhaps the only limitation we would set is similar to that of the hairdresser who will not give you a price for a haircut without the shampoo first 😊.

Sound us out at hermes@capstan.be, and you might be surprised.

Photo credit  Shutterstock