Why assessment translation matters

by John Kleeman, EVP at Learnosity and Questionmark

High quality translation of tests and exams really matters to our world. Assessments (tests and exams) provide huge benefit to organizations, to individuals and to society.

For organizations, good quality tests allow them to choose the best candidates to recruit, promote or develop based on merit. Within any organization there is a risk that hiring managers select people based on how much they like them or how similar they are to themselves. Assessments fairly measure work-related job skills and capabilities. Good assessments also allow an organization to test if its workforce know their procedures and products and check their capability on a regular basis to reduce mistakes and improve effectiveness.

For individuals, assessments allow bright people to show their merit, even if they are distant or disadvantaged. Several hundred years ago, Chinese imperial exams were used to select the best people for imperial official posts, in some cases avoiding the cliques at court or the high-born. Tests and exams perform a similar function today in many parts of the world. Achieving a high score at an exam can allow you to gain a qualification, enter a top university or obtain a top job, regardless of your social background or distance.

Lastly, tests and exams provide benefits to society. They are part of the safety mechanism that ensures people are qualified to do important jobs where there is risk to life or limb – doctors, pilots, engineers and many more. They also allow identification of skill mismatches and redirection of labour to new skills. They make a society’s allocation of people more efficient, allowing skilled people to flow to areas where they are needed and helping people learn new skills.

For these benefits to be achieved, tests and exams need to be as valid, reliable, and fair as they can be. It is worth describing briefly what these terms mean.

Tests and exams need to be as valid, reliable, and fair as they can be

Validity is the degree to which evidence and theory support the interpretation of test scores for proposed uses of tests. Simplistically, a test is valid if it genuinely measures what it claims to do. So, for example, a driving test is more valid if capable and safe drivers pass the test and unsafe ones fail it. Reliability is a measure of the consistency of a test – for example, a driving test that requires 30 minutes of safe driving is more likely to measure someone’s skill accurately than a test with only 3 minutes of driving.

Fairness means that all individuals in the population are treated fairly. A fair test does not disadvantage individuals because of characteristics irrelevant to the construct being measured.

So, this is where we get to translation. Supposing someone is obliged to take a test in a language other than their native language, is that fair?

If you are applying for a job as a teacher of French, then clearly, it is fair that such a test should be in the French language. And there are many other circumstances where it is very reasonable that a test is only in one language.

But many countries have more than one official language. And most large companies employ people who speak different languages. Even some universities and colleges teach in multiple languages. And organizations giving qualifications and certifications (think professional qualifications like accounting or IT skills like knowledge of a company’s software), clearly need to work for people with a variety of native languages.

In all these situations, tests just in a single language (often English) are likely not to be valid and not to be fair. They will not be valid because they do not just measure the skill tested, but they also measure the test taker’s ability to take the test effectively in a second language. And they will not be fair because someone who does not speak the native language of the test is likely to be disadvantaged against a native speaker even though language ability should be irrelevant to what is being measured.

For example, if a capable accountant in a non-English speaking country has to take an English language accountancy test for a qualification, and is compared to an accountant in the United States, how fair is this?

If tests are not valid and not fair, then some of their benefits are lost. No longer do tests provide a meritocracy where the best person can show their skills, unless that best person first masters a foreign language. No longer can organizations use tests to select the best candidates, as there is inherent bias to those who speak the test language. And qualifications and certifications lose some of their global appeal as they measure skill in language as much as skill in the qualification area.

Translating tests well and fairly is a difficult skill. Questions do not always map into different languages straightforwardly. Without careful quality control, translated assessments can be easier or harder in another language. But due to fantastic work by experts, there are generally valid, reliable, and fair ways of translating or adapting assessments to work in different languages well.

I salute those who work in this tricky field. Without you, assessments would be less valuable. They would be constrained to language islands – English assessments in the English-speaking world, French assessments in the Francophone world, Chinese assessments in China and so on. By making assessments translatable, you help make the world of assessments global, and bring the benefits of assessment to all people in the world.

As we now gradually coming out of the COVID-19 pandemic, it will be more important to be inclusive to all the smart people around the world, whatever their native language. I believe that those who translate and adapt assessments can play an important part in bringing us to a better world. Thank you for all you do.