OECD PISA

About PISA

The OECD PISA (Programme for International Student Assessment), collects data from 15-year-old students across the world, assessing education outcomes by measuring knowledge and skills in reading, mathematics and science as well as innovative domains such as collaborative problem-solving, global competence or creative thinking. Over half a million 15-year-olds from 80 countries and economies took the OECD PISA test in 2018. The PISA reports inform education policy-makers worldwide.

Project Scope

A decentralized translation approach was adopted from the outset: with cApStAn’s support before, during and after, translation teams in the participating countries and economies adapt the OECD PISA tests and questionnaires into their languages of instruction.

Our role is to harness the power of state-of-the-art translation technology and mobilise expert verifiers for all different language versions with a view to maximizing cross-linguistic, cross-cultural and cross-regional comparability. From the first survey cycle in 2000 till date, cApStAn has taken the lead responsibility for linguistic quality assurance and quality control of translated/adapted survey instruments. cApStAn is currently verifying materials for PISA 2021, the 8th survey cycle.

Challenges

The main challenges include:

• supporting a mix of professional and non-professional translators in the implementation of a sophisticated translation design;
• training linguists from 70+ countries to apply the same criteria and conventions when reporting on equivalence issues in the language versions they verify; and
• coordinating the revision of between 12 and 15 million words over a period of 6-8 weeks per survey cycle. 

Project execution process

Outcome achieved

• cApStAn provided technical support to countries and maintained communication with 80+ national centres and with 90+ expert verifiers
• Verification feedback was provided for 100+ versions of PISA materials (in 45+ languages)
• On average, each cApStAn linguist verified 160K words over a period of 3 weeks and documented each of his/her interventions
• Compliance with PISA translation and adaptation guidelines were documented for each language version of each question
• The analysis of PISA results shows that language-item and country-item interactions are contained and that the quality of the data allows for meaningful comparisons across countries.

Want to try this out on your materials?

Select some sample items or sample questions, and request a free pilot at hermes@capstan.be or contact us via the form below for more details.