Translation/adaptation of PISA Global Crises Module (GCM)

About the project

The OECD’s Programme for International student Assessment (PISA) developed PISA Global Crises Module (GCM) to collect information on how education systems around the world have responded to the COVID-19 pandemic and how students’ learning experiences and school preparedness have changed. This is in response to a request from the PISA Governing Board (PGB), who felt that it is important for PISA to collect this information and expect that data from the GCM will be useful in policy discussions around mitigating educational disruptions caused by future pandemics or other global crises.

Project Scope with cApStAn

A limited set of school and student questionnaire items were developed following a process that involved leading questionnaire development experts, PISA National Centres, as well as small-scale cognitive interview studies in three countries. GCM Global Crisis Module questions were added after the translations of the PISA Field Trial questionnaires were finalised. As the timeline was a major concern, cApStAn was commissioned to produce 112 language versions for 87 countries/economies (the PISA Participants).

The respective roles of the PISA Participants and cApStAn were thus reversed compared to the conventional PISA translation and verification design, in which PISA Participants produce the translations and cApStAn verifies them.

Challenges

• The main challenge was that the GCM questions were added after the translations of the PISA Field Trial questionnaires had been finalised.
• As PISA participants would be put under extreme time pressure if they were to organise the translation—and, additionally, several PISA national centres were confronted with a capacity shortage due to Covid-19 measures—, cApStAn was commissioned to organise the translations and provide them to the Participants.
• In addition to organising the centralised translation of the questionnaire items, cApStAn also conducted “upstream” work to optimise the source text, via a translatability assessment, and the training of linguists involved.

Translation Process diagram

Change Request and Implementation

 Participants received a preview access to the translations and could suggest changes in an Excel form, providing rationales. The questionnaire developer (ETS) approved or disapproved requests and, once approved, the proposed translation was checked for linguistic appropriateness and implemented by cApStAn linguists. Participants then performed a final review. If no further issues arose, the translation was signed off and the translations were entered in the online survey platform.

All the steps are documented in a tracking form. Below is a fictitious example in such a tracking form (simplified for the illustration purpose).

Outcome achieved

The centralised translation organised by cApStAn allowed to:

• Shorten the timeline by 6 to 10 weeks
• Save Participants the effort of organising translations
• Reduce the amount of communication between Participants and PISA contractors
• Use translation technology to its full extent

The entire linguistic quality assurance process including pre- and post-translation work was aimed at ensuring that the questionnaire items met the highest standards of linguistic, cultural and functional equivalence.

Photo credit Annia Spratt at Unsplash

Want to try this out on your materials?

Select some sample items or sample questions, and request a free pilot at hermes@capstan.be or contact us via the form below for more details.