OECD PISA 2025 National Project Managers’ meeting in Lisbon, 12-17 March 2023
cApStAn is the international contractor in charge of linguistic quality assurance in PISA 2025. This includes training participating countries/economies to follow a standardised translation and adaptation approach and using mature translation technology: cApStAn’s presence was prominent in Lisbon for the first PISA 2025 National Project Managers’ meeting from March 12 to 17, 2023. PISA 2025 is the ninth survey cycle of the Organisation for Economic Co-operation and Development (OECD) Programme for International Student Assessment, the leading comparative survey that measures education outcomes around the world and links them to a variety of contextual variables, so as to provide relevant, meaningful and actionable data to education stakeholders and policy makers. The Lead contractor for PISA 2025 is the Australian Council for Educational Research (ACER), with whom cApStAn has already partnered for five PISA cycles and for many other surveys and assessments.
cApStAn’s team was nine strong: Eli Krajčeva, our PISA Lead Project Manager, coordinated Adrien, Anna, Irene, Josiane, Manuel, Morgane, Roberta and founding partner Steve Dept. One of the main goals of our presentations and workshops was to inform delegates from 90 participating countries about the intricacies of translating and adapting cognitive assessments. “Science” and “Learning in the Digital World” are the two core domains for which new test items have been developed in PISA 2025: countries had already submitted items and had also reviewed the item pool recommended for inclusion in the Field Trial.
On the first day, labelled Day zero, Steve presented the PISA translation and adaptation procedures, timelines, volumes, and workflows to all new participants (countries new to PISA, or new National Project Managers in countries that have already participated. There were also one-to-one consultations with the majority of the new countries.
On the second day, which was a Monday and labelled Day 1, international Survey Director Goran Lazendic introduced this cycle’s requirements, technical standards, and challenges in a plenary session. Several group activities allowed representatives from different countries to exchange experiences.
From then on, the cApStAn team occupied center stage for many of the sessions: lively presentations tailored to the needs of the participants, Q&A sessions, and three concurrent OmegaT workshops: OmegaT is a computer-assisted translation tool (CAT tool) in the form of free and open-source software (FOSS) that will be used by all national teams to translate and adapt, and by cApStAn to verify translations.
There were also face-to-face consultations with delegates of all new countries and all countries that requested consultations – this amounted in close to sixty problem-solving discussions with representatives from PISA Participants, with Béatrice Halleux, the Translation Referee, and with members of the cApStAn team.
A collective diplomatic success was achieved in facilitating cooperation between countries that share the same language of instruction. All Spanish-speaking countries (the Grupo Iberoamericano de PISA) will start from a common version and adapt only what needs to be adapted to local usage and context. The same will happen for all countries/economies that have Russian as one of their languages of instruction. In both of these cooperations, one country team will receive extensive coaching and technical support to produce a version rapidly, following a double translation design. The Chinese economies have also agreed on a cooperation model. It is heartening to see that all reluctance was overcome and resources were shared among countries. It is part of the PISA spirit.
Steve Dept commented that “it was thrilling to see all our long-time PISA friends (there are still several PISA National Project Managers who have worked on PISA from the outset, like cApStAn) and to make new friends. […] A big shout-out to the Australian Council for Educational Research for their remarkable work on preparing this event so efficiently”.
Photo collage, clockwise from top left:
Opening session with speech by the Portuguese Minister for Education, Mr João Costa; at the table, from left: Tue Halgreen, Senior Policy Analyst at OECD, Goran Lazendic, Research Director Systemwide Testing / Manager ACER, Australian Council for Educational Research; Catherine McClellan, Deputy Chief Executive Officer (Assessment) at ACER; Marc Oswald, Co-founder & CEO at Open Assessment Technologies, and Steve Dept, Founding partner of cApStAn Linguistic Quality Control; Group photo, from left to right: Manuel Souto Pico, Josiane Tyburn, Roberta Lizzi, Eli Krajčeva, Morgane Honkou, Irene Liberati, Anna Zemblicka, Adrien Mathot; plenary session; Steve Dept and Morgane Honkou, project manager; work session with cApStAn team members; Adrien Mathot, cApStAn Translation Technologist, evening out in Lisbon for the cApStAn team; work session, Grupo Iberoamericano de PISA; OmegaT help desk with Manuel Souto Pico, Lead Translation Technologist and Adrien Mathot.