31.07.2023

cApStAn LQC Founding Partner Steve Dept and cApStAn Inc Managing Director Musab Hayatli at ESRA Conference in July

Published in:  cApStAn updates, Conferences, Neural Machine Translation, Survey translation and localisation, Surveys and Opinion Polls, Talent Management, Tests and Assessments, Translation industry

cApStAn Founding Partner Steve Dept, from the company’s headquarters in Belgium, and Musab Hayatli, Managing Director of cApStAn Inc, USA are just back from the Biennial ESRA Conference in Milan (July 17-21, 2023) where they were both presenters. As Steve comments: “An impressive conference venue on the Bicocca university campus, insightful sessions, great connections made. Milano’s beauty added to the experience, not to mention indulging in delectable Italian cuisine and the wonderful Italians we met!”

Steve adds that the conference provided a rare opportunity to meet up with partners, colleagues, researchers and co-authors and discuss the joint publications they are working on in collaboration with cApStAn. Close interaction with academia, he stresses, has always put cApStAn apart: this form of cross-pollination provides a robust theoretical underpinning to the wealth of empirical research we conduct in the production environment of surveys in multinational, multiregional and multicultural contexts (3MC Surveys).

Steve Dept

Steve’s talk was about cApStAn’s leadership in the “Rapid Double Translation of the OECD – OCDE PISA Global Crises Module.” This module aimed to collect data on how education systems worldwide responded to the COVID-19 pandemic. cApStAn produced 112 language versions for 87 countries/ economies (PISA Participants). It conducted upstream quality assurance work, including optimizing the source text through a translatability assessment. Participants received preview access to the translations and could suggest edits.

See also our “Case study on the Translation/adaptation of PISA Global Crises Module (GCM)” here

Musab Hayatli

Musab Hayatli was the presenting author for a paper titled “Bipolar and unipolar response scales in translated surveys: the importance of source analysis for multilingual surveys”, prepared together with Dr Dorothée Behr from GESIS – Leibniz Institute for the Social Sciences. Musab and Dorothée explored the impact of scale polarity and language on survey data quality, providing examples of semantic differences in scale options between English and translated languages. The presentation aimed to contribute to the ongoing debate and raise awareness of the influence of translated surveys on outcomes and survey design.

Highlights from sessions attended on the topic of response scale orientation

  • Does a change in orientation of response scales affect the results of life satisfaction measures? One of the sessions, featuring Ms Fabienne Wöhner (University of Bern) as one of the speakers, looked at the reliability and validity of life satisfaction items regarding the positive pole location and the display orientation (horizontal vs. vertical) of the response scale. Data from a university-wide online student survey at the University of Bern in Switzerland (N = 3150) showed that reports of average life satisfaction differs according to scale direction. No differences were found concerning horizontal and vertical orientation.
  • How does scale ordering affect responses? Is there a norm in scale ordering? Do respondents expect “up” and “left” to mean “good”? Ms Freja Wessman from the SOM Institutet presented the results of study investigating how respondents interpret scale ordering. The answer is yes!
  • Dr Femke De Keulenaer from IPSOS drew from the profiling questions in the IPSOS Knowledge Panel to discusse overall question order bias in an online survey setting. Emphasis was placed on which survey topic and types of survey questions show the largest bias. The impact of cultural and language were assessed and comparisons made across socio-demographic and attitudinal characteristics.
  • Mario Callegaro and Yongwei Yang from Google dug into effects of different response scales and how they are presented, e.g. when “dissatisfied” comes 1st, a high proportion of respondents click that option although they intended to say they were very satisfied! An interesting finding.
  • Effects of switching from DFRDD (Dual-frame, random-digit-dialling) to web-based surveys: Patrick Moynihan from ESRA analysed the consequences of the changing survey landscape and in particular if web-based probability panels yield comparable attitudinal measures to DFRDD samples. If not what are the implications of these differences?

Photo captions

From top left: Patrick Moynihan from ESRA, Steve Dept from cApStAn, session chaired by Mario Callegaro & Yongwei Yang from Google, Prof. Randall K. Thomas from Ipsos, cApStAn team members on a night out in Milano, Ms Fabienne Wöhner (University of Bern), the Aula Magna.