Great to be present on site at the ATP Innovations in Testing conference, the world’s leading event in the testing industry
by Pisana Ferrari – cApStAn Ambassador to the Global Village
Steve Dept, cApStAn Founding Partner, and Musab Hayatli, Managing Director of the Philadelphia office, were thrilled to be in Orlando at this year’s ATP Innovations in Testing conference, the world’s leading event in the testing industry. The event had over 1,200 delegates combined between in-person and virtual attendees! In this hybrid conference, the desire to reconnect, share news, and to be part of the dynamics at play was palpable. Colleagues from the industry were eager to make contact – at the same time, social distancing recommendations and wearing masks made some encounters awkward at first, though this was easily overcome. It was great to catch up with old friends and colleagues and network with other experts in the field, after the long Covid pause. Steve and Musab were delighted to intervene as speakers at three different sessions in which the spotlight was on fairness and validity in both monolingual and multilingual tests and on why test design needs to include Diversity, Equity, Inclusion and Bias Reduction (DEI-BR) considerations.
Looking at a testing project through a cultural lens
Steve and Musab’s first presentation on March 22 was in the context of a session titled: “You Can Do Better Than Comply with the Standards: A Collaborative Cross-check of your DEI-BR Processes”. The other speakers were Dr. Christine Mills, Director of Research and Applied Psychometrics, and Kelly Von Lunen, Managing Editor of Learning Products, from Ascend. In this session Ascend Learning talked about their initiative to set up a collaborative cross-check of their existing DEI-BR processes with cApStAn. The presentation was aimed at promoting the development and deployment of more effective and proactive DEI-BR processes within the testing community. Why is this important? The extent to which organizations address DEI-BR in education and credentialing bears directly on the cultural competence students will implement as they transition to be professionals. This is essential for them in order to be able to address the needs of all clients they will encounter in their professional life.
The importance of “fairness” in high stakes tests
Joy L. Matthews-López, President of JML Measurement & Testing Services, LLC, and Steve moderated a “Peas in a Pod” session titled “Fairness in Multilingual Assessments – Leaning on the Standards, and Taking Necessary Trade-Offs in Stride”. “Fairness”, the presenters explained, is a validity issue that touches all aspects of test development, delivery, and maintenance. Each assessment component must be fair: test design, constructs, items/content, policies that guide test administration, and scoring/reporting. Does anything in the source or in (one of) the target language(s) put examinees at an advantage or at a disadvantage? What is the intended use of assessment outcomes (resulting scores), and are these equally interpretable for source and target populations? These were some of the issues raised in this well-attended session. The presenters referred to case studies from certification exams and large-scale international assessments to illustrate good practice, and the attendees shared the challenges they are or have been facing.
DEI-BR in multilingual and multicultural contexts
“Navigating Diversity, Equity and Inclusion in Multilingual and Multicultural Contexts” was the title of Steve’s “Peas in a Pod” session on March 23rd. He explained that cultures and societies are dynamic and so is awareness of social justice issues (#BlackLivesMatter, #MeToo, broader acceptance of non-binary gender identity). Theoretical and practical approaches to fairness in testing need to evolve to take into account societal changes. DEI-BR has become a crucial component in this regard. He said many conference delegates may have had to deal with this the fast-changing testing landscape of the US and Canada when candidates or respondents come from very diverse backgrounds and when the test program needs to be administered in multiple languages and regions. Steve presented examples of content at risk of being at odds with cultural sensitivity in certain contexts, and the audience promptly chimed in with examples from their own certification programmes. A lively discussion ensued, and no-one seemed to want to end the session for the coffee break.
Photo
Highlights from ATP 2022, including Steve Dept together with Ian Castledine, Head of Proposition-High Stakes Assessment at RM Results (top, middle), with William G. Harris aka “G”, CEO of ATP (bottom, left) and Simmy Ziv-el, Chief Strategy and Business Development Officer at Fine Tune (bottom, right). Photo collage by Graphillus.