cApStAn at the E-ATP 2017 Conference in Noordwijk
The venue for the 2017 edition of the European Chapter of the Association of Test Publishers was again a rather classy affair, a five-star hotel with a panoramic view of the North Sea, some 40 km out of Amsterdam. The weather conditions were whimsical and the sea was unruly, so the 365 attendees stayed in and everyone attended as many sessions as they could, starting with the opening keynote on September 27. The stately halls were bustling with energy, and industry leaders mixed with start-ups, test delivery platform architects talked with psychometricians, passionate visionaries talked to seasoned businessmen.
The conference theme was “Advancing Human Capital with Assessment”. I’ll translate this for you: “how to use tests to spot talent in the 2st century”. Why was it relevant for cApStAn? Does cApStAn want to spot more talent than it already has? As a matter of fact, we do, and we use tests for that, too: usually in the form of translated assessment items in which we have planted a variety of errors. We give the testee a set of categories in which to classify the errors they detect, and they are asked to identify and describe the issue, then propose corrective action. We discuss the output with the candidates and—Oh, I did get carried away and drifted away from the subject, sorry about that. No, no, this conference was relevant for cApStAn because multilingual testing has become widespread; because translation quality is crucial in high stakes test; and because cApStAn’s niche is precisely cross-linguistic and cross-cultural comparability of test and questionnaires.
So Devasmita and Steve spent three days learning as much as they could take in about the latest developments in the testing industry. Let me list some, in no particular order: adaptive testing, and the possibilities this opens for an adaptive learning process, whereby the student’s level, speed and item type drive the learning process; harnessing the power of augmented reality and gaming in assessment, so that testing becomes more engaging (the challenges for localisation don’t go away in virtual reality, I can already confirm); possible applications of the blockchain in testing (those who read Steve’s posts in LinkedIn know that he has been following this closely); screening candidates through (partially automated) evaluation of preselected items; remote online proctoring. New ways to test soft skills or situational judgment; using data science to analyse results of technology-enhanced items. We were particularly impressed by the number of relevant questions asked at the end of each session and how well the presenters responded.
Besides sponsoring the opening reception, our own contribution was an ignite presentation: Steve presented 20 slides in 5 minutes to whet attendees’ appetite for knowledge on interaction between machine learning and human discernment. Fifteen seconds per slide, no clicker, just enough time to let the full house catch a glimpse of a new paradigm for test translation and adaptation: more work is done up front so that systems can be trained with good data rather than lots of data. Tedious, repetitive work is taken over by algorithms, and the linguists’ expertise comes in at very specific points in the man-machine workflow. After that presentation, over twenty different people came to Steve and Devasmita with questions, and we had very lively coffee breaks and lunches. No time to put on other weight than that of the business cards we received.
The balance between business-oriented and knowledge-sharing activities is very good. Everyone is aware of the importance of networking, so when people start a conversation they tend to go straight to the point. The listening quality is as good as the pitches, and cApStAn will definitely become even more involved in ATP, E-ATP and I-ATP in the near future.