Testing subject matter, not English proficiency

Requirement:

One of our clients develops skills assessments globally for higher education students taking campus recruitment assessments, entry level engineers, or anyone working in their field of interest. Our client required our assistance in tackling a particular concern of theirs, namely that that the level of English proficiency required to successfully take these assessments may blur the measurement of the required skills and the subject due to possible test challenging if their level of English is insufficient.

Diagnosis:

Our client was interested in assessing what level of reading comprehension in English (e.g., classification according to the Common European Framework of Reference for Languages) is required to take the existing tests; as well as to determine a reading comprehension threshold that strikes a good balance between accessibility for a global for a global audience and robust test items. This was to be achieved by reviewing existing and future task descriptions to stay as close as possible to that threshold.

Approach:

Text Analysis Results per Unit: Here we focus on word choice, and sentence length. Since word and sentence length can be observed from reviewing the source material itself, it doesn’t require extra references beyond the text analysis report calling attention to which units have longer sentences or words. Word choice, however, will require extra references to help the reviewer focus their attention to particularly challenging words per the European Common Framework.

Proposal:

The reviewer will be focusing their edits on two main points: word and sentence length, and word choice. Since word and sentence length can be observed from reviewing the source material itself, it doesn’t require extra references beyond the text analysis report calling attention to which units have longer sentences or words. Word choice, however, will require extra references to help the reviewer focus their attention to particularly challenging words per the European Common Framework.

Want to try this out on your materials?

Sign a mutual NDA and send your content along with the requirements. Request a quote at bizdev@capstan.be or contact us via the form below for more details.

See also, from our blog

 “ Disentangling Proficiency in Programming from Proficiency in English: Codility meets cApStAn ”