Automatic Assessment of University Teachers’ Critical Thinking Levels

The present work describes the structure of a pilot study which was addressed to test a tool developed to automatically assess critical thinking - CT levels through language analysis techniques. Starting from a Wikipedia data- base and lexical analysis procedures based on n-grams, a new approach aimed at the automatic assessment of the open-ended questions, where CT can be detected, is proposed. Automatic assessment is focused on four CT macro-indicators: basic language skills, relevance, importance and novelty. The pilot study was carried out through different workshops adapted from Crithinkedu EU Erasmus + Pro- ject model aimed at training university teachers in the field of CT. The workshops were designed to support the development of CT teaching practices at higher ed- ucation levels and enhance University Teachers’ CT as well. The two-hour work- shops were conducted in two higher educational institutions, the first in the U.S.A (CCRWT Berkeley College NYC, 26 university teachers) and the second in Italy (Inclusive memory project - University Roma Tre, 22 university teachers). After the two workshops, data were collected through an online questionnaire devel- oped and adapted in the framework of the Erasmus + Crithinkedu project. The questionnaire includes both open-ended and multiple-choice questions. The re- sults present CT levels shown by university teachers and which kind of pedagog- ical practices they intend to promote after such an experience within their courses. In addition, a comparison between the values inferred by the algorithm and those calculated by domain human experts is offered. Finally, follow-up ac- tivity is shown taking into consideration other sets of macro-indicators: argumen- tation and critical evaluation.