The present work describes the structure of a pilot study which was addressed to test a tool developed to automatically assess critical thinking - CT levels through language analysis techniques. Starting from a Wikipedia data- base and lexical analysis procedures based on n-grams, a new approach aimed at the automatic assessment of the open-ended questions, where CT can be detected, is proposed. Automatic assessment is focused on four CT macro-indicators: basic language skills, relevance, importance and novelty. The pilot study was carried out through different workshops adapted from Crithinkedu EU Erasmus + Pro- ject model aimed at training university teachers in the field of CT. The workshops were designed to support the development of CT teaching practices at higher ed- ucation levels and enhance University Teachers’ CT as well. The two-hour work- shops were conducted in two higher educational institutions, the first in the U.S.A (CCRWT Berkeley College NYC, 26 university teachers) and the second in Italy (Inclusive memory project - University Roma Tre, 22 university teachers). After the two workshops, data were collected through an online questionnaire devel- oped and adapted in the framework of the Erasmus + Crithinkedu project. The questionnaire includes both open-ended and multiple-choice questions. The re- sults present CT levels shown by university teachers and which kind of pedagog- ical practices they intend to promote after such an experience within their courses. In addition, a comparison between the values inferred by the algorithm and those calculated by domain human experts is offered. Finally, follow-up ac- tivity is shown taking into consideration other sets of macro-indicators: argumen- tation and critical evaluation.

Poce, A., Amenduni, F., Re, M.R., DE MEDIO, C. (2019). Automatic Assessment of University Teachers’ Critical Thinking Levels. INTERNATIONAL JOURNAL: ADVANCED CORPORATE LEARNING., 12(3), 46-58 [10.3991/ijac.v12i3.11259].

Automatic Assessment of University Teachers’ Critical Thinking Levels

Antonella Poce
;
Francesca Amenduni;Maria Rosaria Re;Carlo De Medio
2019-01-01

Abstract

The present work describes the structure of a pilot study which was addressed to test a tool developed to automatically assess critical thinking - CT levels through language analysis techniques. Starting from a Wikipedia data- base and lexical analysis procedures based on n-grams, a new approach aimed at the automatic assessment of the open-ended questions, where CT can be detected, is proposed. Automatic assessment is focused on four CT macro-indicators: basic language skills, relevance, importance and novelty. The pilot study was carried out through different workshops adapted from Crithinkedu EU Erasmus + Pro- ject model aimed at training university teachers in the field of CT. The workshops were designed to support the development of CT teaching practices at higher ed- ucation levels and enhance University Teachers’ CT as well. The two-hour work- shops were conducted in two higher educational institutions, the first in the U.S.A (CCRWT Berkeley College NYC, 26 university teachers) and the second in Italy (Inclusive memory project - University Roma Tre, 22 university teachers). After the two workshops, data were collected through an online questionnaire devel- oped and adapted in the framework of the Erasmus + Crithinkedu project. The questionnaire includes both open-ended and multiple-choice questions. The re- sults present CT levels shown by university teachers and which kind of pedagog- ical practices they intend to promote after such an experience within their courses. In addition, a comparison between the values inferred by the algorithm and those calculated by domain human experts is offered. Finally, follow-up ac- tivity is shown taking into consideration other sets of macro-indicators: argumen- tation and critical evaluation.
2019
Poce, A., Amenduni, F., Re, M.R., DE MEDIO, C. (2019). Automatic Assessment of University Teachers’ Critical Thinking Levels. INTERNATIONAL JOURNAL: ADVANCED CORPORATE LEARNING., 12(3), 46-58 [10.3991/ijac.v12i3.11259].
File in questo prodotto:
Non ci sono file associati a questo prodotto.

I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.

Utilizza questo identificativo per citare o creare un link a questo documento: https://hdl.handle.net/11590/368332
Citazioni
  • ???jsp.display-item.citation.pmc??? ND
  • Scopus ND
  • ???jsp.display-item.citation.isi??? ND
social impact