In this paper, we explore and compare the performance of three conversational chatbots based on large language models, focusing on solutions that are entirely open-source and accessible. The models were tested under realistic computational constraints using Google Colab’s free environment. The evaluation covered two tasks: closed-domain question answering (based solely on internal model knowledge) and standard question answering (with external context provided). Both technical performance and user feedback were considered. The technical assessment included over 250 questions, measuring accuracy, speed, informativeness, and language quality. A user survey involving some participants supplemented these findings.

Sansonetti, G., Micarelli, A. (2026). A Comparative Analysis of Open-Source Conversational Chatbots in Resource-Constrained Environments. In Communications in Computer and Information Science (pp.393-401). Cham : Springer Science and Business Media Deutschland GmbH [10.1007/978-3-032-12767-9_41].

A Comparative Analysis of Open-Source Conversational Chatbots in Resource-Constrained Environments

Sansonetti, Giuseppe
;
Micarelli, Alessandro
2026-01-01

Abstract

In this paper, we explore and compare the performance of three conversational chatbots based on large language models, focusing on solutions that are entirely open-source and accessible. The models were tested under realistic computational constraints using Google Colab’s free environment. The evaluation covered two tasks: closed-domain question answering (based solely on internal model knowledge) and standard question answering (with external context provided). Both technical performance and user feedback were considered. The technical assessment included over 250 questions, measuring accuracy, speed, informativeness, and language quality. A user survey involving some participants supplemented these findings.
2026
9783032127662
Sansonetti, G., Micarelli, A. (2026). A Comparative Analysis of Open-Source Conversational Chatbots in Resource-Constrained Environments. In Communications in Computer and Information Science (pp.393-401). Cham : Springer Science and Business Media Deutschland GmbH [10.1007/978-3-032-12767-9_41].
File in questo prodotto:
Non ci sono file associati a questo prodotto.

I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.

Utilizza questo identificativo per citare o creare un link a questo documento: https://hdl.handle.net/11590/536416
Citazioni
  • ???jsp.display-item.citation.pmc??? ND
  • Scopus 0
  • ???jsp.display-item.citation.isi??? ND
social impact