DIET Classifier Model Analysis for Words Prediction in Academic Chatbot

Abstract

One prevalent conversational system within the realm of natural language processing (NLP) is chatbots, designed to facilitate interactions between humans and machines. This study focuses on predicting frequently asked questions by students using the Duel Intent and Entity Transformer (DIET) Classifier method and assessing the performance of this method. The research involves employing 300 epochs with an 80% training data and 20% testing data split. In this study, the DIET Classifier adopts a multi-task transformer architecture to simultaneously handle classification and entity recognition tasks. Notably, it possesses the capability to integrate diverse word embeddings, such as BERT and GloVe, or pre-trained words from language models, and blend them with sparse words and n-gram character-level features in a plug-and-play manner. Throughout the training process of the DIET Classifier model, data loss and accuracy from both training and testing datasets are monitored at each epoch. The evaluation of the text classification model utilizes a confusion matrix. The accuracy results for testing the DIET Classifier method are presented through four case studies, each comprising 25 text messages and 15 corresponding chatbot responses. The obtained accuracy values range from 0.488 to 0.551, F1-Score values range from 0.427 to 0.463, and precision range from 0.417 to 0.457.