Chatbot Development for an Interactive Academic Information Services using the Rasa Open Source Framework
DOI:
https://doi.org/10.35793/jtek.v10i1.31150Keywords:
Layanan Informasi Akademik, Chatbot, Framework, NLU, Rasa Open SourceAbstract
Abstract — Academic administration information services that are spread through various online media are often not missed by students. Students prefer to ask directly to the manager of the study program using a chat application. However, the lack of human resources on the information provider side means that information services through chat applications cannot be provided optimally. In this study, a chatbot was developed to autonomously serve requests for information from users. The chatbot is developed using the Rasa Open Source framework. This study uses student's Frequently Asked Questions data to the study program as initial training data for chatbots. The data was developed into Natural Language Understanding (NLU) training data and dialogue training data. The number of sentence samples for NLU training was 188 sample sentences and for dialogue training was 31 dialogue samples. The results of the NLU evaluation show that chatbots can understand well the meaning of text messages from users, indicated by the weighted average value for precision 0.995, recall 0.995 and F1-Score 0.995. Meanwhile, the dialogue model evaluation gets an accuracy level of 0.70, a precision value of 0.72 and an F1-score of 0.70 which represent the results of the evaluation of the performance level of the chatbot in predicting the right response for the user.
Â
Keywords — Academic Information Service; Chatbot; Framework; NLU; Rasa Open Source.
Layanan informasi administrasi akademik yang tersebar melalui berbagai macam media online seringkali tidak dilewati oleh mahasiswa. Seringkali mahasiswa lebih memilih bertanya langsung ke pengelola program studi menggunakan aplikasi chat. Namun kurangnya sumber daya manusia di sisi penyedia informasi membuat pelayanan informasi melalui aplikasi chat tidak dapat diberikan dengan maksimal. Pada penelitian ini, dikembangkan sebuah chatbot untuk melayani permintaan informasi dari pengguna secara otonom. Chatbot dikembangkan menggunakan framework Rasa Open Source. Penelitian ini menggunakan data Frequently Asked Questions mahasiswa ke pengelola program studi sebagai data pelatihan awal untuk chatbot. Data tersebut dikembangkan menjadi data pelatihan Natural Language Understanding (NLU) dan data pelatihan dialog. Jumlah sampel kalimat untuk pelatihan NLU sebanyak 188 sampel kalimat dan untuk pelatihan dialog adalah 31 dialog. Hasil evaluasi NLU memperlihatkan chatbot dapat memahami dengan baik maksud pesan teks pengguna, ditunjukkan dengan nilai rata-rata tertimbang untuk precision 0,995, recall 0,995 dan F1-Score 0,995. Sementara untuk evaluasi model dialog mendapatkan tingkat akurasi 0.70, nilai presisi 0,72 dan F1-Score 0,70 yang merepresentasikan hasil evaluasi performansi chatbot dalam memprediksi respon yang tepat untuk pengguna.
References
Direktorat Jenderal Sumber Daya IPTEK dan Pendidikan Tinggi Kementrian Riset Teknologi dan Pendidikan Tinggi, Pedoman Pemilihan Administrasi Akademik Berprestasi Tahun 2017. Kementrian Riset Teknologi dan Pendidikan Tinggi, 2017.
S. Raj, Building Chatbots with Python. 2019.
J. Vogel, Chatbots: Development and Application. Berlin: University of Applied Sciences, 2017.
N. P. Patel, D. R. Parikh, D. A. Patel, and R. R. Patel, “AI and Web-Based Human-Like Interactive University Chatbot (UNIBOT),†Sep. 2019, pp. 148–150, doi: 10.1109/iceca.2019.8822176.
M. Muslih et al., “Developing smart workspace based iot with artificial intelligence using telegram chatbot,†in Proceedings - 2018 4th International Conference on Computing, Engineering, and Design, ICCED 2018, 2019, doi: 10.1109/ICCED.2018.00052.
T. Beysolow II, Applied Natural Language Processing with Python. 2018.
K. Nimavat and P. T. Champaneria, “Chatbots: An Overview Types, Architecture, Tools and Future Possibilities,†Oct. 2017. Accessed: Oct. 21, 2020. [Online]. Available: www.ijsrd.com.
G. L. Ward and G. M. Green, “Pragmatics and Natural Language Understanding,†Language (Baltim)., 1991, doi: 10.2307/415112.
H. Lane, H. Cole, and H. M. Hapke, “Natural Language Processing in Action MEAP Edition,†Manning Publ., 2018, doi: 10.1310/sci2102-166.
R. C. Schank, “Conceptual dependency: A theory of natural language understanding,†Cogn. Psychol., 1972, doi: 10.1016/0010-0285(72)90022-9.
A. Jiao, “An Intelligent Chatbot System Based on Entity Extraction Using RASA NLU and Neural Network,†J. Phys, p. 12014, 2020, doi: 10.1088/1742-6596/1487/1/012014.
D. Braun, A. Hernandez-Mendez, F. Matthes, and M. Langen, “Evaluating Natural Language Understanding Services for Conversational Question Answering Systems,†2018, doi: 10.18653/v1/w17-5522.
“Rasa Architecture Overview.†https://rasa.com/docs/rasa/arch-overview (accessed Oct. 22, 2020).
T. Bocklisch, J. Faulkner, N. Pawlowski, and A. Nichol, “Rasa: Open Source Language Understanding and Dialogue Management,†Dec. 2017, Accessed: May 14, 2020. [Online]. Available: http://arxiv.org/abs/1712.05181.
S. Singh, B. R. Ranoliya, and N. Raghuwanshi, “Chatbot for university related FAQs Differential Privacy View project Social Network Analysis View project Chatbot for University Related FAQs,†ieeexplore.ieee.org, pp. 1525–1530, 2017, doi: 10.1109/ICACCI.2017.8126057.
S. A. and D. John, “Survey on Chatbot Design Techniques in Speech Conversation Systems,†Int. J. Adv. Comput. Sci. Appl., vol. 6, no. 7, 2015, doi: 10.14569/ijacsa.2015.060712.
“Connecting to Messaging and Voice Channels.†https://rasa.com/docs/rasa/messaging-and-voice-channels/ (accessed Oct. 25, 2020).
K. Kuligowska, “Commercial Chatbot: Performance Evaluation, Usability Metrics and Quality Standards of Embodied Conversational Agents,†Prof. Cent. Bus. Res., 2015, doi: 10.18483/pcbr.22.
Downloads
Additional Files
Published
Issue
Section
License
Jurnal Teknik Elektro dan Komputer (JTEK) is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
