A Comparison of OpenNMT Sequence Model for Indonesian Automatic Question Generation

Authors

Yuniar Indrihapsari, Information Technology, Universitas Negeri Yogyakarta, Indonesia
Handaru Jati, Informatics Engineering Education, Universitas Negeri Yogyakarta, Indonesia
N. Nurkhamid, Information Technology, Universitas Negeri Yogyakarta, Indonesia
Ratna Wardani, Informatics Engineering Education, Universitas Negeri Yogyakarta, Indonesia
Pradana Setialana, Information Technology, Universitas Negeri Yogyakarta, Indonesia
Muhamad Izzudin Mahali, Electronic and Computer Engineering, National Taiwan University of Science and Technology, Taiwan, Province of China
Danang Wijaya, Informatics Engineering Education, Universitas Negeri Yogyakarta, Indonesia
Dhista Dwi Nur Ardiansyah, Informatics Engineering Education, Universitas Negeri Yogyakarta, Indonesia
Satya Adhiyaksa Ardy, Information Technology, Universitas Negeri Yogyakarta, Indonesia
Maria Bernadetha Charlotta Wonda Tiala, Information Technology, Universitas Negeri Yogyakarta, Indonesia
Andi Hakim Al-khawarizmi, Information Technology, Universitas Negeri Yogyakarta, Indonesia
Widya Ardiyanto, Information Technology, Universitas Negeri Yogyakarta, Indonesia

Keywords

openNMT, SQuAD 2.0, Indonesian automatic question generator, evaluation process

Document Type

Article

Abstract

Evaluation of learners is a crucial aspect of the educational system. However, creating evaluation instruments is a process that demands teachers' time and energy. The researcher developed the Indonesia Automatic Question Generator in this study using an architecture modified from past studies. The primary goals of this project are (1) to construct an AQG tool utilizing the OpenNMT series and (2) to analyze and compare the model's performance. As a data source, this study employs the SQuAD 2.0 dataset and numerous sequence techniques, including BiGRU, BiLSTM, and Transformer. The researcher trained the models using OpenNMT-py and Google Collaboratory. This approach generates questions that are relevant to the context of the source. This study found that the model was acceptable.

First Page

55

Last Page

63

Page Range

55-63

Issue

1

Volume

8

Digital Object Identifier (DOI)

10.21831/elinvo.v8i1.56491

Source

https://journal.uny.ac.id/index.php/elinvo/article/view/56491

Recommended Citation

Y. Indrihapsari and H. Jati and N. Nurkhamid and R. Wardani and P. Setialana and M. I. Mahali and D. Wijaya and D. D. Ardiansyah and S. A. Ardy and M. B. Tiala and A. H. Al-khawarizmi and W. Ardiyanto, "A Comparison of OpenNMT Sequence Model for Indonesian Automatic Question Generation,", vol. 8, no. 1, pp. 55 - 63, Dec 2023.

The definitive version is available at https://doi.org/10.21831/elinvo.v8i1.56491

References

W. Thalheimer, "The learning benefits of questions," 2003. http://www.worklearning.com/ma/PP_WP003.asp (accessed Aug. 31, 2007).

P. K. M. Debajit Datta, Rishav Agarwal, Ishita Tuteja, V Bhavyashri Vedula, "Optimization of an Automated Examination Generation System Using Hybrid Recurrent Neural Network," International Journal of Interdisciplinary Global Studies, vol. 14, no. 04, 2020.

H. R. S. Flores Veronica Ambassador, Jasa Lie, "Pembangkit Pertanyaan Otomatis pada Materi Pelajaran IPA Berbahasa Indonesia di Tingkat SD Berdasarkan Revisi Taksonomi Bloom," Majalah Ilmiah Teknologi Elektro, vol. 20, no. 2.

S. Rakangor and Y. R. Ghodasara, "Literature Review of Automatic Question Generation Systems," International Journal of Scientific and Research Publications, vol. 5, no. 1, pp. 2250-3153, 2015.

M. Divate and A. Salgaonkar, "Automatic question generation approaches and evaluation techniques," Current Science, vol. 113, no. 9, pp. 1683-1691, 2017, doi: 10.18520/cs/v113/i09/1683-1691.

V. Rus, B. Wyse, P. Pivek, M. Lintean, S. Stoyanchev, and C. Moldovan, "The first question generation shared task evaluation challenge," INLG 2010 - Proceedings of the 6th International Natural Language Generation Conference, pp. 251-254, 2010.

S. Panda and S. Garg, Open and Distance Education in Asia, Middle East Africa and the National Perspectives in a Digital Age. 2019. doi: 10.1007/978-981-13-5787-9_4.

M. Divate et al., E-Learning in European higher education institutions: Results of a mapping survey conducted in october-december 2013, vol. 113, no. December. 2014. doi: 10.1007/978-981-13-5787-9_4.

I. R. Goldbach and F. G. Hamza-Lup, "Survey on e-Learning Implementation in Eastern-Europe Spotlight on Romania," International Conference on Mobile, Hybrid, and On-line Learning, vol. 9, no. February, pp. 5-12, 2017.

X. Yuan et al., "Machine Comprehension by Text-to-Text Neural Question Generation." arXiv, May 15, 2017. Accessed: Jun. 05, 2023. [Online]. Available: http://arxiv.org/abs/1705.02012

S. Sharma, L. E. Asri, H. Schulz, and J. Zumer, "Relevance of Unsupervised Metrics in Task-Oriented Dialogue for Evaluating Natural Language Generation," arXiv:1706.09799 [cs], Jun. 2017, Accessed: Mar. 13, 2022. [Online]. Available: http://arxiv.org/abs/1706.09799

P. Rajpurkar, R. Jia, and P. Liang, "Know What You Don't Know: Unanswerable Questions for SQuAD," arXiv:1806.03822 [cs], Jun. 2018, Accessed: Feb. 22, 2022. [Online]. Available: http://arxiv.org/abs/1806.03822

M. Schuster and K. K. Paliwal, "Bidirectional recurrent neural networks," IEEE Transactions on Signal Processing, vol. 45, no. 11, pp. 2673-2681, 1997, doi: 10.1109/78.650093.

F. J. Muis and A. Purwarianti, "Sequence-to-Sequence Learning for Indonesian Automatic Question Generator," in 2020 7th International Conference on Advance Informatics: Concepts, Theory and Applications (ICAICTA), Tokoname, Japan: IEEE, Sep. 2020, pp. 1-6. doi: 10.1109/ICAICTA49861.2020.9429032.