Skip to main content
Article
Cloze Evaluation for Deeper Understanding of Commonsense Stories in Indonesian
CSRR 2022 - 1st Workshop on Commonsense Representation and Reasoning
  • Fajri Koto, The University of Melbourne, Australia
  • Timothy Baldwin, The University of Melbourne, Australia & Mohamed bin Zayed University of Artificial Intelligence
  • Jey Han Lau, The University of Melbourne, Australia
Document Type
Conference Proceeding
Abstract

Story comprehension that involves complex causal and temporal relations is a critical task in NLP, but previous studies have focused predominantly on English, leaving open the question of how the findings generalize to other languages, such as Indonesian. In this paper, we follow the Story Cloze Test framework of Mostafazadeh et al. (2016) in evaluating story understanding in Indonesian, by constructing a four-sentence story with one correct ending and one incorrect ending. To investigate commonsense knowledge acquisition in language models, we experimented with: (1) a classification task to predict the correct ending; and (2) a generation task to complete the story with a single sentence. We investigate these tasks in two settings: (i) monolingual training and (ii) zero-shot cross-lingual transfer between Indonesian and English. © 2022 Association for Computational Linguistics.

DOI
10.18653/v1/2022.csrr-1.2
Publication Date
5-1-2022
Keywords
  • Classification (of information),
  • Computational linguistics,
  • Natural language processing systems
Comments

IR Deposit conditions: non-described

Citation Information
F. Koto, T. Baldwin, and J.H. Lau, "Cloze Evaluation for Deeper Understanding of Commonsense Stories in Indonesian", in Proceedings of the First Workshop on Commonsense Representation and Reasoning (CSRR 2022), May 2022, pp. 8–16, doi:10.18653/v1/2022.csrr-1.2