Gamified Crowd-sourcing for Word Sense Disambiguation of Turkish

Abstract

Word sense disambiguation (WSD) is the process of determining the correct meaning of a word based on its context in a sentence, a task that remains one of the core challenges in natural language processing (NLP) despite the advancements made by LLMs. Although LLMs have improved WSD performance, challenges remain–especially in nuanced contexts, low-resource languages, and domain-specific terminology. These gaps can hinder the accuracy of LLMs in high-stakes applications and multilingual settings. In response to the need for WSD, our study introduces a novel approach to WSD data collection through gamified crowdsourcing, which, to our knowledge, has not been previously applied in this field. A messaging bot method has been used to engage a wide, diverse audience to generate high-quality WSD data. Using a multiplayer game format, native speakers provide examples for different senses of ambiguous words and rate others’ contributions. Together with the suggested enhancements, this platform attracted a diverse range of participants–spanning ages, backgrounds, and genders–20 times more than a similar crowdsourcing method applied to a different NLP task, and sustained their engagement over an extended period. Unlike conventional academic crowdsourcing pools, this approach focuses on drawing individuals from varied backgrounds beyond academia or AI-focused communities. It not only gathered data but also encouraged participants to evaluate each other’s contributions, creating a rich and reliable dataset. Our findings suggest that gamified crowdsourcing can be a powerful tool for creating WSD corpora. Our approach not only supports future WSD research but also contributes valuable training data for developing more precise and resilient language models.

Citing

Please use the following .bib entry to cite this article:

@article{10.1145/3767160,
  author = {Toruno{\u g}lu Selamet, Dilara and {\c S}enta{\c s}, Ali and Eryi{\u g}it, G{\"u}l{\c s}en},
  title = {Gamified Crowd-sourcing for Word Sense Disambiguation of Turkish},
  year = {2025},
  issue_date = {November 2025},
  publisher = {Association for Computing Machinery},
  address = {New York, NY, USA},
  volume = {24},
  number = {11},
  issn = {2375-4699},
  url = {https://doi.org/10.1145/3767160},
  doi = {10.1145/3767160},
  journal = {ACM Trans. Asian Low-Resour. Lang. Inf. Process.},
  month = nov,
  articleno = {130},
  numpages = {23},
  keywords = {Crowdsourcing, gamification, game with a purpose (GWAP), word sense disambiguation, language resources}
}