Workshop on NLP Applications to Field Linguistics Call for Papers 2027 Shared Task Field Matters 2027 Field Matters 2026 Field Matters 2025 Field Matters 2024 Field Matters 2023 Field Matters 2022
All deadlines are 23:59 UTC-12 (‘Anywhere on Earth’).
This year, we offer a shared task on automatic interlinear glossing for field linguistic annotations.
| source sentence | Zbz | sc | tqwaipele | tqnginipenge | kraesz'mz |
|---|---|---|---|---|---|
| morphological segmentation | zbz | sc | tq-wai-pe-le | tq-ngini-pe-nge | kraesz'mz |
| interlinear gloss | so.then | PFV | RL-do-COS-3MINIA | RL-become-COS-1MINII | sixth |
| translation | So then that's how I became the sixth. | ||||
Figure 1: An Interlinear Glossed Text (IGT) for a sentence in Natugu.
The task is to predict interlinear glosses, which are morpheme-level linguistic annotations, as shown in the third row of the figure above, notably used in field linguistics. Ultimately, such glossing systems aim to help field linguists with the annotation task for languages under documentation, as it is mainly performed manually with little automation.
This Shared Task follows the SIGMORPHON 2023 Shared Task on Interlinear Glossing.
In our Shared Task, the input is mainly the unsegmented source sentence (first row). Some instances will be accompanied by its translation in a high-resource language (usually, in English; fourth row).
The goal is to predict the glossed sentence (third row).
The Shared Task will be in three phases:
The task will cover the following languages:
The corresponding citations for the languages are detailed in a section below.
Submissions to the shared task should follow the rules below:
existing_set (from phase 1; SIGMORPHON languages + Ainu) andnew_set (from phase 2; new languages).Participation can be in only one of the two sets, or in both. However, in any case, a submission should contain predictions for all languages of a set (5 languages for existing_set and X languages for new_set).
Since the test sets of the existing_set are available online, we kindly ask participants not to use them for training
In case of doubt about a rule, please feel free to check with us.
In phase 1, the datasets to consider will be the training and dev sets. Although the test sets for this set of languages are publicly available, they should NOT be used to train the system.
The goal of this set is to have a common ground between submitted systems and previous work.
In phase 2, we will mainly release new test datasets due to licensing restrictions.
Regarding training data for glossing, we restrict the glossing data to those released in the GitHub repository + the publicly accessible multilingual IGT datasets GlossLM and PolyGloss:
If you wish to use other glossed corpora, please let us know beforehand to check for dataset overlap.
Regarding non-glossed data (e.g., unlabelled data, sentences with another type of label, dictionaries, grammar books (without overlapping examples)), any publicly available dataset is allowed.
We will mainly evaluate the quality of the predicted gloss. However, we encourage submissions to be able to handle morphological segmentation (second tier of Figure 1), since field linguists usually analyse sentences into morphemes before producing glosses.
We will use the following metrics to evaluate the predicted gloss quality: Morpheme Error Rate (MER), Word-level Accuracy, and Morpheme-level Accuracy.
The main metric for the ranking is MER, following Ginn et al., (2026), while both accuracy scores follow the previous literature for information purposes.
We will average each language’s evaluation metric scores per language set.
TBA
To acknowledge the SIGMORPHON 2023 Shared Task, please cite the following article:
Here are the two versions of the multilingual glossed corpora, which can be used for training and which also contain some evaluation languages.
We warmly thank the organisers of the SIGMORPHON 2023 Shared Task, especially Michael Ginn, for their help.
We suggest using the Google group for general questions that may interest all participants.
For private questions, please write to Shu Okabe (shu dot okabe at tum dot de).