CoNLL 2017 shared task: Multilingual parsing from raw text to universal dependencies

Zeman D.

;Popel M.;Straka M.;Hajič J.;Nivre J.;Ginter F.;Luotolahti J.;Pyysalo S.;Petrov S.;Potthast M.;Tyers F.;Badmaeva E.;Gökırmak M.;Nedoluzhko A.;Cinková S.;Hajič J.;Jr.;Hlaváčová J.;Kettnerová V.;Urešová Z.;Kanerva J.;Ojala S.;Missilä A.;Manning C.;Schuster S.;Reddy S.;Taji D.;Habash N.;Leung H.;de Marneffe M. -C.;Sanguinetti M.;Simi M.;Kanayama H.;de Paiva V.;Droganova K.;Alonso H. M.;Çöltekin Ç.;Sulubacak U.;Uszkoreit H.;Macketanz V.;Burchardt A.;Harris K.;Marheinecke K.;Rehm G.;Kayadelen T.;Attia M.;Elkahky A.;Yu Z.;Pitler E.;Lertpradit S.;Mandl M.;Kirchner J.;Alcalde H. F.;Strnadová J.;Banerjee E.;Manurung R.;Stella A.;Shimada A.;Kwak S.;Mendonça G.;Lando T.;Nitisaroj R.;Li J.

2017-01-01

---

Abstract

The Conference on Computational Natural Language Learning (CoNLL) features a shared task, in which participants train and test their learning systems on the same data sets. In 2017, one of two tasks was devoted to learning dependency parsers for a large number of languages, in a real-world setting without any gold-standard annotation on input. All test sets followed a unified annotation scheme, namely that of Universal Dependencies. In this paper, we define the task and evaluation methodology, describe data preparation, report and analyze the main results, and provide a brief categorization of the different approaches of the participating systems.