Grammaticality Metrics Save

evaluation suite for testing automatic grammatical error corrections

Project README

Metrics for evaluating grammatical error corrections

These metrics were used in Courtney Napoles, Keisuke Sakaguchi, and Joel Tetreault. There's No Comparison: Reference-less Evaluation Metrics in Grammatical Error Correction. EMNLP 2016

If you use this code or the accompanying CodaLab evaluation, please cite:

  author    = {Napoles, Courtney  and  Sakaguchi, Keisuke  and  Tetreault, Joel},
  title     = {There's No Comparison: Reference-less Evaluation Metrics in Grammatical Error Correction},
  booktitle = {Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing},
  month     = {November},
  year      = {2016},
  address   = {Austin, Texas},
  publisher = {Association for Computational Linguistics},
  pages     = {2109--2115},
  url       = {}

Online evaluation

The CodaLab evaluation can be used for to evaluate grammatical error correcions of the CoNLL-2014 shared task test set.


  1. codalab/
    • Code for evaluating GEC output of the CoNLL 2014 test set using combination of metrics and reference sets.
    • The platform for scoring output can be found at
    • This contains the an error-count method using Language Tool and interpolations of LT with existing GEC metrics GLEU, I-measure, and M2.
  2. heilman-et-al/
Open Source Agenda is not affiliated with "Grammaticality Metrics" Project. README Source: cnap/grammaticality-metrics
Open Issues
Last Commit
6 years ago

Open Source Agenda Badge

Open Source Agenda Rating