Dice Group Gerbil Versions Save

• Bug Fixes
  • Dependency issues
  • Exception was thrown when no answer was sent (Fixed)
  • Default Language is English, was null before
  • Fixed: QA Matching problems with language tag
  • URL Validator had some problems, fixed by using URI instead (which is more correct).
• Enhancements
  • #250, #344 - added option to upload dataset and answer file
  • HTTPS instead of HTTP for maven repository
  • More convenient addition of a QA System (Removed the need of adding system in NLIWODBasedSystem)
• Clean Code
  • Added additional F-Measure test
• Additions
  • #234 Added LCQUAD Dataset
  • QALD 9
  • QALD Generator CLI
  • dbpedia entity V1 added
• Removals

• Several bug fixes
  • Dependency Issues
  • fixed correlation figure bug
  • Relaxed Redirection handling
  • Several Annotators using the Wrong IP
    • Rewritten WAT and TagMe2, now working again (#207)
  • Updated MSBNC Dataset
  • #348
• Enhancements
  • Added DB Unit Tests
  • Derczynski dataset renamed
  • Maven using HTTPS instead of HTTP
• Clean Code
  • Added Test Coverage to Codacy and Travis
  • Codacy Clean Code fixes
• Removals
  • DoSeR is not existing anymore thus removed

Features

• Added the handling of changes that were done applied to the document by the benchmarked system (see #205)
• Improved database structure to increase performance (#260)
• Added OpenTapioca annotator (#316)

Bug Fixes

• Fixed dataset properties
• Fixed the usage of a wrong matching type for RE
• Fixed experiment type hierarchy
• Fixed FOX URL
• Updated AGDISTIS service URL and renamed it to AGDISTIS/MAG(#257)
• Added the handling of empty datasets (#295)

Features

• Added Tasks
  • Relation Extraction (RE): Extract the Relation to known Entities
  • OKE2018Task4: RE + A2KB
• Added Datasets
  • OKE 2018 trainings sets

Bug Fixes

• OKE 2017 configuration added
• DBpedia Spotlight URL fixed to new URL
• Available Microposts datasets may have empty lines with only IDs which caused Errors, this is fixed by ignoring those lines. (#206)

Known Problems

Features

• Added tooltips in front-end
• Added Lucene Indexed based Same As Rretrieval to improve runtime of experiments
• QA Answers can now be logged into csv files for debug comparison (use org.aksw.gerbil.qa.matching.printAnswers=true in gerbil.properties)
• Added multilingual support with language code (e.g. fr for French) WARNING: To behold your previous results, you have to uncomment the following line once: database-schema

Systems

Added 2 pre implemented systems

• Qanary

• OKBQA

• File Based Annotators can now be in QALD or ExtendedJson format

Datasets

• Added QALD 7 training datasets
• Fixed bug with QALD 6 datasets not loading
• File Based Datasets can now be in QALD or ExtendedJson format

Features

• Added annotators
  • DoSeR (#152)
  • NERFGUN (#160)
  • PBOH (#153)
  • xLisa (#113)
• Added datasets
  • Derczynski (#45)
  • ERD14 and GERDAQ (#158)
  • Microposts 2015 and 2016 (#82, #132)
  • Ritter (#43)
  • Senseval 2 and 3 (#17)
  • UMBC (#44)
  • WSDM 2012 (#46)
• Introduced the RT2KB experiment type that comprises recognition and typing of entities
• Introduced index based sameAs relation retrieval and entity checking for KBs that do not change very often (e.g., DBpedia). Downloading the indexes is optional and GERBIL can run without them (but has the same performance drawbacks as the last versions).
• A warning should be shown in the GUI if the server is busy at the moment.
• Implemented checks for certain datasets and annotators. If dataset files are missing (because of licenses) or API keys of annotators, they are not available in the front end.

Bug Fixes

• Fixed the problem that the NIFWSTestingController was not able to check whether a NIF-based web service is available for ETyping experiments. (#151)
• Fixed encoding problems by setting the encoding to UTF-8 where needed instead of relying on the default encoding (#165)

Known Problems

• Because of the retrieving of sameAs relations as well as the checking of all entity URIs, the evaluation takes more time than in older versions. With the newly introduced indexes this times is reduced by one half.

• Eased the access to internal writers and parsers of the different document (list) and NIF writer and parser classes.
• Fixed the parsing and writing of TypedSpanImpl instances. (#150)

Features

• Added the Microposts 2013 dataset (#41).
• Added the Derczynski dataset (#45).
• Added the UMBC dataset (#44).
• Added the xLisa annotator (#113).

Bug Fixes

• Implemented the HTTPS connection to d4science.org for the new TagMe 2 instance.
• Added a simple retry strategy if an org.apache.http.NoHttpResponseException is thrown (#121).
• Removed the Wikiminer since this service does not exist anymore.

Known Problems

• Because of the retrieving of sameAs relations as well as the checking of all entity URIs, the evaluation takes more time than in older versions.

This release is an extension of GERBIL to benchmark Question Answering systems. This is the first release of GERBIL for this area and the first comprehensive online benchmark platform for QA.

Features

• This release implemetns the QA experiment type as well as various sub experiments, see https://github.com/AKSW/gerbil/wiki/Question-Answering
• Implemented adapters for 4 different QA systems
  • HAWK http://aksw.org/Projects/HAWK.html
  • QAKIS http://qakis.org/qakis2/
  • SINA http://sina.aksw.org/
  • YODA http://live.ailao.eu/
• Added 16 datasets of the QALD challenge (http://qald.sebastianwalter.org/) as well as the NLQ (http://www.okbqa.org/) dataset
• Implemented the upload of QALD datasets as well as QALD answer files.

Features

• Decreased the time needed to load the most recent results (#77)

Bug Fixes

• Fixed the problem that uploaded NIF-file-based datasets did not have been initialized using the EntityChecker and SameAsRetriever mechanism.
• Fixed the AIDA/CoNLL Dataset adapter. (#118)
• Fixed a bug that prevented the correct loading of a dataset if more than one annotator should be tested.
• Fixed the RDF representation of experiment results.

Known Problems

• Because of the retrieving of sameAs relations as well as the checking of all entity URIs, the evaluation takes more time than in older versions.