Hierarchical Word Sense Disambiguation Using WordNet Senses Save

Word Sense Disambiguation

Word sense disambiguation (WSD) is the ability to identify the meaning of words in context. We address this problem using series of end-to-end neural architectures using bidirectional Long Short Term Memory (LSTM). We propose two variants for WSD: an end-to-end word specific neural model and all-words neural model. In the word specific models we have to train models for every disambiguation target word. We addressed this issue using the all-words model which rely on sequence learning. We also used POS tags to improve the performance. We tried different variants of attention mechanisms for the all-words model. Performance was boosted by using convolutional neural networks (CNN) which captures local features around the words that is normally what humans do for predicting the senses. We further improved the performance using hierarchical models. We used POS tags as hierarchy and used two variants as soft masking and hard masking.

Methods

• Word Specific Model trained on Four Word Dataset
• Word Specific Model trained on One Million Dataset
• All-words Model
• Hierarchical Model

Best Models

• All-words Model+CNN
• All-words Hierarchical Model+Soft Masking
• All-words Hierarchical Model+Hard Masking

Details

For detailed information about models and results:

• Report
• Presentation

All words Models

All-words Hierarchical Model+Soft Masking

All-words Hierarchical Model+Hard Masking

Basic Model

Basic Model+Local Attention

Basic Model+Local Attentionn+Hidden States

Basic Model+Local Attentionn+Hidden States+CRF

Basic Model+Gated Attention

Basic Model+CNN

Word Specific Models

Basic Model

Files with name as Model-1-multigpu-1.ipynb are the basic models

Basic Model+POS Tags

Files with name as Model-1-multigpu-2.ipynb are the basic models

Basic Model+POS Tags+CRF

Files with name as Model-1-multigpu-3.ipynb are the basic models

Word specific hierarchical model

Files with name as Model-1-multigpu-4.ipynb are the basic models