Adversarial Training Methods

Adversarial and virtual adversarial training methods for semi-supervised text classification.

Based on the paper: Adversarial training methods for semi-supervised text classification, ICLR 2017, Miyato T., Dai A., Goodfellow I.

Without the pre-training phase

Requirements

Dataset creation

Download the IMDB dataset.

Then we have to do the following steps:

• prepare dataset in order to train GloVe using glove_training_preparation.ipynb
• set the vectors length at 256 and remove words appearing in less than 3 reviews in the GloVe training script
• run GloVe training script
• create the embedding matrix (used by the network) and the dictionary (used to convert reviews in sequences of index) using pretrained_glove_embedding_script.ipynb
• convert reviews in sequences of index using dataset_script.pynb and test_dataset_script.ipynb

Training

5% of the training set is used for validation

use paper_network.ipynb or pyramidal_network.ipynb according to the network architecture you want to use.

Paper model	Pyramidal model

Results

Sequences are truncated at 1200 (pyramidal model), 600 and 400 to test the sensitivity of the model to reviews lengths. In particular we cut off or add zero-padding at the initial part of the review.