Compress Fasttext Versions Save

Closing https://github.com/avidale/compress-fasttext/issues/19#event-10655768121 by removing pqkmeans dependency that has un-installable dependencies of its own.

• Wrap up the refactoring related to new gensim version
• add FastTextTransformer, a scikit-learn-like wrapper for feature extraction

Support of gensim>=4.0.0 and deprecation of earlier gensim

New released models

Russian models based on geowac_tokens_none_fasttextskipgram_300_5_2020 from RusVectores, 1.9GB:

English models based on cc.en.300.bin from the Facebook website, 7.2GB:

Much more small models for various languages can be found at https://zenodo.org/record/4905385.

• Publish more compressed models and compare their quality
• Make the compressed models downloadable

• require sklearn and pqkmeans only in the [full] setup mode

Now attempts of arithmetic operations on compressed matrices do not raise errors. However, they lead to conversion of these matrices to numpy.array, which uses time and memory.

Now prune_ft_freq method takes into account not only n-gram frequency, but also the norm of its embedding. This improves model compression accuracy for the same model size.

We publish the code for compressing Gensim FastText models and using their small versions.

We also publish 4 compressed versions of the ruscorpora_none_fasttextskipgram_300_2_2019 model from RusVectores.