Dccuchile Wefe Versions Save

What's Changed

• Fix bug in RIPA metric:

Previously it was considering n-1 words for the 2 sets of targets, therefore, it was omitting the last words of the targets, which was incorrect with respect to the original definition of the paper.

Now it correctly occupies all the target words.

• Added project logos to documentation.
• Added pre-commit to github actions, which improves automatic QA.
• Several code style and typing corrections given by flake, black, ruff and isort.

New Contributors

• @h4c5 made their first contribution in https://github.com/dccuchile/wefe/pull/43

Associated PR

• Add Logos by @pbadillatorrealba in https://github.com/dccuchile/wefe/pull/40
• fix: include last pair in RIPA score by @h4c5 in https://github.com/dccuchile/wefe/pull/43
• Add precommit job to github actions and correct linter issues by @pbadillatorrealba in https://github.com/dccuchile/wefe/pull/44
• Release 0.4.1 by @pbadillatorrealba in https://github.com/dccuchile/wefe/pull/45

Full Changelog: https://github.com/dccuchile/wefe/compare/0.4.0...0.4.1

Version 0.4.0 Changelog

• 3 new bias mitigation methods (debias) implemented: Double Hard Debias, Half Sibling Regression and Repulsion Attraction Neutralization.
• The library documentation of the library has been restructured. Now, the documentation is divided into user guide and theoretical framework The user guide does not contain theoretical information. Instead, theoretical documentation can be found in the conceptual guides.
• Improved API documentation and examples. Added multilingual examples contributed by the community.
• The user guides are fully executable because they are now on notebooks.
• There was also an important improvement in the API documentation and in metrics and debias examples.
• Improved library testing mechanisms for metrics and debias methods.
• Fixed wrong repr of query. Now the sets are in the correct order.
• Implemented repr for WordEmbeddingModel.
• Testing CI moved from CircleCI to GithubActions.
• License changed to MIT.

This version mainly fixes a bug in RNSB and updates case study scores in examples. The changes are as follows:

• Fixed RNSB bug where the classification labels were interchanged and could produce erroneous results when the attributes are of different sizes.
• Fixed RNSB replication notebook
• Update of WEFE case study scores.
• Improved documentation examples for WEAT, RNSB, RIPA.
• Holdout parameter added to RNSB, which allows to indicate whether or not a holdout is performed when training the classifier.
• Improved printing of the RNSB evaluation.

This new version includes a new debias module as well as a complete refactoring to the preprocessing of word embeddings. Changelog:

• Implemented Bolukbasi et al. 2016 Hard Debias.
• Implemented Thomas Manzini et al. 2019 Multiclass Hard Debias.
• Implemented a fetch function to retrieve gn-glove female-male word sets.
• Moved the transformation logic of words, sets and queries to embeddings to its own module: preprocessing
• Enhanced the preprocessor_args and secondary_preprocessor_args metric preprocessing parameters to an list of preprocessors preprocessors together with the parameter strategy indicating whether to consider all the transformed words ('all') or only the first one encountered ('first').
• Renamed WordEmbeddingModel attributes model and model_name to wv and name respectively.
• Renamed every run_query word_embedding argument to model in every metric.

Some bug fixes and RIPA integration prior to the release of the new version.

The main change in this version is an improvement in how WEFE transforms word sets into embeddings. Now all this work is contained in the WordEmbeddingModel class. It also contains several improvements, both to this process and to the library in general. See the changelog for more information.

Note: Contains changes that may not be compatible with previous versions.

• Renamed optional run_query parameter warn_filtered_words to warn_not_found_words.
• Added word_preprocessor_args parameter to run_query that allows to specify transformations prior to searching for words in word embeddings.
• Added secondary_preprocessor_args parameter to run_query which allows to specify a second pre-processor transformation to words before searching them in word embeddings. It is not necessary to specify the first preprocessor to use this one.
• Implemented __getitem__ function in WordEmbeddingModel. This method allows to obtain an embedding from a word from the model stored in the instance using indexers.
• Removed underscore from class and instance variable names.
• Improved type and verification exception messages when creating objects and executing methods.
• Fix an error that appeared when calculating rankings with two columns of aggregations with the same name.
• Ranking correlations are now calculated using pandas corr method.
• Changed metric template, name and short_names to class variables.
• Implemented random_state in RNSB to allow replication of the experiments.
• run_query now returns as a result the default metric requested in the parameters and all calculated values that may be useful in the other variables of the dictionary.
• Fixed problem with api documentation: now it shows methods of the classes.
• Implemented p-value for WEAT