? Survey of Deep Metric Learning

         

Traditionally, they have defined metrics in a variety of ways, including pairwise distance, similarity, and probability distribution.

? I hope many researchers will be able to do good research thanks to this repository.

? Updated frequently.

---

Contents

• Pairwise cost methods (stand alone)
• Distribution or Ad-hoc methods
• Probabilistic methods
• Boost-like methods
• Unsupervised methods
• Applications
  • Re-identification
  • Face verification
  • Face recognition
  • Segmentation
  • Image registration
  • Few (zero)-shot approach
  • 3D reconstruction
  • Action localization
  • Adversarial attack
  • Text documentation
  • Pill identification
• Related works
  • Neurips
  • ICLR
  • ICML
  • Others
• Study materials
  • Tutorial
  • Lectures
  • Repository
• Challenges

---

1️⃣ Pairwise cost methods (stand alone)

• Dimensionality Reduction by Learning an Invariant Mapping (Contrastive) [:dizzy::dizzy::dizzy:] (CVPR 2006) [Paper][Caffe][Tensorflow][Keras][Pytorch1][Pytorch2]

• From Point to Set: Extend the Learning of Distance Metrics (ICCV 2013) [Paper]

• FaceNet: A Unified Embedding for Face Recognition and Clustering (Triplet) [:dizzy::dizzy::dizzy:] (CVPR 2015) [Paper][Tensorflow][Pytorch]

• Deep Metric Learning via Lifted Structured Feature Embedding (LSSS) [:dizzy:] (CVPR 2016) [Paper][Chainer][Caffe][Pytorch1][Pytorch2][Tensorflow]

• Improved Deep Metric Learning with Multi-class N-pair Loss Objective (N-pair) [:dizzy:] (NIPS 2016) [Paper][Pytorch][Chainer]

• Beyond triplet loss: a deep quadruplet network for person re-identification (Quadruplet) (CVPR 2017) [Paper]

• Deep Metric Learning via Facility Location (CVPR 2017) [Paper][Tensorflow]

• No Fuss Distance Metric Learning using Proxies (Proxy NCA) [:dizzy::dizzy:] (ICCV 2017) [Paper][Pytorch1][Pytorch2][Chainer]

• Sampling Matters in Deep Embedding Learning (Margin) (ICCV 2017) [Paper][Pytorch][TensorFlow][MXNet]

• Deep Metric Learning with Angular Loss (Angular) (CVPR 2017) [Paper][Tensorflow][Chainer]

• Deep Metric Learning by Online Soft Mining and Class-Aware Attention (AAAI 2019) [Paper]

• Ensemble Deep Manifold Similarity Learning using Hard Proxies (CVPR 2019) [Paper]

• Deep Metric Learning Beyond Binary Supervision (Log_ratio) (CVPR 2019) [Paper][Pytorch]

• A Theoretically Sound Upper Bound on the Triplet Loss for Improving the Efficiency of Deep Distance Metric Learning (CVPR 2019) [Paper]

• Ranked List Loss for Deep Metric Learning (RLL) (CVPR 2019) [Paper][Matlab]

• Deep Metric Learning to Rank (FastAP) (CVPR 2019) [Paper][Matlab]

• SoftTriple Loss: Deep Metric Learning Without Triplet Sampling (Soft-Trip) (ICCV 2019) [Paper][Tensorflow]

• Curvilinear Distance Metric Learning (CDML) (Neurips 2019) [Paper]

• Proxy Anchor Loss for Deep Metric Learning (Proxy-Anchor) [:dizzy::dizzy:] (CVPR 2020) [Paper] [Pytorch]

• Optimizing Rank-based Metrics with Blackbox Differentiation (RaMBO) [:dizzy:] (CVPR 2020) [Paper]

• ProxyNCA++: Revisiting and Revitalizing Proxy Neighborhood Component Analysis (Proxy++) (ECCV 2020) [Paper][PyTorch]

• Fewer is More: A Deep Graph Metric Learning Perspective Using Fewer Proxies (ProxyGML) (NeurIPS 2020) [Paper][PyTorch]

• Deep Metric Learning with Spherical Embedding (NeurIPS 2020) [Paper]

• SLADE: A Self-Training Framework For Distance Metric Learning (CVPR 2021) [Paper]

• Learning Intra-Batch Connections for Deep Metric Learning (ICML 2021) [Paper][Pytorch]

---

2️⃣ Distribution or Ad-hoc methods

• Image Set Classification Using Holistic Multiple Order Statistics Features and Localized Multi-Kernel Metric Learning (ICCV 2013) [Paper]

• Deep Metric Learning for Practical Person Re-Identification (Binomial deviance) (ICPR 2014) [Paper][Tensorflow][Pytorch]

• Learning Deep Embeddings with Histogram Loss (Histogram) [:dizzy::dizzy:] (NIPS 2016) [Paper][Tensorflow][Pytorch][Caffe]

• Learning Deep Disentangled Embeddings With the F-Statistic Loss (F-stat) (NIPS 2018) [Paper][Tensorflow]

• Deep Metric Learning via Subtype Fuzzy Clustering (SCDM) (PR 2020) [Paper]

• Deep Asymmetric Metric Learning via Rich Relationship Mining (DAML) (CVPR 2019) [Paper]

• Hardness-Aware Deep Metric Learning (HDML) [:dizzy::dizzy:] (CVPR 2019) [Paper][Tensorflow]

• Signal-to-Noise Ratio: A Robust Distance Metric for Deep Metric Learning (DSML) (CVPR 2019) [Paper]

• Multi-similarity Loss with General Pair Weighting for Deep Metric Learning (MSLoss) (CVPR 2019) [Paper][Pytorch]

• Deep Meta Metric Learning (DMML) (ICCV 2019) [Paper][Pytorch]

• Symmetrical Synthesis for Deep Metric Learning (Symm) (AAAI 2020) [Paper] [Tensorflow]

• Embedding Expansion: Augmentation in Embedding Space for Deep Metric Learning (EE) (CVPR 2020) [Paper] [Mxnet]

• Cross-Batch Memory for Embedding Learning (CVPR 2020) [Paper] [Pytorch]

• Distance Metric Learning with Joint Representation Diversification (JRD) (ICML 2020) [Paper][Pytorch]

• Revisiting Training Strategies and Generalization Performance in Deep Metric Learning (ICML 2020) [Paper][PyTorch]

• PADS: Policy-Adapted Sampling for Visual Similarity Learning (PADS) [:dizzy:] (CVPR 2020) [Paper][PyTorch]

• A Metric Learning Reality Check [:dizzy::dizzy:] (ECCV 2020) [Paper][Pytorch]

• Circle Loss: A Unified Perspective of Pair Similarity Optimization (CircleLoss) (CVPR 2020) [Paper][PyTorch]

• RankMI: A Mutual Information Maximizing Ranking Loss (RankMI) (CVPR 2020) [Paper]

• Virtual sample-based deep metric learning using discriminant analysis (PR 2020) [Paper]

• Provably Robust Metric Learning (NeurIPS 2020) [Paper]

• Deep Metric Learning with Graph Consistency (AAAI 2021) [Paper]

• Multi-level Distance Regularization for Deep Metric Learning (AAAI 2021) [Paper]

• Proxy Synthesis: Learning with Synthetic Classes for Deep Metric Learning (AAAI 2021) [Paper][PyTorch]

• Noise-resistant Deep Metric Learning with Ranking-based Instance Selection (PRISM) (CVPR 2021) [Paper][PyTorch]

• Simultaneous Similarity-based Self-Distillation for Deep Metric Learning (S2SD) (ICML 2021) [Paper][Pytorch]

---

3️⃣ Probabilistic methods

• Latent Coincidence Analysis: A Hidden Variable Model for Distance Metric Learning (NIPS 2012) [Paper]

• Information-theoretic Semi-supervised Metric Learning via Entropy Regularization (ICML 2014) [Paper]

• Learning Deep Disentangled Embeddings With the F-Statistic Loss (F-stat) (NIPS 2018) [Paper][Tensorflow]

---

4️⃣ Boost-like methods

• BIER-Boosting Independent Embeddings Robustly (BIER) [:dizzy::dizzy:] (ICCV 2017) [Paper][Tensorflow]

• Hard-Aware Deeply Cascaded Embedding (ICCV 2017) [Paper][Caffe]

• Learning Spread-out Local Feature Descriptors (ICCV 2017) [Paper]

• Deep Adversarial Metric Learning (CVPR 2018) [Paper][Chainer]

• Deep Randomized Ensembles for Metric Learning (ECCV 2018) [Paper][Pytorch]

• Attention-based Ensemble for Deep Metric Learning (ECCV 2018) [Paper]

• Deep Metric Learning with Hierarchical Triplet Loss (ECCV 2018) [Paper]

• Hybrid-Attention based Decoupled Metric Learning for Zero-Shot Image Retrieval (CVPR 2019) [Paper] [Caffe]

• Divide and Conquer the Embedding Space for Metric Learning (CVPR 2019) [Paper] [Pytorch]

• Stochastic Class-based Hard Example Mining for Deep Metric Learning (CVPR 2019) [Paper]

• Deep Metric Learning with Tuplet Margin Loss (ICCV 2019) [Paper]

• Metric Learning with HORDE: High-Order Regularizer for Deep Embeddings (ICCV 2019) [Paper][Keras]

• MIC: Mining Interclass Characteristics for Improved Metric Learning [:dizzy:] (ICCV 2019) [Paper][Pytorch]

• DiVA: Diverse Visual Feature Aggregation for Deep Metric Learning (DIVA) (ECCV 2020) [Paper] [PyTorch]

• The Group Loss for Deep Metric Learning (GroupLoss) (ECCV 2020) [Paper][PyTorch]

---

5️⃣ Unsupervised methods

• Unsupervised Embedding Learning via Invariant and Spreading Instance Feature [:dizzy::dizzy::dizzy:] (CVPR 2019) [Paper][Pytorch]

• Unsupervised Deep Metric Learning with Transformed Attention Consistency and Contrastive Clustering Loss (ECCV 2020) [Paper]

• Unsupervised Hyperbolic Metric Learning [:dizzy:] (CVPR 2021) [Paper]

---

6️⃣ Applications

Re-identification

• Person Re-Identification using Kernel-based Metric Learning Methods (ECCV 2014) [Paper][Matlab]

• Similarity Learning on an Explicit Polynomial Kernel Feature Map for Person Re-Identification (CVPR 2015) [Paper]

• Learning to rank in person re-identification with metric ensembles (CVPR 2015) [Paper]

• Person Re-identification by Local Maximal Occurrence Representation and Metric Learning (CVPR 2015) [Paper][Matlab]

• Learning a Discriminative Null Space for Person Re-identification (CVPR 2016) [Paper][Matlab]

• Similarity Learning with Spatial Constraints for Person Re-identification (CVPR 2016) [Paper][Matlab]

• Consistent-Aware Deep Learning for Person Re-identification in a Camera Network (CVPR 2016) [Paper]

• Re-ranking Person Re-identification with k-reciprocal Encoding (CVPR 2017) [Paper][Caffe]

• Scalable Person Re-identification on Supervised Smoothed Manifold (CVPR 2017) [Paper]

• One-Shot Metric Learning for Person Re-identification (CVPR 2017) [Paper]

• Point to Set Similarity Based Deep Feature Learning for Person Re-identification (CVPR 2017) [Paper]

• Consistent-Aware Deep Learning for Person Re-identification in a Camera Network (CVPR 2017) [Paper]

• Cross-view Asymmetric Metric Learning for Unsupervised Person Re-identification (ICCV 2017) [Paper][Matlab]

• Efficient Online Local Metric Adaptation via Negative Samples for Person Re-Identification (ICCV 2017) [Paper]

• Mask-guided Contrastive Attention Model for Person Re-Identification (CVPR 2018) [Paper][Caffe]

• Efficient and Deep Person Re-Identification using Multi-Level Similarity (CVPR 2018) [Paper]

• Group Consistent Similarity Learning via Deep CRF for Person Re-Identification (CVPR 2018) [Paper][Pytorch]

• Perceive Where to Focus: Learning Visibility-aware Part-level Features for Partial Person Re-identification (CVPR 2019) [Paper]

• Invariance Matters: Exemplar Memory for Domain Adaptive Person Re-identification (CVPR 2019) [paper] [Pytorch]

• Learning to Reduce Dual-level Discrepancy for Infrared-Visible Person Re-identification (CVPR2019) [Paper]

• Densely Semantically Aligned Person Re-Identification (CVPR 2019) [Paper]

• Generalizable Person Re-identification by Domain-Invariant Mapping Network (CVPR 2019) [Paper]

• Re-ranking via Metric Fusion for Object Retrieval and Person Re-identification (CVPR 2019) [Paper]

• Weakly Supervised Person Re-Identification (CVPR 2019) [Paper]

• Towards Rich Feature Discovery with Class Activation Maps Augmentation for Person Re-Identification (CVPR 2019) [Paper]

• Joint Discriminative and Generative Learning for Person Re-identification (CVPR 2019) [Paper]

• Unsupervised Person Re-identification by Soft Multilabel Learning (CVPR 2019) [Paper] [Pytorch]

• Patch-based Discriminative Feature Learning for Unsupervised Person Re-identification (CVPR 2019) [Paper]

• Attribute-Driven Feature Disentangling and Temporal Aggregation for Video Person Re-Identification (CVPR 2019) [Paper]

• AANet: Attribute Attention Network for Person Re-Identifications (CVPR 2019) [Paper]

• VRSTC: Occlusion-Free Video Person Re-Identification (CVPR 2019) [paper]

• Adaptive Transfer Network for Cross-Domain Person Re-Identification (CVPR 2019) [Paper]

• Pyramidal Person Re-IDentification via Multi-Loss Dynamic Training (CVPR 2019) [Paper]

• Interaction-and-Aggregation Network for Person Re-identification (CVPR 2019) [Paper]

• Vehicle Re-identification with Viewpoint-aware Metric Learning (ICCV 2019) [Paper]

• Distilled Person Re-identification: Towards a More Scalable System (CVPR 2019) [Paper]

• Unsupervised Person Re-Identification via Multi-Label Classification (CVPR 2020) [Paper]

• Style Normalization and Restitution for Generalizable Person Re-identification (CVPR 2020) [Paper]

• Hi-CMD: Hierarchical Cross-Modality Disentanglement for Visible-Infrared Person Re-Identification (CVPR 2020) [Paper][PyTorch]

• AD-Cluster: Augmented Discriminative Clustering for Domain Adaptive Person Re-identification (CVPR 2020) [Paper]

• The Dilemma of TriHard Loss and an Element-Weighted TriHard Loss for Person Re-Identification (NeurIPS 2020) [Paper][PyTorch]

Face verification

• Discriminative Deep Metric Learning for Face Verification in the Wild (CVPR 2014) [Paper]

• Fantope Regularization in Metric Learning (CVPR 2014) [Paper]

• Deep Transfer Metric Learning (CVPR 2015) [Paper]

• BioMetricNet: deep unconstrained face verification through learning of metrics regularized onto Gaussian distributions (ECCV 2020) [Paper]

Face recognition

• Large Scale Metric Learning from Equivalence Constraints (CVPR 2012) [Paper]

• Fusing Robust Face Region Descriptors via Multiple Metric Learning for Face Recognition in the Wild (CVPR 2013) [Paper]

• Similarity Metric Learning for Face Recognition (ICCV 2013) [Paper]

• Projection Metric Learning on Grassmann Manifold with Application to Video based Face Recognition (CVPR 2015) [Paper]

Segmentation

• Point Cloud Oversegmentation with Graph-Structured Deep Metric Learning (CVPR 2019) [Paper]

• 3D Instance Segmentation via Multi-Task Metric Learning (ICCV 2019) [Paper]

Image registration

• Metric Learning for Image Registration (CVPR 2019) [Paper]

Few (zero)-shot approach

• RepMet: Representative-based metric learning for classification and few-shot object detection (CVPR 2019) [Paper] [Pytorch]

• Revisiting Metric Learning for Few-Shot Image Classification (arXiv 2019) [Paper]

• Model-Agnostic Metric for Zero-Shot Learning (WACV 2020) [Paper]

3D reconstruction

• Learning Embedding of 3D models with Quadric Loss (BMVC 2019) [Paper][Pytorch]

Action localization

• Weakly Supervised Temporal Action Localization Using Deep Metric Learning (WACV2020) [Paper][Pytorch]

Adversarial attack

• Metric Learning for Adversarial Robustness (Neurips 2019) [Paper][Tensorflow]

Text documentation

• Fast(er) Reconstruction of Shredded Text Documents via Self-Supervised Deep Asymmetric Metric Learning (CVPR 2020) [Paper][Code]

Pill identification

• ePillID Dataset: A Low-Shot Fine-Grained Benchmark for Pill Identification (CVPR 2020) [Paper][Code]

---

7️⃣ Related works

Neurips

• Distance Metric Learning for Large Margin Nearest Neighbor Classification (Neurips 2005) [Paper][Journal][Python]

• First approach of local metric learning

• Metric Learning by Collapsing Classes (Neurips 2005) [Paper]

• Online Metric Learning and Fast Similarity Search (Neurips 2008) [Paper]

• Sparse Metric Learning via Smooth Optimization (Neurips 2009) [Paper]

• Metric Learning with Multiple Kernels (Neurips 2011) [Paper]

• Hamming Distance Metric Learning (Neurips 2012) [Paper][Matlab]

• Parametric Local Metric Learning for Nearest Neighbor Classification (Neurips 2012) [Paper]

• A representative approach of local metric learning

• Non-linear Metric Learning (Neurips 2012) [Paper]

• Latent Coincidence Analysis: A Hidden Variable Model for Distance Metric Learning (Neurips 2012) [Paper]

  • They deal with probabilistic model based on EM algorithm

• Semi-Crowdsourced Clustering: Generalizing Crowd Labeling by Robust Distance Metric Learning (Neurips 2012) [Paper]

• Similarity Component Analysis (Neurips 2013) [Paper]

• Discriminative Metric Learning by Neighborhood Gerrymandering (Neurips 2014) [Paper]

• Log-Hilbert-Schmidt metric between positive definite operators on Hilbert spaces (Neurips 2014) [Paper]

• Metric Learning for Temporal Sequence Alignment (Neurips 2014) [paper]

• Sample complexity of learning Mahalanobis distance metrics (Neurips 2015) [Paper]

• Regressive Virtual Metric Learning (Neurips 2015) [Paper]

• What Makes Objects Similar: A Unified Multi-Metric Learning Approach (Neurips 2016) [Paper]

• Improved Error Bounds for Tree Representations of Metric Spaces (Neurips 2016) [Paper]

• What Makes Objects Similar: A Unified Multi-Metric Learning Approach (Neurips 2016) [Paper]

• Learning Low-Dimensional Metrics (Neurips 2017) [Paper]

• Generative Local Metric Learning for Kernel Regression (Neurips 2017) [Paper]

• Persistence Fisher Kernel: A Riemannian Manifold Kernel for Persistence Diagrams (Neurips 2018) [Paper][Matlab]

• Bilevel Distance Metric Learning for Robust Image Recognition (Neurips 2018) [Paper]

• Adapted Deep Embeddings: A Synthesis of Methods for k-Shot Inductive Transfer Learning (Neurips 2018) [Paper][Tensorflow]

• Fast Low-rank Metric Learning for Large-scale and High-dimensional Data (Neurips 2019) [Paper][Matlab]

• Metric Learning for Adversarial Robustness (Neurips 2019) [Paper][Tensorflow]

• Region-specific Diffeomorphic Metric Mapping (Neurips 2019) [Paper][Pytorch]

• Fast Low-rank Metric Learning for Large-scale and High-dimensional Data (FLRML) (Neurips 2019) [Paper][Matlab]

• Contrastive Learning with Adversarial Examples (NeurIPS 2020) [Paper]

• Simultaneous Preference and Metric Learning from Paired Comparisons (NeurIPS 2020) [Paper][MatLab]

• Multi-task Batch Reinforcement Learning with Metric Learning (NeurIPS 2020) [Paper]

ICLR

• Deep Metric Learning Using Triplet Network (ICLR 2015 workshop) [Paper][Keras][Torch]

• Metric Learning with Adaptive Density Discrimination (Magnet loss) (ICLR 2016) [Paper][Pytorch1][Pytorch2][Tensorflow]

• Semi-supervised Deep Learning by Metric Embedding (ICLRW 2017) [Paper][Torch(Lua)]

• Learning Wasserstein Embedding (ICLR 2018) [Paper][Keras]

• Smoothing the Geometry of Probabilistic Box Embeddings (ICLR 2019) [Paper][Tensorflow]

  • New type of embedding method

• Unsupervised Domain Adaptation for Distance Metric Learning (ICLR 2019) [Paper]

• ROTATE: Knowledge Graph Embedding bt Relational Rotation in Complex Space (ICLR 2019) [Paper][Pytorch]

  • Define relationship by using rotation in vector space

• Conditional Network Embeddings (ICLR 2019) [Paper][Matlab]

  • Add additional information with respect to given structural properties

• Contrastive Learning with Hard Negative Samples (ICLR 2021) [Paper][PyTorch]

ICML

• Gromov-Wasserstein Learning for Graph Matching and Node Embedding (ICML 2019) [Paper][Pytorch]

  • Propose novel framework btw. relation graph and embedding space

• Hyperbolic Disk Embeddings for Directed Acyclic Graphs (ICML 2019) [Paper][Luigi]

  • Propose embedding framework on quasi-metric space

Others

• A unifying mutual information view of metric learning: cross-entropy vs. pairwise losses (ECCV 2020) [Paper][PyTorch]

• Quadruplet Selection Methods for Deep Embedding Learning (ICIP 2019) [Paper]

• Cross-Batch Memory for Embedding Learning (ArXiv 2020) [Paper]

• Calibrated neighborhood aware confidence measure for deep metric learning (ArXiv 2020) [Paper]

• Diversified Mutual Learning for Deep Metric Learning (ArXiv 2020) [Link]

• Deep Metric Learning Based on Rank-sensitive Optimization of Top-k Precision (CIKM 2020) [Paper]

• Training Deep Retrieval Models with Noisy Datasets: Bag Exponential Loss (PR2021) [Paper]

• Group Softmax Loss with Discriminative Feature Grouping (WACV2021) [Paper]

• A Multi-class Hinge Loss for Conditional GANs (WACV2021) [Paper]

• Embedding Transfer with Label Relaxation for Improved Metric Learning (CVPR 2021) [Paper]

---

8️⃣ Study materials

Tutorial

• Metric learning tutorial (ICML 2010) [Video]

• Metric Learning and Manifolds: Preserving the Intrinsic Geometry (MS research 2016) [VIdeo]

• Visual Search (Image Retrieval) and Metric Learning (CVPR 2018) [Video]

• Image Retrieval in the Wild (CVPR 2020) [Video]

Lecture

• Topology and Manifold (International Winter School on Gravity and Light 2015) [Video]

• Metric learning lecture (Waterloo University) [Video]

• Understanding of Mahalanobis distance [Video]

• Metric Learning by Caltech (2018) [Video]

Repository

• Various metric loss implementation (written by Pytorch) [Site]

• A metric learning reality check [Site]

• Person re-identification in Pytorch [Site]

Challenges

• TO BE UPDATED

---

Milestone

• Add Pairwise cost methods

• Add Distribution or other variant methods

• Add Probabilistic methods

• Add Boost-like methods

• Add applications

• Add study materials

• Add brief descriptions