This is a record of current experiment result on some realistic noisy label datasets

1. Clothing1M dataset

from: CVPR-15: Learning from Massive Noisy Labeled Data for Image Classification

• 14 classes: T-shirt, Shirt, Knitwear, Chiffon, Sweater, Hoodie, Windbreaker, Jacket, Down Coat, Suit, Shawl, Dress, Vest, and Underwear
• noisy labeled training dataset ($D_\eta$): $10^6$
• clean train data($D_c$): 47,570
• clean validation set: 14,313
• clean test set: 10,526

It's not column-diagonally dominant, thus small-loss trick may not work. But if examples in noisy class 3 and noisy class 5 have been swapped, it may become column-diagonally dominant, in which case small-loss trick may work.

1. CVPR-15: Learning from Massive Noisy Labeled Data for Image Classification

2. CVPR-17: Making Deep Neural Networks Robust to Label Noise: A Loss Correction Approach

3. CVPR-18: Joint Optimization Framework for Learning With Noisy Labels

4. CVPR-18: CleanNet: Transfer Learning for Scalable Image Classifier Training With Label Noise

5. ECCV-18: CurriculumNet: Weakly Supervised Learning from Large scale Web Images

6. ECCV-18: Deep Bilevel Learning

69.9(only use noisy training data) -> 79.9(fine-tuning)

6. NIPS-18: Masking: A new perspective of noisy supervision

7. AAAI-19: Safeguarded Dynamic Label Regression for Noisy Supervision(journal verison)

8. CVPR-19：Probabilistic End-to-End Noise Correction for Learning with Noisy Labels

9. CVPR-19: Learning to Learn from Noisy Labeled Data

10. CVPR-19: Weakly Supervised Image Classification through Noise Regularization

10. CVPR-19: MetaCleaner: Learning to Hallucinate Clean Representations for Noisy-Labeled Visual Recognition

11. ICML-19: Unsupervised Label Noise Modeling and Loss Correction

about 71%

10. NIPS-19: L_DMI: A Novel Information-theoretic Loss Function for Training Deep Nets Robust to Label Noise

11. NIPS-19: Are Anchor Points Really Indispensable in Label-Noise Learning?

10. Meta-Weight-Net: Learning an Explicit Mapping For Sample Weighting

15. ICCV-19: Deep Self-Learning From Noisy Labels

16. ICCV-19: Symmetric Cross Entropy for Robust Learning with Noisy Labels

17. ICCV-19: O2U-Net: A Simple Noisy Label Detection Approach for Deep Neural Networks

18. Arixv-19: Improving MAE’s Fitting Ability while Preserving Its Noise-robustness

19. [ICLR-20: underview: DivideMiX: Learning with noisy labels as semi-supervised learning]

2. WebVison dataset

from: Arxiv17: Webvision database: Visual learning and understanding from web data

• 1,000 classes: concepts in ImageNet ILSVRC12
• noise rate: 20-40%

1. CVPR-18: CleanNet: Transfer Learning for Scalable Image Classifier Training With Label Noise

2. ECCV-18: CurriculumNet: Weakly Supervised Learning from Large scale Web Images

3. ICML-18: MentorNet: Learning Data-Driven Curriculum for Very Deep Neural Networks on Corrupted Labels

use all data or only use the first 50 classes of Google image subset

4. AAAI-19: Safeguarded Dynamic Label Regression for Noisy Supervision

5. Arxiv19: A simple yet effective Baseline for Robust Deep Learning with Noisy Labels

use all 1000 classes

6. ICML-19: Understanding and Utilizing Deep Neural Networks Trained with Noisy Labels

only use the first 50 classes of Google image subset

7. NIPS-19: Combinatorial Inference against Label Noise

only use the randomly selected 100 classes

8. ICLR-20 underreview: IEG: Robust neural networks training to tackel severe label noise

9. ICLR-20: underview: DivideMiX: Learning with noisy labels as semi-supervised learning

3. Food101-N

from: CVPR-18: CleanNet: Transfer Learning for Scalable Image Classifier Training With Label Noise

• 101 food classes
• 310k image, 55k verification,
• noise rate: 20%

1. CVPR-18: CleanNet: Transfer Learning for Scalable Image Classifier Training With Label Noise

2. ECCV-18: CurriculumNet: Weakly Supervised Learning from Large scale Web Images

  not use Food101N created by cleanNet paper, but use Food101 and inject 20% noise

3. CVPR-19: MetaCleaner: Learning to Hallucinate Clean Representations for Noisy-Labeled Visual Recognition

4. ICCV-19: Deep self-Learning from Noisy Labels

5. ICLR-20: underview: Prestopping: How does Early stopping help generalization against label noise

4. ANIMAL10N

from: ICML-19: SELFIE: Refurbishing Unclean Samples for Robust Deep Learning

1. ICML-19: SELFIE: Refurbishing Unclean Samples for Robust Deep Learning

2. ICLR-2020: Prestopping: How Does Early Stopping Help Generalization Against Label Noise?