RETFound - A foundation model for retinal imaging

Key Features

This is the repo for the paper RETFound: a foundation model for generalizable disease detection from retinal images led by Yukun Zhou, Daniel C. Alexander, and Pearse A. Keane:

• RETFound is pre-trained on 1.6 million retinal images with self-supervised learning
• RETFound has been validated in multiple disease detection tasks
• RETFound can be efficiently adapted to customised tasks

Links

• Paper
• Model
• Code

Details

Medical artificial intelligence (AI) offers great potential for recognizing signs of health conditions in retinal images and expediting the diagnosis of eye diseases and systemic disorders. However, the development of AI models requires substantial annotation and models are usually task-specific with limited generalizability to different clinical applications. Here, we present RETFound, a foundation model for retinal images that learns generalizable representations from unlabelled retinal images and provides a basis for label-efficient model adaptation in several applications. Specifically, RETFound is trained on 1.6 million unlabelled retinal images by means of self-supervised learning and then adapted to disease detection tasks with explicit labels. We show that adapted RETFound consistently outperforms several comparison models in the diagnosis and prognosis of sight-threatening eye diseases, as well as incident prediction of complex systemic disorders such as heart failure and myocardial infarction with fewer labelled data. RETFound provides a generalizable solution to improve model performance and alleviate the annotation workload of experts to enable broad clinical AI applications from retinal imaging.

Dataset Links

• IDRID
• MESSIDOR-2
• APTOS-2019
• PAPILA
• Glaucoma Fundus
• JSIEC
• Retina
• OCTID
• UK Biobank

Get Started

Main Requirements

torch==1.8.1
pandas==0.25.3
timm==0.3.2
tqdm

Installation

conda create -n retfound python=3.7.5 -y
conda activate retfound

cd RETFound_MAE
pip install -r requirement.txt

Download Model

Data Preparation

Organise your data into this directory structure (using IDRiD as an example)

Training

python -m torch.distributed.launch --nproc_per_node=1 --master_port=48798 main_finetune.py \
    --batch_size 16 \
    --world_size 1 \
    --model vit_large_patch16 \
    --epochs 50 \
    --blr 5e-3 --layer_decay 0.65 \
    --weight_decay 0.05 --drop_path 0.2 \
    --nb_classes 5 \
    --data_path ./IDRiD_data/ \
    --task ./finetune_IDRiD/ \
    --finetune ./RETFound_cfp_weights.pth

Testing

python -m torch.distributed.launch --nproc_per_node=1 --master_port=48798 main_finetune.py \
    --eval --batch_size 16 \
    --world_size 1 \
    --model vit_large_patch16 \
    --epochs 50 \
    --blr 5e-3 --layer_decay 0.65 \
    --weight_decay 0.05 --drop_path 0.2 \
    --nb_classes 5 \
    --data_path ./IDRiD_data/ \
    --task ./internal_IDRiD/ \
    --resume ./finetune_IDRiD/checkpoint-best.pth

🙋‍♀️ Feedback and Contact

Please contact [email protected] or [email protected] if you have questions.

🛡️ License

This project is under the CC-BY-NC 4.0 license. See LICENSE for details.

🙏 Acknowledgement

The code is based on MAE. We thank the authors for releasing the code. We also appreciate the data organisers for sharing their data.

📝 Citation

If you find this repository useful, please consider citing this paper:

@article{zhou2023foundation,
  title={A foundation model for generalizable disease detection from retinal images},
  author={Zhou, Yukun and Chia, Mark A and Wagner, Siegfried K and Ayhan, Murat S and Williamson, Dominic J and Struyven, Robbert R and Liu, Timing and Xu, Moucheng and Lozano, Mateo G and Woodward-Court, Peter and others},
  journal={Nature},
  pages={1--8},
  year={2023},
  publisher={Nature Publishing Group UK London}
}