Mask R-CNN Model to detect the area of damage on a car. The rationale for such a model is that it can be used by insurance companies for faster processing of claims if users can upload pics and they can assess damage from them. This model can also be used by lenders if they are underwriting a car loan especially for a used car.
This is an implementation of Mask R-CNN on Python 3, Keras, and TensorFlow. The model generates bounding boxes and segmentation masks for each instance of car in the image. It's based on Feature Pyramid Network (FPN) and a ResNet50/ResNet101 backbone.
The rationale for such a model is that it can be used by insurance companies for faster processing of claims if users can upload pics and they can assess damage from them. This model can also be used by lenders if they are underwriting a car loan especially for a used car.
Mask R-CNN, extends Faster R-CNN by adding a branch for predicting an object mask in parallel with the existing branch for bounding box recognition. Mask R-CNN is an instance segmentation model that allows identifying pixel wise location for predefined class. Mask R-CNN is different from classical object detection models like Faster R-CNN where, in addition to identifying the class and its bounding box location, it can also color pixels in the bounding box that correspond to that class.
A detailed description of how Mask R-CNNN works in available at How Mask R-CNN works
To help with debugging and understanding the model, car-damage-detection.ipynb.ipynb provides visualizations and allow running the model step by step to inspect the output at each point. Here are the explanations:
The dataset is made up of 68 images (50 train + 16 validation + 2 test) from the internet.
Sample Image 1 | Sample Image 2 |
---|---|
A Mask R-CNN model requires the images in training dataset to be annotated, which is to have the region of damage in an image identified and the boundary of the damaged area marked accurately. An example of annotation tool is the VGG Image Annotator(VIA). The annotations is then saved as a json format file in the dataset directory.
This custom Mask R-CNN is built on top of Matterport Mask R-CNN Github repository.
Images and annotations from dataset will be loaded by the load_custom() function in CustomDataset Class.
Visualization of mask on the annotated image is done in car-damage-detection.ipynb
Training and evaluation code is in custom.py
.
The mask_rcnn_coco.h5, a pre-trained weights on coco dataset is used to train this model.
You can import this module in Jupyter notebook (see the provided notebooks for examples) or you can run it directly from the command line as such:
# Train a new model starting from pre-trained COCO weights
python custom.py train --dataset="D:/Tech Projects/car-damage-detection/Mask_RCNN_damage_detector/" --weights=coco
# Train a new model starting from ImageNet weights
python custom.py train --dataset="D:/Tech Projects/car-damage-detection/Mask_RCNN_damage_detector/" --weights=imagenet
# Continue training the last model you trained. This will find
# the last trained weights in the model directory.
python custom.py train --dataset="D:/Tech Projects/car-damage-detection/Mask_RCNN_damage_detector/" --weights=last
You can also run the COCO evaluation code with:
# Run COCO evaluation on the last trained model
python custom.py evaluate --dataset="D:/Tech Projects/car-damage-detection/Mask_RCNN_damage_detector/" --weights=last
The training schedule, learning rate, and other parameters should be set in custom.py
.
The model weights is inspected in car-damage-detection.ipynb.ipynb.Link last training checkpoint for model for validation. This step performs a sanity check if your weights and biases are properly distributed.
Inspect model by performing prediction on test and validation to test the accuracy.
The repository includes:
Python 3.7, TensorFlow 1.3, Keras 2.0.8 and other common packages listed in requirements.txt
.
To train or test on MS COCO, you'll also need:
If you use Docker, the code has been verified to work on this Docker container.
Clone this repository
Install dependencies
pip3 install -r requirements.txt
Run setup from the repository root directory
python3 setup.py install
Download pre-trained COCO weights (mask_rcnn_coco.h5) from the releases page.
(Optional) To train or test on MS COCO install pycocotools
from one of these repos. They are forks of the original pycocotools with fixes for Python3 and Windows (the official repo doesn't seem to be active anymore).