Includes: Learning data augmentation strategies for object detection | GridMask data augmentation | Augmentation for small object detection in Numpy. Use RetinaNet with ResNet-18 to test these methods on VOC and KITTI.
This project is built for testing multiple data augmentations for object detection:
Zoph B, Cubuk E D, Ghiasi G, et al. Learning data augmentation strategies for object detection[J]. arXiv preprint arXiv:1906.11172, 2019. pdf | github
Chen P. GridMask data augmentation[J]. arXiv preprint arXiv:2001.04086, 2020. pdf | github
Kisantal M, Wojna Z, Murawski J, et al. Augmentation for small object detection[J]. arXiv preprint arXiv:1902.07296, 2019. pdf
Make sure the file "/augmentation_zoo/Myautoaugment_utils.py" is in project folder.
from Myautoaugment_utils import AutoAugmenter
# if you want to use the learned augmentation policy custom or v0-v4(v4 was recommended):
autoaugmenter = AutoAugmenter('v4')
# or if you want to use some spatial transformation or color distortion data augmentation,
# add the data augmentation method that you want to use to the policy_test in Myautoaugment_utils.py
# and set the prob and magnitude. For excample:
# def policy_vtest():
# policy = [
# [('Color', 0.0, 6), ('Cutout', 0.6, 8)],
# ]
# return policy
autoaugmenter = AutoAugmenter('test')
# Input:
# Sample = {'img': img, 'annot': annots}
# img = [H, W, C], RGB, value between [0,1]
# annot = [xmin, ymin, xmax, ymax, label]
# Return:
# Sample = {'img': img, 'annot': annots}
Sample = autoaugmenter(Sample)
# Use in Pytorch
dataset = Dataset(root, transform=transforms.Compose([autoaugmenter]))
Make sure the file "/augmentation_zoo/MyGridMask.py" is in project folder. And the input and output requirements are same as above
from MyGridMask import GridMask
GRID = False
GRID_ROTATE = 1
GRID_OFFSET = 0
GRID_RATIO = 0.5
GRID_MODE = 1
GRID_PROB = 0.5
Gridmask = GridMask(True, True, GRID_ROTATE,GRID_OFFSET,GRID_RATIO,GRID_MODE,GRID_PROB)
Sample = Gridmask(Sample)
This method includes 3 Copy-Pasting Strategies:
I code it in this way:
Algorithm: Augmentation for small object detection
Input: Sample x, Policy v, Threshold thresh, Prob prob
function SmallObjectAugmentation(x, v, thresh, prob)
Perform the function with the probability of prob
img, annots = x[‘img’], x[‘annot’]
for annot in annots do
if issmallobject(annot, thresh) do
small_object_list.append(annot)
end if
end for
copy_times and copy_object_num were decided by v
shuffle the small_object_list
for idx in range(copy_object_num) do
to_be_copied_annot = small_object_list[idx]
for _ in range(copy_times) do
new_annot = create_copy_annot(to_be_copied_annot, annots)
if new_annot is not None do
img = add_patch_in_img(new_annot, to_be_copied_annot, img)
annots.append(new_annot)
end if
end for
end for
return {‘img’: img, ‘annot’:annots}
End function
To use this method, make sure the file "/augmentation_zoo/SmallObjectAugmentation.py" is in project folder. And the input and output requirements are same as above
""" SMALL OBJECT AUGMENTATION """
# Defaultly perform Policy 2, if you want to use
# Policy 1, make SOA_ONE_OBJECT = Ture, or if you
# want to use Policy 3, make SOA_ALL_OBJECTS = True
SOA_THRESH = 64*64
SOA_PROB = 1
SOA_COPY_TIMES = 3
SOA_EPOCHS = 30
SOA_ONE_OBJECT = False
SOA_ALL_OBJECTS = False
augmenter = SmallObjectAugmentation(SOA_THRESH, SOA_PROB, SOA_COPY_TIMES, SOA_EPOCHS, SOA_ALL_OBJECTS, SOA_ONE_OBJECT)
Sample = augmenter(Sample)
I use the RetinaNet with ResNet-18, testing in VOC and KITTI. VOC_BATCH_SIZE = 8, KITTI_BATCH_SIZE = 24
DataSets | No Augmentation | Random Flip | Autoaugmenter('v1') | Autoaugmenter('v4') | GridMask | Small Object Augmentation |
---|---|---|---|---|---|---|
VOC | 0.61492 | 0.63738 | 0.63651 | 0.62267 | 0.65605 | 0.63870 |
KITTI | 0.60375 | 0.63077 | 0.58631 | 0.64347 | 0.71868 | 0.62622 |
KITTI | Car | van | truck | pedestrian | Person_sitting | cyclist | Tram | Misc | mAP |
---|---|---|---|---|---|---|---|---|---|
No Augmenation | 0.80197 | 0.64498 | 0.84922 | 0.52134 | 0.27078 | 0.50485 | 0.79602 | 0.47798 | 0.60375 |
Random Flip | 0.82147 | 0.66679 | 0.85982 | 0.54333 | 0.35177 | 0.53576 | 0.80296 | 0.46428 | 0.63077 |
AutoAugmenter(‘v1’) | 0.82838 | 0.55684 | 0.74368 | 0.55461 | 0.38256 | 0.48259 | 0.74964 | 0.39217 | 0.58631 |
AutoAugmenter(‘v4’) | 0.82566 | 0.64851 | 0.87592 | 0.54426 | 0.40881 | 0.56872 | 0.80106 | 0.47480 | 0.64347 |
GridMask | 0.85529 | 0.75980 | 0.91472 | 0.59351 | 0.51708 | 0.62842 | 0.86256 | 0.61804 | 0.71867 |
Small Object Augmentation | 0.83064 | 0.60390 | 0.85146 | 0.55325 | 0.42748 | 0.50965 | 0.76821 | 0.46511 | 0.62621 |