Image augmentation for machine learning experiments.
The segmentation map augmentation was previously previously a wrapper around heatmap augmentation. This patch introduces independent methods for segmentation map augmentation. This makes the augmentation of such inputs faster and more memory efficient. The internal representation (int instead of floats) also becomes more intuitive.
This improvement leads to some breaking changes. To adapt to the new version, the following steps should be sufficient for most users:
SegmentationMapOnImage
to SegmentationMapsOnImage
(Map -> Maps).SegmentationMapsOnImage.get_arr_int()
to
SegmentationMapsOnImage.get_arr()
.nb_classes
from all calls of SegmentationMapsOnImage
.background_threshold
from all
calls as it is no longer supported.draw_foreground_mask
from all calls of
SegmentationMapsOnImage.draw_on_image()
as it is no longer supported.SegmentationMapsOnImage
is always an
int-like (int, uint or bool). Float arrays are now deprecated.SegmentationMapsOnImage.draw()
and
SegmentationMapsOnImage.draw_on_image()
, as both of these now return a
list of drawn images instead of a single array. (For a segmentation map
array of shape (H,W,C)
they return C
drawn images. In most cases C=1
,
so simply call draw()[0]
or draw_on_image()[0]
.)SegmentationMapsOnImage.arr
is accessed anywhere, the
respective code can handle the new int32
(H,W,#maps)
array form.
Previously, it was float32
and the channel-axis had the same size as the
max class id (+1) that could appear in the map.<augmenter>.augment()
or <augmenter>()
that
provide segmentation maps as numpy arrays (i.e. bypassing
SegmentationMapsOnImage
) use the shape (N,H,W,#maps)
as
(N,H,W)
is no longer supported.numpy 1.17 introduces a new API for random number generation. This patch
adapts imgaug
to automatically use the new API if it is available and
fall back to the old one otherwise. To achieve that, the module
imgaug.random
is introduced, containing the new standard random number
generator imgaug.random.RNG
. You can create a new RNG using a seed value
via RNG(seed)
and it will take care of the rest. It supports all sampling
functions that numpy.random.RandomState
and numpy.random.Generator
support. This new random number generator is now supposed to be used
wherever previously numpy.random.RandomState
would have been used.
(For most users, this shouldn't change anything. Integer seeds are
still supported. If you used RandomState
anywhere, that is also still
supported.)
Breaking changes related to this patch:
imgaug.random.seed()
to set a custom seed.)imgaug.SEED_MIN_VALUE
and imgaug.SEED_MAX_VALUE
were
removed. They are now in imgaug.random
.imgaug.CURRENT_RANDOM_STATE
was removed.
Use imgaug.random.get_global_rng()
instead.numpy 1.17 uses a new implementation of clip()
, which turns int64
values
into float64
values. As a result, it is no longer safe to use int64
in
many augmenters and other functions/methods and hence these inputs are now
rejected. This affects at least ReplaceElementwise
and thereby Dropout
,
CoarseDropout
, Salt
, Pepper
, SaltAndPepper
, CoarseSalt
,
CoarsePepper
and CoarseSaltAndPepper
. See the ReadTheDocs documentation
page about dtype support for more details.
In relation to this change, parameters in imgaug.parameters
that previously
returned int64
were modified to now return int32
instead. Analogously,
float64
results were changed to float32
.
The following new augmenters were added to the library:
Canny edge detection (#316):
imgaug.augmenters.edges.Canny
. Performs canny edge detection and colorizes
the resulting binary image in random ways.Pooling (#317):
imgaug.augmenters.edges.AveragePooling
. Performs average pooling using a
given kernel size. Very similar to AverageBlur
.imgaug.augmenters.edges.MaxPooling
. Performs maximum pooling using a
given kernel size.imgaug.augmenters.edges.MinPooling
. Analogous.imgaug.augmenters.edges.MedianPooling
. Analogous.Hue and Saturation (#210, #319):
imgaug.augmenters.color.WithHueAndSaturation
. Apply child augmenters to
images in HSV
colorspace. Automatically accounts for the hue being in
angular representation.imgaug.augmenters.color.AddToHue
. Adds a defined value to the hue of each
pixel in input images.imgaug.augmenters.color.AddToSaturation
. Adds a defined value to the
saturation of each pixel in input images.imgaug.augmenters.color.MultiplyHueAndSaturation
. Multiplies the hue and/or
saturation of all pixels in input images.imgaug.augmenters.color.MultiplyHue
. Analogous, affects always only the hue.imgaug.augmenters.color.MultiplySaturation
. Analogous, affects always only
the saturation.Color Quantization (#347):
imgaug.augmenters.color.UniformColorQuantization
. Uniformly splits all
possible colors into N
different ones, then finds for each pixel in an
image among the N
colors the most similar one and replaces that pixel's
color with the quantized color.imgaug.augmenters.color.KMeansColorQuantization
. Groups all colors in an
each into N
different ones using k-Means clustering. Then replaces each
pixel'S color, analogously to UniformColorQuantization
.Voronoi (#348):
imgaug.augmenters.segmentation.Voronoi
. Queries a point sampler to
generate a large number of (x,y)
coordinates on an image. Each such
coordinate becomes a voronoi cell. All pixels within the voronoi cell
are replaced by their average color. (Similar to Superpixels
, this
augmenter also supports to only replace p%
of all cells with their
average color.)imgaug.augmenters.segmentation.UniformVoronoi
. Shortcut to call Voronoi
with a uniform points sampler. That sampler places N
points on an image
using uniform distributions (i.e. they are randomly spread over the image.)imgaug.augmenters.segmentation.RegularGridVoronoi
. Shortcut to call
Voronoi
with a regular grid points sampler. That points sampler generates
coordinate on a regular grid with H
rows and W
cols. Some of these points
can be randomly dropped to generate a less regular pattern.imgaug.augmenters.segmentation.RelativeRegularGridVoronoi
. Same as
RegularGridVoronoi
, but instead of using absolute numbers for H
and W
,
they are defined as relative amounts w.r.t. image shapes, leading to more
rows/cols on larger images.One of the long term goals of the library is to move as much augmentation
logic as possible out of Augmenter
instances and into functions. This
patch therefore adds several new augmentation functions:
imgaug.min_pool()
. #369imgaug.median_pool()
. #369augmenters.segmentation.segment_voronoi()
. #348augmenters.flip.fliplr()
. #385augmenters.flip.flipud()
. #385augmenters.color.change_colorspace_()
. #409augmenters.color.change_colorspace_batch_()
. #409augmenters.arithmetic.add_scalar()
. #411augmenters.arithmetic.add_elementwise()
. #411augmenters.arithmetic.replace_elementwise_()
. #411augmenters.arithmetic.compress_jpg()
. #411The color space naming within the library had become rather messy in the past as there were many colorspace-related augmenters, with some of them not using constants for colorspace names/IDs and others defining their own ones. This patch introduces a unified colorspace naming system for which the following constants were added:
imgaug.CSPACE_RGB
imgaug.CSPACE_BGR
imgaug.CSPACE_GRAY
imgaug.CSPACE_CIE
imgaug.CSPACE_YCrCb
imgaug.CSPACE_HSV
imgaug.CSPACE_HLS
imgaug.CSPACE_Lab
imgaug.CSPACE_Luv
imgaug.CSPACE_YUV
imgaug.CSPACE_ALL
All colorspace-related augmenters should now support these constants.
Additionally, support for rarely used colorspaces -- mainly CIE
, YCrCb
,
Luv
and YUV
-- was previously unverified or non-existent. These colorspaces
are now tested for the underlying transformation functions and should be
supported by most colorspace-related augmenters. (Some augmenters may still
define their own subset of actually sensible colorspaces and only accept
these.)
The methods imap_batches()
and imap_batches_unordered()
of
imgaug.multicore.Pool
have now the new argument output_buffer_size
.
The argument set the maximum number of batches that may be handled anywhere
in the augmentation pipeline at a given time (i.e. in the steps "loaded and
waiting", "in augmentation" or "augmented and waiting"). It denotes the
total number of batches over all processes. Setting this argument to
an integer value avoids situations where Pool
eats up all the available
memory due to the data loading and augmentation running faster than the
training.
Augmenter.augment_batches()
now uses a default value of 10*C
for output_buffer_size
, where C
is the number of available logical CPU
cores.
The algorithms for Fliplr
and Flipud
were reworked to be as fast as
possible. In practice this should have no noticeable effects as both augmenters
were already very fast. (#385)
Furthermore, all assert statements within the library were changed from
do_assert()
to standard assert
statements. This is a bit less secure
(as assert
statements can be optimized away), but should have a small
positive impact on the performance. (#387)
Large parts of the library were also refactored to reduce code duplication
and decrease the complexity of many functions. This should make future
improvements easier, but is expected to have a very small negative impact on
the performance due to an increased number of function calls.
It is also expected that numpy 1.17 can make some operations slower. This
is because (a) creating and copying random number generaters has become slower
and (b) clip()
overall seems to be slower.
imgaug uses quite many assert
statements and other checks on input data
to fail early instead of late. This is supposed to improve usability, but that
goal was not always reached as many errors had no associated error
messages. This patch changes that. Now, all assert
statements and other
checks have an associated error message. This should protect users from having
to wade through the library's code in order to understand the root cause of
errors.
Some augmenters were previously defined as functions returning other
augmenters with appropriate settings. This could lead to confusing effects,
where seemingly instantiating an augmenters would lead to the instantiation
of a completely different augmenter. Hence, most of these augmenters were
switched from functions to classes. (The classes are now inheriting from the
previously returned augmenters, i.e. instanceof
checks should still work.)
This affects: AdditiveGaussianNoise
, AdditiveLaplaceNoise
,
AdditivePoissonNoise
, Dropout
, CoarseDropout
, ImpulseNoise
,
SaltAndPepper
, CoarseSaltAndPepper
, Salt
, CoarseSalt
, Pepper
,
CoarsePepper
, SimplexNoiseAlpha
, FrequencyNoiseAlpha
, MotionBlur
,
MultiplyHueAndSaturation
, MultiplyHue
, MultiplySaturation
, AddToHue
,
AddToSaturation
, Grayscale
, GammaContrast
, SigmoidContrast
,
LogContrast
, LinearContrast
, Sharpen
, Emboss
, EdgeDetect
,
DirectedEdgeDetect
, OneOf
, AssertLambda
, AssertShape
, Pad
, Crop
,
Clouds
, Fog
and Snowflakes
.
Not yet switched are: InColorspace
(deprecated),
ContrastNormalization
(deprecated), HorizontalFlip
(pure alias
for Fliplr
), VerticalFlip
(pure alias for Flipud
)
and Scale
(deprecated).
Feeding images with height and/or width of 0
or a channel axis of size 0
into augmenters would previously often result in crashes. This was also the
case for input arrays with more than 512
channels. Some of these errors
also included segmentation faults or endlessly hanging programs. Most
augmenters and helper functions were modified to be more robust towards
such unusual inputs and will no longer crash.
It is still good practice to avoid such inputs. Note e.g. that some helper functions -- like drawing routines -- may still crash. The unittests corresponding to this change also only cover image data. Using other inputs, e.g. segmentation maps, might still induce problems.
The following (public) functions were added to the library (not listing functions that were already mentioned above):
imgaug.is_np_scalar()
. #366dtypes.normalize_dtypes()
. #366dtypes.normalize_dtype()
. #366dtypes.change_dtypes_()
. #366dtypes.change_dtype_()
. #366dtypes.increase_itemsize_of_dtype()
. #366imgaug.warn()
function. #367imgaug.compute_paddings_to_reach_multiples_of()
. #369imgaug.pad_to_multiples_of()
. #369augmentables.utils.copy_augmentables
. #410validation.convert_iterable_to_string_of_types()
. #413validation.is_iterable_of()
. #413validation.assert_is_iterable_of()
. #413random.supports_new_rng_style()
. #375random.get_global_rng()
. #375random.seed()
. #375random.normalize_generator()
. #375random.normalize_generator_()
. #375random.convert_seed_to_generator()
. #375random.convert_seed_sequence_to_generator()
. #375random.create_pseudo_random_generator_()
. #375random.create_fully_random_generator()
. #375random.generate_seed_()
. #375random.generate_seeds_()
. #375random.copy_generator()
. #375random.copy_generator_unless_global_generator()
. #375random.reset_generator_cache_()
. #375random.derive_generator_()
. #375random.derive_generators_()
. #375random.get_generator_state()
. #375random.set_generator_state_()
. #375random.is_generator_equal_to()
. #375random.advance_generator_()
. #375random.polyfill_integers()
. #375random.polyfill_random()
. #375The following (public) classes were added (not listing classes that were already mentioned above):
augmenters.edges.IBinaryImageColorizer
. #316augmenters.edges.RandomColorsBinaryImageColorizer
. #316augmenters.segmentation.IPointsSampler
. #348augmenters.segmentation.RegularGridPointsSampler
. #348augmenters.segmentation.RelativeRegularGridPointsSampler
. #348augmenters.segmentation.DropoutPointsSampler
. #348augmenters.segmentation.UniformPointsSampler
. #348augmenters.segmentation.SubsamplingPointsSampler
. #348testutils.ArgCopyingMagicMock
. #413The image colorization is used for Canny
to turn binary images into color
images.
The points samplers are currently used within Voronoi
.
Due to fast growth of the library in the past, a significant amount of messy code had accumulated. To fix that, a lot of time was spend to refactor the code throughout the whole library to reduce code duplication and improve the general quality. This also included a rewrite of many outdated docstrings. There is still quite some mess remaining, but the current state should make it somewhat easier to add future improvements.
As part of the refactorings, a few humongously large unittests were also split up into many smaller tests. The library has now around 3000 unique unittests (i.e. each unittest function is counted once, even it is called many times with different parameters).
Related PRs:
The following functions/classes/arguments are now deprecated:
imgaug.augmenters.meta.clip_augmented_image_
.
Use imgaug.dtypes.clip_()
or numpy.clip()
instead. #398imgaug.augmenters.meta.clip_augmented_image
.
Use imgaug.dtypes.clip_()
or numpy.clip()
instead. #398imgaug.augmenters.meta.clip_augmented_images_
.
Use imgaug.dtypes.clip_()
or numpy.clip()
instead. #398imgaug.augmenters.meta.clip_augmented_images
.
Use imgaug.dtypes.clip_()
or numpy.clip()
instead. #398imgaug.normalize_random_state
.
Use imgaug.random.normalize_generator
instead. #375imgaug.current_random_state
.
Use imgaug.random.get_global_rng
instead. #375imgaug.new_random_state
.
Use class imgaug.random.RNG
instead. #375imgaug.dummy_random_state
.
Use imgaug.random.RNG(1)
instead. #375imgaug.copy_random_state
.
Use imgaug.random.copy_generator
instead.imgaug.derive_random_state
.
Use imgaug.random.derive_generator_
instead. #375imgaug.normalize_random_states
.
Use imgaug.random.derive_generators_
instead. #375imgaug.forward_random_state
.
Use imgaug.random.advance_generator_
instead. #375imgaug.augmenters.arithmetic.ContrastNormalization
.
Use imgaug.augmenters.contrast.LinearContrast
instead. #396X
in imgaug.augmentables.kps.compute_geometric_median()
.
Use argument points
instead. #402cval
in imgaug.pool()
, imgaug.avg_pool()
and
imgaug.max_pool()
. Use pad_cval
instead. #369The following changes were made to the dependencies of the library:
scikit-image
to
0.14.2
. #377, #399opencv-python
to opencv-python-headless
.
This should improve support for some system without GUIs. #324pytest-subtests
for the library's unittests. #366The library was added to conda-forge
so that it can now be installed via
conda install imgaug
. (The conda-forge channel must be added first,
see installation docs or README.) #320 #339
Polygon.clip_out_of_image()
,
which would lead to exceptions if a polygon had overlap with an image,
but not a single one of its points was inside that image plane.multicore
methods falsely not accepting
augmentables.batches.UnnormalizedBatch
.Rot90
now uses subpixel-based coordinate remapping.
I.e. any coordinate (x, y)
will be mapped to (H-y, x)
for a rotation by
90deg.
Previously, an integer-based remapping to (H-y-1, x)
was used.
Coordinates are e.g. used by keypoints, bounding boxes or polygons.augmenters.arithmetic.Invert
min_value
and/or max_value
arguments were
set, uint64
is no longer a valid input array dtype for Invert
.
This is due to a conversion to float64
resulting in loss of resolution.Invert
in rare cases restoring dtypes improperly.dtypes.gate_dtypes()
crashing if the input was one or more numpy
scalars instead of numpy arrays or dtypes.augmenters.geometric.PerspectiveTransform
producing invalid
polygons (more often with higher scale
values). #338external/poly_point_isect_py2py3.py
related to
floating point inaccuracies (changed an epsilon from 1e-10
to 1e-4
,
rounded some floats). #338Superpixels
breaking when a sampled n_segments
was <=0
.
n_segments
is now treated as 1
in these cases.ReplaceElementwise
both allowing and disallowing dtype int64
. #346BoundingBox.deepcopy()
creating only shallow copies of labels. #356dtypes.change_dtypes_()
#366
round
being ignored if input images were a list.dtypes.get_minimal_dtype()
failing if argument arrays
contained
not exactly two items. #366CloudLayer.get_parameters()
resulting in errors. #309SimplexNoiseAlpha
and FrequencyNoiseAlpha
not handling
sigmoid
argument correctly. #343SnowflakesLayer
crashing for grayscale images. #345Affine
heatmap augmentation crashing for arrays with more than
four channels and order!=0
. #381Affine
. #381Polygon.clip_out_of_image()
crashing if the intersection between
polygon and image plane was an edge or point. #382Polygon.clip_out_of_image()
potentially failing for polygons
containing two or fewer points. #382Polygon.is_out_of_image()
returning wrong values if the image plane
was fully contained inside the polygon with no intersection between the
image plane and the polygon edge. #382Fliplr
and Flipud
using for coordinate-based inputs and image-like
inputs slightly different conditions for when to actually apply
augmentations. #385Convolve
using an overly restrictive check when validating inputs
for matrix
w.r.t. whether they are callables. The check should now also
support class methods (and possibly various other callables). #407CropAndPad
, Pad
and PadToFixedSize
still clipping cval
samples
to the uint8
. They now clip to the input array's dtype's value range. #407WithColorspace
not propagating polygons to child augmenters. #409WithHueAndSaturation
not propagating segmentation maps and polygons
to child augmenters. #409AlphaElementwise
to blend coordinates (for keypoints, polygons,
line strings) on a point-by-point basis following the image's average
alpha value in the sampled alpha mask of the point's coordinate.
Previously, the average over the whole mask was used and then either all
points of the first branch or all of the second branch were used as the
augmentation output. This also affects SimplexNoiseAlpha
and
FrequencyNoiseAlpha
. #4100
or the channels
were >512
. See further above for more details. #433Rot90
not supporting imgaug.ALL
. #434PiecewiseAffine
possibly generating samples for non-image data
when using absolute_scale=True
that were not well aligned with the
corresponding images. #437This update mainly covers the following topics:
For the Polygon and Line String augmentation, new classes and methods had to be added. The previous file for that was imgaug/imgaug.py
, which however was already fairly large. Therefore, all classes and methods related to augmentable data were split off and moved to imgaug/augmentables/<type>.py
. The new modules and their main contents are:
imgaug.augmentables.batches
: Contains Batch
, UnnormalizedBatch
.imgaug.augmentables.utils
: Contains utility functions.imgaug.augmentables.bbs
: Contains BoundingBox
, BoundingBoxesOnImage
.imgaug.augmentables.kps
: Contains Keypoint
, KeypointsOnImage
.imgaug.augmentables.polys
: Contains Polygon
, PolygonsOnImage
.imgaug.augmentables.lines
: Contains LineString
, LineStringsOnImage
.imgaug.augmentables.heatmaps
: Contains HeatmapsOnImage
.imgaug.augmentables.segmaps
: Contains SegmentationMapOnImage
.Currently, all augmentable classes can still be created via imgaug.<type>
, e.g. imgaug.BoundingBox
still works.
Changes related to the new modules:
Keypoint
, KeypointsOnImage
and imgaug.imgaug.compute_geometric_median
to augmentables/kps.py
.BoundingBox
, BoundingBoxesOnImage
to augmentables/bbs.py
.Polygon
, PolygonsOnImage
and related classes/functions to augmentables/polys.py
.HeatmapsOnImage
to augmentables/heatmaps.py
.SegmentationMapOnImage
to augmentables/segmaps.py
.Batch
to augmentables/batches.py
.imgaug.augmentables.utils
.
normalize_shape()
.project_coords()
._interpolate_points()
, _interpolate_point_pair()
and _interpolate_points_by_max_distance()
to imgaug.augmentables.utils
and made them public functions.__init__()
of PolygonsOnImage
, BoundingBoxesOnImage
, KeypointsOnImage
to make use of imgaug.augmentables.utils.normalize_shape()
.KeypointsOnImage.on()
to use imgaug.augmentables.utils.normalize_shape()
.Keypoint.project()
to use imgaug.augmentables.utils.project_coords()
.Polygons were already part of imgaug
for quite a while, but couldn't be augmented yet. This version adds methods to perform such augmentations. It also makes some changes to the Polygon
class, see the list of changes below.
Example for polygon augmentation:
import imgaug as ia
import imgaug.augmenters as iaa
from imgaug.augmentables.polys import Polygon, PolygonsOnImage
image = ia.quokka(size=0.2)
psoi = PolygonsOnImage([
Polygon([(0, 0), (20, 0), (20, 20)])
], shape=image.shape)
image_aug, psoi_aug = iaa.Affine(rotate=45).augment(
images=[image],
polygons=[psoi]
)
See imgaug-doc/notebooks for a jupyter notebook with many more examples.
Changes related to polygon augmentation:
_ConcavePolygonRecoverer
to imgaug.augmentables.polys
.PolygonsOnImage
to imgaug.augmentables.polys
.augment_polygons()
to Augmenter
._augment_polygons()
to Augmenter
._augment_polygons_as_keypoints()
to Augmenter
.polygons
to imgaug.augmentables.batches.Batch
.polygons_aug
and polygons_unaug
to imgaug.augmentables.batches.Batch
.Augmenter.augment_batches()
.Polygon.height
.Polygon.width
.Polygon.to_keypoints()
.Polygon.draw_on_image()
and PolygonsOnImage.draw_on_image()
.raise_if_too_far_away=True
to Polygon.change_first_point_by_coords()
.imgaug.quokka_polygons()
function to generate example polygon data.Polygon.draw_on_image()
, PolygonsOnImage.draw_on_image()
LineString
methods.size
and size_perimeter
to control polygon line thickness.alpha_perimeter
to alpha_line
, color_perimeter
to color_line
to align with LineStrings
.alpha_fill
to alpha_face
and color_fill
to color_face
.Polygon.clip_out_of_image()
from MultiPolygon
to list
of Polygon
.
This breaks for anybody who has already used Polygon.clip_out_of_image()
.Polygon.exterior_almost_equals()
to accept lists of tuples as argument other_polygon
.color
and alpha
in Polygon.draw_on_image()
and PolygonsOnImage.draw_on_image()
to represent
the general color and alpha of the polygon. The colors/alphas of the inner area, perimeter and points are derived from
color
and alpha
(unless color_inner
, color_perimeter
or color_points
are set (analogous for alpha)).Polygon.project()
to use LineString.project()
.Polygon.shift()
to use LineString.shift()
.Polygon.exterior_almost_equals()
, Polygon.almost_equals()
LineString.coords_almost_equals()
.interpolate
to points_per_edge
.other_polygon
to other
.color_line
to color_lines
, alpha_line
to alpha_lines
in Polygon.draw_on_image()
and PolygonsOnImage.draw_on_image()
.Polygon.clip_out_of_image(image)
not handling image
being a tuple.Polygon.is_out_of_image()
falsely only checking the corner points of the polygon.This version adds Line String augmentation. Line Strings are simply lines made up of consecutive corner points that are connected by straight lines. Line strings have similarity with polygons, but do not have a filled inner area and are not closed (i.e. first and last coordinate differ).
Similar to other augmentables, line string are represented with the classes LineString(<iterable of xy-coords>)
and LineStringsOnImage(<iterable of LineString>, <shape of image>)
. They are augmented e.g. via Augmenter.augment_line_strings(<iterable of LineStringsOnImage>)
or Augmenter.augment(images=..., line_strings=...)
.
Example:
import imgaug as ia
import imgaug.augmenters as iaa
from imgaug.augmentables.lines import LineString, LineStringsOnImage
image = ia.quokka(size=0.2)
lsoi = LineStringsOnImage([
LineString([(0, 0), (20, 0), (20, 20)])
], shape=image.shape)
image_aug, lsoi_aug = iaa.Affine(rotate=45).augment(
images=[image],
line_strings=[lsoi]
)
See imgaug-doc/notebooks for a jupyter notebook with many more examples.
Augmentation of different data corresponding to the same image(s) has been a bit convoluted in the past, as each data type had to be augmented on its own. E.g. to augment an image and its bounding boxes, one had to first switch the augmenters to deterministic mode, then augment the images, then the bounding boxes. This version adds methods that perform these steps in one call. Specifically, Augmenter.augment(...)
is used for that, which has the alias Augmenter.__call__(...)
. One argument can be used for each augmentable, e.g. bounding_boxes=<bounding box data>
.
Example:
import imgaug as ia
import imgaug.augmenters as iaa
from imgaug.augmentables.kps import Keypoint, KeypointsOnImage
image = ia.quokka(size=0.2)
kpsoi = KeypointsOnImage([Keypoint(x=0, y=10), Keypoint(x=10, y=5)],
shape=image.shape)
image_aug, kpsoi_aug = iaa.Affine(rotate=(-45, 45)).augment(
image=image,
keypoints=kpsoi
)
This will automatically make sure that image and keypoints are rotated by corresponding amounts.
Normalization methods have been added to that class, which allow it to process many more different inputs than just variations of *OnImage
.
Example:
import imgaug as ia
import imgaug.augmenters as iaa
image = ia.quokka(size=0.2)
kps = [(0, 10), (10, 5)]
image_aug, kps_aug = iaa.Affine(rotate=(-45, 45)).augment(
image=image, keypoints=kps)
Examples for other inputs that are automatically handled by augment()
:
list([N,4] ndarray)
for bounding boxes. (One list for images,
then N
bounding boxes in (x1,y1,x2,y2)
form.)list(list(list(tuple)))
for line strings. (One list for images,
one list for line strings on the image, one list for coordinates within
the line string. Each tuple must contain two values for xy-coordinates.)list(list(imgaug.augmentables.polys.Polygon))
for polygons.
Note that this "skips" imgaug.augmentables.polys.PolygonsOnImage
.In python <3.6, augment()
is limited to a maximum of two inputs/outputs and if two inputs/outputs are used, then one of them must be image data and such (augmented) image data will always be returned first,
independent of the argument's order. E.g. augment(line_strings=<data>, polygons=<data>)
would be invalid due to not containing image data. augment(polygons=<data>, images=<data>)
would still return the images first, even though they are the second argument.
In python >=3.6, augment()
may be called with more than two arguments and will respect their order.
Example:
import numpy as np
import imgaug as ia
import imgaug.augmenters as iaa
image = ia.quokka(size=0.2)
kps = [(0, 10), (10, 5)]
heatmap = np.zeros((image.shape[0], image.shape[1]), dtype=np.float32)
rotate = iaa.Affine(rotate=(-45, 45))
heatmaps_aug, images_aug, kps_aug = rotate(
heatmaps=[heatmap],
images=[image],
keypoints=[kps]
)
To use more than two inputs/outputs in python <3.6, add the argument return_batch=True
, which will return an instance of imgaug.augmentables.batches.UnnormalizedBatch
.
Changes related to the augmentation interface:
Augmenter.augment()
method.Augmenter.augment_batch()
method.
Augmenter.augment_batches()
and multicore routines.imgaug.augmentables.batches.UnnormalizedBatch
.imgaug.augmentables.normalization
for data normalization routines.augment_batches()
:
UnnormalizedBatch
as input. It is automatically normalized before augmentation and unnormalized afterwards.
This allows to use Batch
instances with non-standard datatypes.Batch
(and UnnormalizedBatch
).Batch
(and non-UnnormalizedBatch
) inputs to augment_batches()
as deprecated.Batch.deepcopy()
batch.deepcopy(images_aug=...)
.Keypoint augmentation
Keypoint.draw_on_image()
.alpha
argument to KeypointsOnImage.draw_on_image()
. This can break code that relied on the order of arguments of the method (though will usually only have visual consequences).KeypointsOnImage
and Keypoint
copying:
keypoints
and shape
to KeypointsOnImage.deepcopy()
.keypoints
and shape
to KeypointsOnImage.copy()
.Keypoint.copy()
.Keypoint.deepcopy()
.
Keypoint
to use deepcopy()
to create copies of itself (instead of instantiating new instances via Keypoint(...)
).KeypointsOnImage.deepcopy()
now uses Keypoint.deepcopy()
to create Keypoint copies, making it more flexible.KeypointsOnImage
to use KeypointsOnImage.deepcopy()
in as many methods as possible to create copies of itself.Affine
, AffineCv2
, PiecewiseAffine
, PerspectiveTransform
, ElasticTransformation
, Rot90
to use KeypointsOnImage.deepcopy()
and Keypoint.deepcopy()
during keypoint augmentation.Keypoint.draw_on_image()
to draw a rectangle for the keypoint so long as any part of that rectangle is within the image plane. (Previously, the rectangle was only drawn if the integer xy-coordinate of the point was inside the image plane.)KeypointsOnImage.draw_on_image()
to raise an error if an input image has shape (H,W)
.KeypointsOnImage.draw_on_image()
to handle single-number inputs for color
.KeypointsOnImage.from_coords_array()
from_xy_array()
.coords
to xy
.staticmethod
to classmethod
.KeypointsOnImage.get_coords_array()
to_xy_array()
.KeypointsOnImage.draw_on_image()
to use Keypoint.draw_on_image()
.Heatmap augmentation
Affine
, PiecewiseAffine
, ElasticTransformation
to always use order=3
for heatmap augmentation.HeatmapsOnImage
that validates whether the input array is within the desired value range [min_value, max_value]
from a hard exception to a soft warning (with clipping). Also improved the error message a bit.Deprecation warnings:
imgaug.imgaug.DeprecationWarning
. The builtin python DeprecationWarning
is silent since 2.7, which is why now a separate deprecation warning is used.imgaug.imgaug.warn_deprecated()
.
imgaug.imgaug.deprecated
decorator.
Bounding Boxes:
BoundingBox.extract_from_image()
the arguments pad
and pad_max
.BoundingBox.contains()
to also accept Keypoint
.BoundingBox.project(from, to)
to also accept images instead of shapes.thickness
in BoundingBox.draw_on_image()
to size
in order to match the name used for keypoints, polygons and line strings. The argument thickness
will still be accepted, but raises a deprecation warning.thickness
in BoundingBoxesOnImage.draw_on_image()
to size
in order to match the name used for keypoints, polygons and line strings. The argument thickness
will still be accepted, but raises a deprecation warning.BoundingBox
to reduce code repetition.BoundingBox.extract_from_image()
. Improved some code fragments that looked wrong.BoundingBoxesOnImage.draw_on_image()
to improve efficiency by evading unnecessary array copies.Other:
cval
and mode
to PerspectiveTransform
(PR #301). This breaks code that relied on the order of the arguments and used keep_size
, name
, deterministic
or random_state
as positional arguments.dtypes.clip_()
function.imgaug.imgaug.flatten()
that flattens nested lists/tuples.PerspectiveTransform
to ensure minimum height and width of output images (by default 2x2
). This prevents errors in polygon augmentation (possibly also in keypoint augmentation).imgaug.augmenters.blend.blend_alpha()
to no longer enforce a channel axis for foreground and background image.imgaug/parameters.py
to reorder classes within the file.requirements.txt
.blend.blend_alpha()
if dtype numpy.float128 does not exist.ChangeColorspace
when cv2.COLOR_Lab2RGB
was actually called cv2.COLOR_LAB2RGB
in the local OpenCV installation (analogous for BGR). (PR #263)ReplaceElementwise
always sampling replacement per channel.draw_text()
due to arrays that could not be set to writeable after drawing the text via PIL.parameters.Subtract
.angle_between_vectors()
.KeypointsOnImage
instances
Rot90
not changing KeypointsOnImage.shape
if .keypoints
was empty.Affine
not changing KeypointsOnImage.shape
if .keypoints
was empty.PerspectiveTransform
not changing KeypointsOnImage.shape
if .keypoints
was empty.Resize
not changing KeypointsOnImage.shape
if .keypoints
was empty.CropAndPad
not changing KeypointsOnImage.shape
if .keypoints
was empty. (Same for Crop
, Pad
.)PadToFixedSize
not changing KeypointsOnImage.shape
if .keypoints
was empty.CropToFixedSize
not changing KeypointsOnImage.shape
if .keypoints
was empty.KeepSizeByResize
not changing KeypointsOnImage.shape
if .keypoints
was empty.Affine
heatmap augmentation producing arrays with values outside the range [0.0, 1.0]
when order
was set to 3
.PiecewiseAffine
heatmap augmentation producing arrays with values outside the range [0.0, 1.0]
when order
was set to 3
.SegmentationMapOnImage
falsely checking if max class index is <= nb_classes
instead of < nb_classes
.dtypes.clip_to_value_range_()
and dtypes.restore_dtypes_()
causing errors when clip value range exceeded array dtype's value range.dtypes.clip_to_value_range_()
and dtypes.restore_dtypes_()
when the input array was scalar, i.e. had shape ()
.JpegCompression
on windows (possibly also affected other systems). #297