Datashader Versions Save

Quickly and accurately render even the largest data.

v0.16.0

5 months ago

Datashader 0.16.0 is a significant release adding support for rendering GeoPandas GeoDataFrames directly rather than having to convert them to SpatialPandas first. Support for GeoPandas geometry types in Datashader Canvas functions is as follows:

  • Canvas.line: LineString, MultiLineString, MultiPolygon, Polygon
  • Canvas.point: MultiPoint, Point
  • Canvas.polygons: MultiPolygon, Polygon

There is also support in Canvas.line for a new data type which is a 2D xarray.DataArray (within an xarray.Dataset) containing the coordinates of multiple lines that share the same x coordinates.

The DataShape package is now vendored in Datashader as it has not been maintained for a number of years and is not accepting updates.

Thanks to new contributor @J08ny and regular contributors @Hoxbro and @ianthomas23.

Enhancements:

  • Support rendering of GeoPandas GeoDataFrames as lines, points and polygons (#1285, #1293, #1297)
  • Implement lines using 2D xarray with common x coordinates (#1282)

General code improvements:

  • Add debug logging to compiler module (#1280)
  • Vendor DataShape (#1284)
  • Don't use object as base class (#1286)
  • Fix typos using codespell (#1288)
  • Fix float16 being a floating type. (#1290)
  • Simplify line _internal_build_extend (#1294)

Improvements to CI:

  • Update to latest holoviz_tasks (#1281)
  • Update codecov configuration (#1292)
  • Add pre-commit (#1295, #1296)

Compatibility:

  • Support Pandas 2.1 (#1276, #1287)
  • Replace np.NaN with np.nan (#1289)
  • Drop support for Python 3.8 (#1291)

v0.15.2

7 months ago

This release adds antialiased line support for inspection reductions such as max_n and where, including within categorical by reductions. It also improves support for summary reductions and adds CUDA implementations of std and var reductions.

Thanks to regular contributors @Hoxbro, @ianthomas23, @maximlt and @thuydotm.

Enhancements:

  • Antialiasing line support for inspection reductions:

    • Pre-compile antialias stage 2 combination (#1258)
    • Antialiased min and max row index reductions (#1259)
    • CPU shift_and_insert function (#1260)
    • Refactor of CUDA *_n reductions (#1261)
    • Support antialiased lines in *_n reductions (#1262)
    • Replace accumulate with copy on first call to antialiased stage 2 combine (#1264)
    • Separate where combine_cpu functions by ndim (#1265)
    • Antialiased line support for where reductions (#1269)
  • Improved support for summary reductions:

    • Support by reduction within summary reduction (#1254)
    • Support summary containing by reduction with other reductions (#1257)
    • Support summary containing multiple where with the same selector (#1271)
  • CUDA support for std and var reductions (#1267)

General code improvements:

  • Remove pyarrow pin (#1248)

Improvements to CI:

  • Update holoviz_tasks to v0.1a15 (#1251)
  • Use holoviz_tasks/install action for docs (#1272)

Improvements to documentation:

  • Update readme to include Python 3.11 (#1249)
  • Correct links to pandas docs (#1250)
  • Remove twitter from index page (#1253)
  • Create FUNDING.yml (#1263)

v0.15.1

8 months ago

This release contains an important bug fix to ensure that categorical column order in maintained across dask partitions. It also adds support for categorical inspection reductions such as by(max_n). The only missing functionality for inspection reductions is now antialiased lines, which in planned for the next release.

Thanks to contributors @ianthomas23, @maximlt and @philippjfr.

Bug fixes:

  • Fix single category reductions (#1231)
  • Ensure categorical column order is the same across dask partitions (#1239)

Enhancements:

  • Categorical inspection reductions:

    • Support by(max_n) and by(min_n) (`#1229)
    • Categorical max_row_index, max_n_row_index and min equivalents (#1233)
    • Use enum for row index column rather than None (#1234)
    • Add support for categorical where reductions (#1237)
    • Add tests for handling of NaNs in where reductions (#1241)
  • General code improvements:

    • Only check dask.DataFrame dtypes of columns actually used (#1236)
    • Remove all use of OrderedDict (#1242)
    • Separate out 3d and 4d combine functions (#1243)
    • Reorganise antialiasing code (#1245)

Improvements to CI:

  • Bump holoviz tasks (#1240)
  • Add image is close test helper (#1244)

Improvements to documentation:

  • Update to Google Analytics 4 (#1228)
  • Rename pyviz-dev as holoviz-dev (#1232)

v0.15.0

9 months ago

This release provides significant improvements for inspection reductions by adding new first_n, last_n, max_n and min_n reductions, and providing Dask and CUDA support for all existing and new inspection reductions including where. It also provides support for Numba 0.57, NumPy 1.24 and Python 3.11, and drops support for Python 3.7.

Thanks to first-time contributors @danigm and @Jap8nted, and also regulars @Hoxbro, @philippjfr and @ianthomas23.

Enhancements:

  • Inspection reductions:

    • Reduction append functions return index not boolean (#1180)
    • first_n, last_n, max_n and min_n reductions (#1184)
    • Add cuda argument to _build_combine (#1194)
    • Support max_n and min_n reductions on GPU (#1196)
    • Use fast cuda mutex available in numba 0.57 (#1212)
    • Dask support for first, last, first_n and last_n reductions (#1214)
    • Wrap use of cuda mutex in where reductions (#1217)
    • Cuda and cuda-with-dask support for inspection reductions (#1219)
  • x and y range attributes on returned aggregations (#1198)

  • Make datashader.composite imports lazy for faster import time (#1222)

  • Improvements to CI:

    • Cancel concurrent test workflows (#1208)
  • Improvements to documentation:

    • Inspection reduction documentation (#1186, #1190)
    • Upgrade to latest nbsite and pydata-sphinx-theme (#1221)
    • Use geodatasets in example data

Bug fixes:

  • Fix conversion from cupy in categorical rescale_discrete_levels (#1179)
  • Validate canvas width, height (#1183)
  • Support antialiasing in pipeline API (#1213)

Compatibility:

  • NumPy 1.24 compatibility (#1176, #1185, #1218)

  • Numba 0.57 compatibility (#1205)

v0.14.4

1 year ago

This release adds a new where reduction that provides improved inspection capabilities and adds support for colormaps that are tuples of hex values. There are also various bug fixes and compatibility improvements.

Thanks to @ianthomas23, @maximlt and @Hoxbro.

Enhancements:

  • New where reduction to provide improved inspection functionality:

    • Add new where reduction (#1155)
    • Where reduction using dataframe row index (#1164)
    • CUDA support for where reduction (#1167)
    • User guide page for where reduction (#1172)
  • Support colormaps that are tuples of hex values (#1173)

  • Add governance docs (#1165)

  • Improve documentation build system (#1170, #1171)

  • Improvements to CI:

    • Rename default branch from master to main (#1156)
    • Use holoviz_task install action (#1163)

Bug fixes:

  • Validate calculated log canvas range (#1154)

  • Better validate canvas.line() coordinate lengths (#1160)

  • Return early in eq_hist() if all data masked out (#1168)

Compatibility:

  • Follow recommended numba best practice.

    • Ensure cuda functions are correctly jitted (#1153)
    • nopython=True everywhere (#1162)
  • Update dependencies:

    • Pip pyarrow in tests dependencies (#1174)

v0.14.3

1 year ago

This release fixes a bug related to spatial indexing of spatialpandas.GeoDataFrames, and introduces enhancements to antialiased lines, benchmarking and GPU support.

Thanks to first-time contributors @eriknw and @raybellwaves, and also @ianthomas23 and @maximlt.

Enhancements:

  • Improvements to antialiased lines:

    • Fit antialiased line code within usual numba/dask framework (#1142)
    • Refactor stage 2 aggregation for antialiased lines (#1145)
    • Support compound reductions for antialiased lines on the CPU (#1146)
  • New benchmark framework:

    • Add benchmarking framework using asv (#1120)
    • Add cudf, dask and dask-cudf Canvas.line benchmarks (#1140)
  • Improvements to GPU support:

    • Cupy implementation of eq_hist (#1129)
  • Improvements to documentation:

  • Improvements to dependency management (#1111, #1116)

  • Improvements to CI (#1132, #1135, #1136, #1137, #1143)

Bug fixes:

  • Ensure spatial index _sindex is retained on dataframe copy (#1122)

v0.14.2

1 year ago

This is a bug fix release to fix an important divide by zero bug in antialiased lines, along with improvements to documentation and handling of dependencies.

Thanks to @ianthomas23 and @adamjhawley.

Enhancements:

  • Improvements to documentation:

    • Fix links in docs when viewed in browser (#1102)
    • Add release notes (#1108)
  • Improvements to handling of dependencies:

    • Correct dask and bokeh dependencies (#1104)
    • Add requests as an install dependency (#1105)
    • Better handle returned dask npartitions in tests (#1107)

Bug fixes:

  • Fix antialiased line divide by zero bug (#1099)

v0.14.1

1 year ago

This release provides a number of important bug fixes and small enhancements from Ian Thomas along with infrastructure improvements from Maxime Liquet and new reductions from @tselea.

Enhancements:

  • Improvements to antialiased lines:
    • Support antialiased lines for categorical aggregates (#1081,#1083)
    • Correctly handle NaNs in antialiased line coordinates (#1097)
  • Improvements to rescale_discrete_levels for how='eq_hist':
    • Correct implementation of rescale_discrete_levels (#1078)
    • Check before calling rescale_discrete_levels (#1085)
    • Remove empty histogram bins in eq_hist (#1094)
  • Implementation of first and last reduction (#1093) for data types other than raster.

Bug fixes:

  • Do not snap trimesh vertices to pixel grid (#1092)
  • Correctly orient (y, x) arrays for xarray (#1095)
  • Infrastructure/build fixes (#1080, #1089, #1096)

v0.14.0

1 year ago

This release has been nearly a year in the making, with major new contributions from Ian Thomas, Thuy Do Thi Minh, Simon Høxbro Hansen, Maxime Liquet, and James Bednar, and additional support from Andrii Oriekhov, Philipp Rudiger, and Ajay Thorve.

Enhancements:

  • Full support for antialiased lines of specified width (#1048, #1072). Previous antialiasing support was limited to single-pixel lines and certain floating-point reduction functions. Now supports arbitrary widths and arbitrary reduction functions, making antialiasing fully supported. Performance ranges from 1.3x to 14x slower than the simplest zero-width implementation; see benchmarks.
  • Fixed an issue with visibility on zoomed-in points plots and on overlapping line plots that was first reported in 2017, with a new option rescale_discrete_levels for how='eq_hist' (#1055)
  • Added a categorical color_key for 2D (unstacked) aggregates (#1020), for producing plots where each pixel has at most one category value
  • Improved docs:
    • A brand new polygons guide (#1071)
    • A new guide to 3D aggregations using by, now documenting using categorizer objects to do 3D numerical binning (#1071)
    • Moved documentation for spreading to its own section so it can be presented at the right pipeline stage (was mixed up with colormapping before) (#1071)
    • Added rescale_discrete_levels example (#1071)
    • Other misc doc cleanup (#1035, #1037, #1058, #1074, #1077)

Bugfixes:

  • Fixed details of the raster coordinate calculations to match other primitives, making it simpler to overlay separately rendered results (#959, #1046)
  • Various fixes and extensions for cupy/CUDA, e.g. to use cuda for category_binning, spread, and dynspread, including cupy.interp where appropriate (#1015, #1016, #1044, #1050, #1060)
  • Infrastructure/build/ecosystem fixes (#1022, #1025, #1027, #1036, #1045, #1049, #1050, #1057, #1061, #1062, #1063, #1064)

Compatibility:

  • Canvas.line() option antialias=True is now deprecated; use line_width=1 (or another nonzero value) instead. (#1048)
  • Removed long-deprecated bokeh_ext.py (#1059)
  • Dropped support for Python 2.7 (actually already dropped from the tests in Datashader 0.12) and 3.6 (no longer supported by many downstream libraries like rioxarray, but several of them are not properly declaring that restriction, making 3.6 much more difficult to support.) (#1033)
  • Now tested on Python 3.7, 3.8, 3.9, and 3.10. (#1033)

v0.13.0

2 years ago

Version 0.13.0

Thanks to Jim Bednar, Nezar Abdennur, Philipp Rudiger, and Jean-Luc Stevens.

Enhancements:

  • Defined new dynspread metric based on counting the fraction of non-empty pixels that have non-empty pixels within a given radius. The resulting dynspread behavior is much more intuitive than the old behavior, which counted already-spread pixels as if they were neighbors (#1001)
  • Added ds.count() as the default reduction for ds.by (#1004)

Bugfixes:

  • Fixed array-bounds reading error in dynspread (#1001)
  • Fix color_key argument for dsshow (#986)
  • Added Matplotlib output to the 3_Interactivity getting started page. (#1009)
  • Misc docs fixes (#1007)
  • Fix nan assignment to integer array in RaggedArray (#1008)

Compatibility:

  • Any usage of dynspread with datatypes other than points should be replaced with spread(), which will do what was probably intended by the original dynspread call, i.e. to make isolated lines and shapes visible. Strictly speaking, dynspread could still be useful for other glyph types if that glyph is contained entirely in a pixel, e.g. if a polygon or line segment is located within the pixel bounds, but that seems unlikely.
  • Dynspread may need to have the threshold or max_px arguments updated to achieve the same spreading as in previous releases, though the new behavior is normally going to be more useful than the old.