Spatial Framework For Hadoop Versions Save

The Spatial Framework for Hadoop allows developers and data scientists to use the Hadoop data processing system for spatial data analysis.

v2.2.0

3 years ago
  • #165 JSON SerDe Compatibility with Hive-3.1 (randallwhitman)
  • #171 ST_BinEnvelope does not work with Decimal in Hive 2.3 (akashkatipally)
1c9854cb6a9390982786d2fc8a463f8b2600ac1a  hive/target/spatial-sdk-hive-2.2.0.jar.asc
0a0a362831238443b19c8f20aa5615266597d436  hive/target/spatial-sdk-hive-2.2.0.jar
28f916e3e9c544778bcddf43e3019d466b7a0374  hive/target/spatial-sdk-hive-2.2.0-javadoc.jar.asc
73037bf935390d05e13a38f3a9ab4fde212a5301  hive/target/spatial-sdk-hive-2.2.0-javadoc.jar
be9b121ccb9019edd0262792c4809d7b55482a05  hive/target/spatial-sdk-hive-2.2.0.pom.asc
87102f46d3fa33384bf2b8a97590e2c7d8e81864  hive/target/spatial-sdk-hive-2.2.0.pom
df6b46f8a5a788a9251a7c8d3290a5d691f0d329  hive/target/spatial-sdk-hive-2.2.0-sources.jar.asc
8da03be7ce471465bead5796513740c4a3869d4c  hive/target/spatial-sdk-hive-2.2.0-sources.jar

71fcd649b03fcffeca1fa95187dc5134f43e7c53  json/target/spatial-sdk-json-2.2.0.jar.asc
841bfa3061f8b1c83d46201c83fd98c7ec513e85  json/target/spatial-sdk-json-2.2.0.jar
c31882545ab2e70293f9ea01cffcdd98168dfa92  json/target/spatial-sdk-json-2.2.0-javadoc.jar.asc
d1ba6f211550e7ddf9222bed26d9795052ea7a14  json/target/spatial-sdk-json-2.2.0-javadoc.jar
632bd47f958e25386d619bacdc812c65aed32253  json/target/spatial-sdk-json-2.2.0.pom.asc
05fa04b442bfc190d9439bea749284deac26d47e  json/target/spatial-sdk-json-2.2.0.pom
0fd4ede7264051e43277a7966fdc9d4f74d7d7d4  json/target/spatial-sdk-json-2.2.0-sources.jar.asc
6fd97c1c2df411352e8c3726634d66e57e7e4cef  json/target/spatial-sdk-json-2.2.0-sources.jar

v2.1.0

5 years ago
  • #153 ST_Centroid: Centroid not Envelope Center (randallwhitman)
  • #155 ST_GeomFromShape: description (randallwhitman)
  • #154 Upgrade to Geometry v2.2 and Jackson v2.9.6 (randallwhitman)
  • #144 Fix test that broke on daylight shift (saifalharthi)

0390eec048930e1b1718a0bb888d74ce3f59ec91 spatial-sdk-hive-2.1.0.jar.asc b301b5bb8662f931c9d23f8da72994d406afcf1d spatial-sdk-hive-2.1.0-javadoc.jar.asc 6a5446e5497cff2bd9d020d3f981ca2b88c504b6 spatial-sdk-hive-2.1.0.pom.asc 33b138668b87262b864ea6e833ddccc73495cd7a spatial-sdk-hive-2.1.0-sources.jar.asc 58312158cbe58ac16025b14e5fce935508ced95a spatial-sdk-json-2.1.0.jar.asc cb0ab7bf1a1fe4e1b35e7d07be40a34ed52748a9 spatial-sdk-json-2.1.0-javadoc.jar.asc 7723bff0f6f9639a7497e8515c65e3934f85386d spatial-sdk-json-2.1.0.pom.asc d1d33f9babda07cf47cb3224bfb8ccc57d99cd52 spatial-sdk-json-2.1.0-sources.jar.asc dccb307a0f7cd32509c7ac0923348546b50226ed spatial-sdk-hive-2.1.0.jar 70a66242d4800ef15965ea17aed9368ad1f5efc0 spatial-sdk-hive-2.1.0-javadoc.jar ea1ae1f9617e06f876e7c5e4b9b13b6cc1fecc20 spatial-sdk-hive-2.1.0-sources.jar 2ec3424c637cb14b2fdf94a506826ad4e2b74754 spatial-sdk-json-2.1.0.jar 1c3ada28ac7b710424b818daf4f99966f5a7ccd4 spatial-sdk-json-2.1.0-javadoc.jar db8d1009e9954e1d3b0e207e8c84c37cbbe23b1a spatial-sdk-json-2.1.0-sources.jar fc43ca6f821ff979a664983695679958c421bf4d spatial-sdk-hive-2.1.0.pom 8405adc5d567651740938fd628caaa3d88c58051 spatial-sdk-json-2.1.0.pom

v2.0.0

6 years ago
  • #127 JSON Changes - Geometry v2 (see notes below on upgrading)
  • #130 Compile JSON SerDe classes with Hive-2.3 (dropped support for Hive-0.10)
  • #101 Esri JSON SerDe with Hive Date type column
  • #128 Obsolete the deprecated old InputFormat and SerDe names (see notes below on upgrading)
  • #113 Disable doclint for Java-1.8+

Spatial Framework for Hadoop v2 requires Geometry v2, and if any JSON capabilities are used, the classpath must include v2 rather than v1 of Jackson. Also, v2 requires a minimum of Hive-0.11 version. (On adding support for Hive v2.3, support for v0.10 was dropped - for Hive-0.11, building from source requires Hive-0.12, after which all applicable unit tests still pass with v0.11 of Hive.)

Updating to Spatial Framework for Hadoop v2 from v1: for any Hive table that uses a JSON InputFormat or SerDe with the old names that were deprecated in v1.2 and obsoleted in v2.0, upgrading requires an alter table statement, to update the name of the InputFormat and the SerDe.

  • EnclosedJsonInputFormat -> EnclosedEsriJsonInputFormat
  • UnenclosedJsonInputFormat -> UnenclosedEsriJsonInputFormat
  • JsonSerde -> EsriJsonSerDe

Example ALTER TABLE statement:

ALTER TABLE test15json SET FILEFORMAT
 INPUTFORMAT 'com.esri.json.hadoop.UnenclosedEsriJsonInputFormat'
 OUTPUTFORMAT 'org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat'
 SERDE 'com.esri.hadoop.hive.serde.EsriJsonSerDe';

If upgrading from v1.0 or v1.1, consider upgrading to v1.2 on the way to v2, if there are tables requiring such alter table - as v1.2 supports both the old and new naming, it would allow a rolling upgrade, without interrupting availabilty of all spatially-enabled tables during the upgrade.

SHA1 checksums:

a70fcc80cd44492ba33d66e5d2d82878dc69e5fc  spatial-sdk-hive-2.0.0.jar.asc
3edb4bcce8e5bc22b565457b27702950e5a72bff  spatial-sdk-hive-2.0.0.jar
1bda088ddf26f00c5344c9a1ee89519f783e64f8  spatial-sdk-hive-2.0.0-javadoc.jar.asc
2d5eea537e774416f3d3dc02d65ae4d8d4a40041  spatial-sdk-hive-2.0.0-javadoc.jar
900cdaaae7c76af315a71ce376f52d985a086fa0  spatial-sdk-hive-2.0.0.pom.asc
88ec7b4bb533559dc4cfd693dff0a477c3ef32fa  spatial-sdk-hive-2.0.0.pom
7ac3ce00f54e937e0cb67585b01efa60bbe816e9  spatial-sdk-hive-2.0.0-sources.jar.asc
8f10c25d19ed5127b1b8ccd5c418a772b69839c8  spatial-sdk-hive-2.0.0-sources.jar

5950ab7b04274d6b6cb6d160ba81c83065696da0  spatial-sdk-json-2.0.0.jar.asc
24ecc5e84dff1e85162fd384d0478473dae6b0d7  spatial-sdk-json-2.0.0.jar
22b1b1294982446ac0084b6e508b5c3bd556069f  spatial-sdk-json-2.0.0-javadoc.jar.asc
a37649dce0c9716e18e51d252897ceea16b117f7  spatial-sdk-json-2.0.0-javadoc.jar
a6e8de1b6662b98faf42b934b1feed3d47b62f87  spatial-sdk-json-2.0.0.pom.asc
4fa91928427fbe9782da6c6fed8188f4a7ca8014  spatial-sdk-json-2.0.0.pom
575e65f2216b1ecd7e0a1adb7a1233be576eb02d  spatial-sdk-json-2.0.0-sources.jar.asc
5b96008f6c0d026f6183baa5a2dfe8cef568c3ec  spatial-sdk-json-2.0.0-sources.jar

v1.2.0

7 years ago
  • #81 GeoJson InputFormat
  • #80 GeoJson SerDe
  • #97 JSON SerDe under SparkSQL
  • #77 JSON constructors getDisplayString with Hive .12 & .13
  • #91 EnclosedJsonInputFormat for MapReduce v2
  • #103 ST_GeometryFromShape and ST_AsShape - Esri shape format
  • #109 ST_SetSRID: preserve MultiPolygon geometry type
  • #110 ST_MultiPolygon constructor and positive ST_Area
  • #88 Fix String comparison in EnclosedJsonRecordReader

Requires v1.2 or v1.2.1 of the Esri Geometry API for Java.

  • 5dbf3ac26d519d2df57b633024cf25cf377a4a1f spatial-sdk-hive-1.2.0.jar
  • 6c39f6efef22f2c69b80e87a76576b88cbcaa28f spatial-sdk-json-1.2.0.jar

v1.1

9 years ago

Release 1.1 includes binning functions and a substantial performance speedup. It requires v1.2 or v1.2.1 of the Esri Geometry API for Java.