Spark Lucenerdd Versions Save

Spark RDD with Lucene's query and entity linkage capabilities

v0.2.5

7 years ago

Changelog:

*Update to Spark 2.1.0 *Update to Scalatest 3.x *Add toString method in SparkScoreDoc *Update minor versions on dependencies joda-time and algebird *Fix return type on scripts

v0.2.4

7 years ago

Changelog:

  • Update Spark version to 2.0.2
  • Add moreLikeThis functionality in LuceneRDD

v0.2.3

7 years ago

Changelog:

General

  • Introduce Versionable trait

Linkage:

  • Multithreaded search on each partition (LuceneRDD, ShapeLuceneRDD)
  • Remove unnecessary topK monoid reduce
  • Response is Array[List[SparkScoreDoc] not List[SparkScoreDoc]

LuceneRDD:

  • Add deduplication method dedup

ShapeLuceneRDD

  • Add implicitis for DataFrames

v0.2.2

7 years ago

Changelog:

  • Unpersist broadcast variables on all link methods
  • Add .version method on all main RDDs
  • Rename GridLoader to PrefixTreeLoader
  • Remove multivalued fields in SparkDoc
  • Configurable prefix tree in ShapeLuceneRDD (via spatial.prefixtree.name={geohash|quad}

v0.2.1

7 years ago

Changelog:

  • All used classes are registered under LuceneRDDKryoRegistrator.
  • Bump Spark version to 2.0.1

v0.0.24

7 years ago
  • Fix critical bug on TopKMonoids for LuceneRDD and FacetedLuceneRDD
  • Trait SparkScoreDocAggregatable is removed.
  • Set TopKMonoid size to topK input parameter (instead of MaxDefaultTopK) in LuceneRDD.link

v0.0.22

7 years ago
  • Fixed #48