Pisa Versions Save

Note that there has been a lot of breaking changes since the last release. We are working on a stable release, but until then breaking changes will be common.

What's Changed

• Produce URLs and lexicons when partitioning by @elshize in https://github.com/pisa-engine/pisa/pull/356
• Add ABOUT page by @JMMackenzie in https://github.com/pisa-engine/pisa/pull/359
• Updated TBB by @amallia in https://github.com/pisa-engine/pisa/pull/360
• Shard tools by @elshize in https://github.com/pisa-engine/pisa/pull/358
• Refactoring by @amallia in https://github.com/pisa-engine/pisa/pull/361
• Verbose flag for BP by @elshize in https://github.com/pisa-engine/pisa/pull/362
• Stem queries by @amallia in https://github.com/pisa-engine/pisa/pull/365
• Unify document reordering in one tool by @elshize in https://github.com/pisa-engine/pisa/pull/363
• Update .travis.yml by @amallia in https://github.com/pisa-engine/pisa/pull/371
• Implement CLI scorer params by @JMMackenzie in https://github.com/pisa-engine/pisa/pull/366
• Change app to allow quantization with optional scorer by @JMMackenzie in https://github.com/pisa-engine/pisa/pull/374
• Compile Headers workflow by @elshize in https://github.com/pisa-engine/pisa/pull/375
• Fix for reorder tool not printing new identifier order by @JMMackenzie in https://github.com/pisa-engine/pisa/pull/386
• Clean up wand requirements by @JMMackenzie in https://github.com/pisa-engine/pisa/pull/388
• Fix CLI issue where quantization is not considered by @JMMackenzie in https://github.com/pisa-engine/pisa/pull/389
• Fix deprecated TBB calls by @JMMackenzie in https://github.com/pisa-engine/pisa/pull/387
• Krovetz thread by @JMMackenzie in https://github.com/pisa-engine/pisa/pull/390
• Streaming index compression for block index by @elshize in https://github.com/pisa-engine/pisa/pull/384
• Minor bug in app by @JMMackenzie in https://github.com/pisa-engine/pisa/pull/394
• Refactor cursor interface by @elshize in https://github.com/pisa-engine/pisa/pull/376
• Stream builder for quantized case by @elshize in https://github.com/pisa-engine/pisa/pull/391
• Move most test to Actions by @elshize in https://github.com/pisa-engine/pisa/pull/377
• Maxscore perf by @elshize in https://github.com/pisa-engine/pisa/pull/395
• Reduce number of test queries by @JMMackenzie in https://github.com/pisa-engine/pisa/pull/405
• MacOS GH actions by @amallia in https://github.com/pisa-engine/pisa/pull/408
• Added kth_threshold tool by @amallia in https://github.com/pisa-engine/pisa/pull/406
• clang-6 & code coverage by @elshize in https://github.com/pisa-engine/pisa/pull/409
• Remove optimal_hybrid_index by @elshize in https://github.com/pisa-engine/pisa/pull/412
• Use TBB thread local storage by @elshize in https://github.com/pisa-engine/pisa/pull/378
• Create docker.yml by @amallia in https://github.com/pisa-engine/pisa/pull/413
• Regression experiment documentation by @amallia in https://github.com/pisa-engine/pisa/pull/354
• More robust memory mapping by @elshize in https://github.com/pisa-engine/pisa/pull/370
• Update progress.hpp by @amallia in https://github.com/pisa-engine/pisa/pull/417
• Update badge by @amallia in https://github.com/pisa-engine/pisa/pull/414
• Fix variable name and change tests to use MemorySource by @elshize in https://github.com/pisa-engine/pisa/pull/420
• Fix BP memory issue by @elshize in https://github.com/pisa-engine/pisa/pull/424
• Fix bug of top-k threshold task by @amallia in https://github.com/pisa-engine/pisa/pull/426
• Update kth_threshold.cpp by @amallia in https://github.com/pisa-engine/pisa/pull/427
• Catch and log NoSuchFile error by @elshize in https://github.com/pisa-engine/pisa/pull/428
• Fix lexicon issue in term processor and refactor binary search by @elshize in https://github.com/pisa-engine/pisa/pull/434
• Make NoSuchFile inherit from std::exception by @elshize in https://github.com/pisa-engine/pisa/pull/436
• AOL query extractor by @amallia in https://github.com/pisa-engine/pisa/pull/296
• Taily stats by @elshize in https://github.com/pisa-engine/pisa/pull/430
• Update documentation by @elshize in https://github.com/pisa-engine/pisa/pull/438
• Fix reading bytes from bit vector by @elshize in https://github.com/pisa-engine/pisa/pull/443
• Read collection with lexicon by @elshize in https://github.com/pisa-engine/pisa/pull/437
• Use env files in CI by @elshize in https://github.com/pisa-engine/pisa/pull/451
• Modify count-postings tool by @elshize in https://github.com/pisa-engine/pisa/pull/450
• Return times when extracting taily scores by @elshize in https://github.com/pisa-engine/pisa/pull/452
• Enable counting postings by term by @elshize in https://github.com/pisa-engine/pisa/pull/453
• Extract maxscores by @elshize in https://github.com/pisa-engine/pisa/pull/454
• Explicitly use Ubuntu 18.04 in CI by @elshize in https://github.com/pisa-engine/pisa/pull/457
• Fix CI problems by @elshize in https://github.com/pisa-engine/pisa/pull/461
• Add CMake options for fPIC by @JMMackenzie in https://github.com/pisa-engine/pisa/pull/460
• Make parsing libraries private dependencies by @elshize in https://github.com/pisa-engine/pisa/pull/463
• Add inline to free function in header by @elshize in https://github.com/pisa-engine/pisa/pull/466
• Add patch for accepting non-trec identifiers from WARC files by @JMMackenzie in https://github.com/pisa-engine/pisa/pull/468
• Remove PSTL by @elshize in https://github.com/pisa-engine/pisa/pull/472
• Paralelize algorithms by @elshize in https://github.com/pisa-engine/pisa/pull/473
• Obey query term weights by @JMMackenzie in https://github.com/pisa-engine/pisa/pull/467
• Fix MacOS build by @JMMackenzie in https://github.com/pisa-engine/pisa/pull/477
• Move forward index builder implementation to cpp by @elshize in https://github.com/pisa-engine/pisa/pull/478
• Use DocId and Score aliases by @elshize in https://github.com/pisa-engine/pisa/pull/476
• Fix threshold logic by @JMMackenzie in https://github.com/pisa-engine/pisa/pull/429
• Extract invert functions to cpp by @elshize in https://github.com/pisa-engine/pisa/pull/479
• Move implementation to cpp files by @elshize in https://github.com/pisa-engine/pisa/pull/480
• Separates bit_vector_builder from bit_vector by @elshize in https://github.com/pisa-engine/pisa/pull/481
• Throw exception when write fails for block_freq_index by @elshize in https://github.com/pisa-engine/pisa/pull/484
• Handle I/O errors with exception by @elshize in https://github.com/pisa-engine/pisa/pull/485
• Remove std iterator by @elshize in https://github.com/pisa-engine/pisa/pull/487
• Multiply block max score by term weight in cursor (#474) by @elshize in https://github.com/pisa-engine/pisa/pull/488
• Move TemporaryDirectory to pisa namespace (#419) by @elshize in https://github.com/pisa-engine/pisa/pull/489
• Update GCC version for MacOS build by @elshize in https://github.com/pisa-engine/pisa/pull/491
• Whitespace tokenizer by @elshize in https://github.com/pisa-engine/pisa/pull/496
• CLI option to configure log level by @elshize in https://github.com/pisa-engine/pisa/pull/497
• Improve block codecs by @elshize in https://github.com/pisa-engine/pisa/pull/492
• Selecting tokenizer at query time by @elshize in https://github.com/pisa-engine/pisa/pull/499
• Implement TextAnalyzer by @elshize in https://github.com/pisa-engine/pisa/pull/503
• Custom sift down for binary heap by @elshize in https://github.com/pisa-engine/pisa/pull/507
• Fix log of zero by @elshize in https://github.com/pisa-engine/pisa/pull/510
• Address Codacy warnings by @elshize in https://github.com/pisa-engine/pisa/pull/512
• Rank start from 1 instead of 0 by @amallia in https://github.com/pisa-engine/pisa/pull/516
• C++20 Support by @elshize in https://github.com/pisa-engine/pisa/pull/513
• Bug fixes by @elshize in https://github.com/pisa-engine/pisa/pull/517
• Lib updates by @elshize in https://github.com/pisa-engine/pisa/pull/518
• Accumulator concept by @elshize in https://github.com/pisa-engine/pisa/pull/519
• Upgrade Google Benchmark to 1.7.1 by @elshize in https://github.com/pisa-engine/pisa/pull/520
• Split CI into push, trunk, and PR flows by @elshize in https://github.com/pisa-engine/pisa/pull/523
• Use thread local, as opposed to static, lexer by @elshize in https://github.com/pisa-engine/pisa/pull/528
• Pass last_level by value by @elshize in https://github.com/pisa-engine/pisa/pull/529
• Upgrade to the latest Porter2 version by @elshize in https://github.com/pisa-engine/pisa/pull/530
• Remove UB and read-after-buffer in varintgb by @elshize in https://github.com/pisa-engine/pisa/pull/531
• Fix TREC run format by @amallia in https://github.com/pisa-engine/pisa/pull/522
• Add missing bound check in score_opt_partition by @elshize in https://github.com/pisa-engine/pisa/pull/532
• Update test Docker images for Clang by @elshize in https://github.com/pisa-engine/pisa/pull/534
• Use rapidcheck to test codecs by @elshize in https://github.com/pisa-engine/pisa/pull/535
• oneTBB & other updates by @elshize in https://github.com/pisa-engine/pisa/pull/538
• Allow for system Boost installation by @elshize in https://github.com/pisa-engine/pisa/pull/540
• Replace boost::variant with std::variant by @elshize in https://github.com/pisa-engine/pisa/pull/544
• thread_local Porter2 by @elshize in https://github.com/pisa-engine/pisa/pull/542
• Add byte padding to block index after postings by @elshize in https://github.com/pisa-engine/pisa/pull/545
• Migrate to mdBook documentation by @elshize in https://github.com/pisa-engine/pisa/pull/549
• Fix unaligned read in bitvector by @elshize in https://github.com/pisa-engine/pisa/pull/550
• Add Release build to CI on trunk by @elshize in https://github.com/pisa-engine/pisa/pull/551
• Upgrade CLI11 to 2.3.2 by @elshize in https://github.com/pisa-engine/pisa/pull/546
• Benchmark docker by @elshize in https://github.com/pisa-engine/pisa/pull/548
• Update docs about algorithms by @elshize in https://github.com/pisa-engine/pisa/pull/552
• Add new line for proper formatting by @pombredanne in https://github.com/pisa-engine/pisa/pull/554
• Remove assert from next_geq by @elshize in https://github.com/pisa-engine/pisa/pull/555
• Remove configuration.hpp by @elshize in https://github.com/pisa-engine/pisa/pull/556
• Update clang-format rules by @elshize in https://github.com/pisa-engine/pisa/pull/559
• Add stdc++fs to target_link_libraries by @elshize in https://github.com/pisa-engine/pisa/pull/560
• Add metric evaluation to benchmark Docker by @elshize in https://github.com/pisa-engine/pisa/pull/564
• Warn about missing terms and removed stopwords by @elshize in https://github.com/pisa-engine/pisa/pull/563
• Query refactoring by @elshize in https://github.com/pisa-engine/pisa/pull/561
• Source setvars.sh when installing TBB from package by @elshize in https://github.com/pisa-engine/pisa/pull/567
• Append sourcing setvars.sh to .bashrc by @elshize in https://github.com/pisa-engine/pisa/pull/568

New Contributors

• @pombredanne made their first contribution in https://github.com/pisa-engine/pisa/pull/554

Full Changelog: https://github.com/pisa-engine/pisa/compare/v0.8.2...v0.9.0

• cppcoreguidelines-pro-type-member-init (#349)
• Extract compression to headers (#350)
• Plaintext tokenizer (#351)

• Replace unnecessary raw loops (#338)
• Fix readability-braces-around-statements warnings (#340)
• Fix hicpp-use-auto warnings
• Revert "Fix hicpp-use-auto warnings"
• Fix hicpp-use-auto warnings (#341)
• [clang-tidy] hicpp-move-const-arg (#342)
• modernize-use-equals-default + modernize-use-using (#344)
• Readability implicit bool conversion (#345)
• Readability uppercase literal suffix (#346)
• hicpp-explicit-conversions (#347)
• Update CMakeLists.txt (#348)

• Binary lib (#312)
• Change default run name from "R0" to "PISA" (#314)
• 99% quantile in queries tool (#315)
• QLD fix (#316)
• Fix maskedvbyte and varintgb buffer initialization when encoding (#318)
• Thresholds (#320)
• Add support for rerunning unsafe queries (#319)
• Common CLI arguments (#313)
• Quantized index (#321)
• Clang format (#324)
• Support for clang-tidy (#326)
• Fix CLI in create_freq_index (#329)
• Finish clang-format and add Travis check (#327)
• Some minor fixes to quantization (#331)
• Small tweak to app (#333)
• Update bit_vector.hpp (#335)
• Removed almost all config (#330)
• drop default logger in create_freq_index (#334)
• Fix 336 (#337)

• Fix: issue with using an old version of trecpp

• Added node_modules to gitignore
• Log stats on the inverted index (#208)
• Update TBB and PSTL (#211)
• Integrate new trecpp version (#210)
• Introduce error limit for gumbo (#213)
• Updated CLI11 (#212)
• Update trecpp (#214)
• Fix sorting term bug (#209)
• Fix memory leak in cleantext (#215)
• Add run_id (#216)

• Update README.md
• Update README.md
• Update recursive_graph_bisection.cpp (#202)
• Update recursive_graph_bisection.cpp (#204)

• Fix python script to read the collection (#197)