This is a minor release. snorkel 0.5.1 is released, releasing sybil 0.5.1 as well.

Miscellany

• [idiomatic go] updating code style
• [query] interstitial stage of querying can choose which metric to sort by before pruning, was using "count" as default before
• [fake data] add fake data generation through https://github.com/plaitpy/plaitpy
• [cli] improve output for insufficent/invalid arguments (#33)

Features

• add loghist support
• add per block query cache support
• add loglogbeta usage for count distinct
• add combine step to middle of aggregation, allowing for freeing memory to OS earlier and better high cardinality column handling
• add initial distributed query support (list tables, get table info and run query are supported)
• add a node aggregator for combining results from distributed queries

Bugfixes

• lower total number of opened files when decoding large tables
• support quoted CSV better (using encoding/csv now)
• correct treatment of eq and neq int filters when examining block extents

Fixes & Features

• auto compact during ingestion
• add export to TSV option
• support custom field and filter delimiters
• don't double print samples in ingestion log during samples queries
• support for ingesting directly from file (for use with fluentd) instead of stdin

New Features & Behavior:

• add support for gzip compressed files in DB
• CLI is quiet by default
• experimentally embedding luajit mapreduce
• static linking of sybil with cgo
• add bounce rate to sessionization code

New Features:

• None

Bugfixes:

• Fixing broken tests & data races
• Fixing cache lock recovery
• Fixing group by issue with time series

New Features:

• Recycle records memory (and avoid usage of GC heap)
• Extreme outlier detection and removal (more than 10,000 STD DEVs away)
• Deal with high volume datasets by caching capped block info for datasets with large block counts (caches 64 blocks per file)

Bugfixes:

• Correctness: Filter samples in row store when using filters and samples query
• Threadsafety: Grab lock for accessing block list when loading row store records into row store block

Small fixes for compiling on multiple architectures

update for building with go 1.6

• fix deleting block from table blocklist during aggregation race condition
• read from the ingestion log when getting samples

small bugfix release

• small speed improvements in unpacking columns
• fix GROUP BY with missing column values
• don't align time filters automatically unless we are doing a time query

New Features:

• support for per table memory limits
• better printing for console based time series

Bugfixes:

• remove excess docs from repo
• don't create new dataset dirs willy nilly during query or digest, only during ingestion
• don't read ingestion log unless specified on command line