Pilosa Versions Save

This version contains 1 contribution from 1 contributor. There are 4 files changed; 170 insertions; and 30 deletions.

Fixed

• Add logic that clears fragment.rowCache after importValue (#2084)

This version contains 99 contributions from 11 contributors. There are 94 files changed; 9,453 insertions; and 6,121 deletions.

Attention: Pilosa 1.4.0 changes the way that integer fields are stored. The upgrade from old format to new is handled automatically, however you will not be able to downgrade to 1.3 should you wish to do so. We always recommend taking a backup of your Pilosa data directory before upgrading Pilosa, but doubly so with this release.

Added

• Update "Getting Started" documentation (#2028)
• Add ability to disable tracing and use nopTracer (#2029)
• Add test for no containers (#2016)
• Add naive implementations of Roaring and fuzz test (#2023)
• Add fuzzing code and readme.md to explain the fuzzer (#2004)
• Add MinRow and MaxRow calls (#1983)
• Add Prometheus stats backend (#1992)
• Add extra tracing spans and metadata (#1939)
• Add more Debugf() statements to the holder open process (#1950)
• Add ability to post schema using holder.applySchema (#1956)

Changed

• Update CircleCI build with Go 1.13 and run enterprise tests (#2064)
• Update Alpine to 3.9.4 in Dockerfile (#2001)
• Add Prometheus tests, refactor http stats as middleware, minor fixes (#1994)
• Add confirmation logic to catch false nodeLeave events (#1993)
• Improve TopN() errors (#1978)
• Make integer fields unbounded by using sign+magnitude representation (#1902)
• Simplify contributing instructions by removing weird upstream thing (#1966)

Fixed

• Default BSI base value to min, max, or 0 depending on the min/max range (#2050)
• Add worker pool for query processing (#2034)
• Move Range deprecation message to higher level (#2033)
• Use lock in view.deleteFragment while altering fragments (#2026)
• Fix malformed offset bug in readOffsets and readWithRuns (#2021)
• Fix various container iteration bugs in Roaring (#2019)
• Fix malformed bitmap handling (#2017)
• Fix fuzzer errors in roaring (#2012)
• Save all state files atomically to avoid corruption (#2000)
• Fix slice container updates (#1997)
• Fix out of bounds panic to show error (#1975)
• Fix error message returned by regex on field and index names (#1973)
• Fix filter calls in GroupBy not being translated (#1970)
• Fix TranslateFile behavior when reopened (#1954)
• Remove buggy shard validation code (#1951)
• Fix some lint warnings raised in VS-Code (#1947)

Performance

• Address some startup speed and performance issues (#1988)
• Add a worker pool for importRoaring jobs (#2048)
• Use UnionInPlace for computing time rows which involve multiple views (#2041)
• Improve ingest performance with snapshot queue and unmarshaling improvements (#2024)
• Improve row cache (#1974)

Removed

• Remove extraneous stat tags to improve prometheus performance (#1996)

This version contains 1 contribution from 1 contributor. There are 6 files changed; 10 insertions; and 95 deletions.

Fixed

• Remove shard validation to fix bug where some nodes weren't loading their fragments. #1951 (#1964)

This version contains 98 contributions from 10 contributors. There are 144 files changed; 12,635 insertions; and 4,341 deletions.

Added

• Add license headers and CI check (#1940)
• Add support to modify shard width at build time (#1921)
• Add 'bench' Makefile target and run fewer concurrency level benchmarks (#1915)
• Add server stats to /info endpoint (#1859)
• Implement config options for block profile rate and mutex fraction (#1910)
• Implement global open file counter using syswrap (to scale past system open file limits) (#1906)
• Implement global mmap counter with fallback (to scale past system mmap limits) (#1903)
• Add shard width to index info in schema (allows client to get shard width at run time) (#1881)
• Add shift operator (#1761)
• Support advertise address and listen on 0.0.0.0 (#1832)
• Added convenience function to efficiently calculate size of a roaring bitmap in bytes (#1839)
• Make sure more tests and benchmarks can have their temp dir set by flag (#1831)
• Add sliceascending/slicedescending striped benchmarks (#1763)
• Add setValue test and benchmarks (#1820)
• Add a test for groupby filter with RangeLTLT (#1818)
• Add tests for GroupBy with keys; removes unused Bit message from proto (#1811)

Fixed

• Update to latest memberlist fork with race fixes (#1944)
• Return original error instead of cause in handler (#1943)
• Validate (and panic) on duplicate PQL arguments (#1938)
• Add correct content type to query responses Fixes #1873 (#1936)
• Address race condition by getting cluster nodes with lock (#1931)
• Make sure to unmap containers before modifying (#1876)
• Avoid probable race when creating fragments (#1863)
• Improve help strings for metrics options (#1887)
• Ensure ClearRow() arguments get translated (#1848)
• Prevent omitting zero ids on columnattrs (#1846)
• Set cache size to 0 if cache type is none (#1842)
• Prevent deadlock in replication logic on reopening a store (#1834)
• Pass loggers around properly in gossip (#1835)
• Include read lock in cluster.Nodes() (#1836)
• Raise an error on Rows() query against a time field with noStandardView: true (#1826)
• Don't delete test fragment data (part of repo) (#1827)
• Fix bug on upper end of bsi range queries (#1822)
• Group by fixes (#1802)

Changed

• Switch to GolangCI lint (#1924)
• Return empty result set when query empty (#1937)
• Add Go 1.12 to CircleCI (#1909)
• Ignore fragment files from shards node doesn't own (#1900)
• Go module support. Use Modules instead of dep for dependencies (#1616)
• Merge Range() into Row() call. (#1804)
• Add from/to range arguments to Rows() call (#1851)
• Fixes Store call error messages, Rows doesn't need field argument (#1830)

Performance

• BTree performance improvements (#1916)
• Make Containers smaller, especially when they have small contents (#1901)
• Address UnionInPlace performance regressions (#1897)
• Small write path for import-roaring. Makes small imports faster (#1892)
• Small write path for imports (#1871)
• Remove copy for pilosa roaring files (#1865)
• Disable anti-entropy if not using replication [performance] (#1814)
• Group By—skip 0 counts as early as possible (#1803)

This version contains 155 contributions from 11 contributors. There are 113 files changed; 19,085 insertions; and 4,323 deletions.

Added

• Cancel queries on Context.Done() (#1773)
• Union In Place (#1766, #1774)
• Import benchmarking (#1771)
• Add GroupBy() Filter (#1753)
• Add /internal/translate/keys endpoint (#1751)
• CircleCI: Add race detector to parallel build, default to Go 1.11. (#1756)
• Add distributed tracing. (#1684)
• Add NoStandardView field option (#1733)
• Add some stat tracking to roaring implementation (#1743)
• Add cluster fault testing using docker-compose and pumba (#1717)
• Allow backslash, carriage return in PQL strings (#1713)
• Add base system, curl and jq for debug and checks (#1707)
• Add Rows and GroupBy functionality (#1647)
• Add clear functional option for imports (#1699)
• Implement tracking of available shards to help support sparse datasets (#1600, #1695, #1624, #1663)
• Add missing rowID/Key columnID/Key tests (#1683)
• Add Store() operation to PQL (#1666)
• Add diagnostics CPUArch field (#1671)
• Add CircleCI step to generate Docker image and push to Docker hub (#1673)
• Implement ClearRow() query (#1645)
• Add support for Bool fields (#1658)
• Make translate map size configurable (#1653)
• Add DirectAdd function to roaring.Bitmap (#1646)
• Implement Roaring import (#1622, #1738)
• Add Not() query (#1635)
• Implement Options call and excludeRowAttrs, excludeColumns, columnAttrs and shards args (#1631)
• Add field options to pilosa import (#1625)
• Implement column existence tracking (#1788, #1672, #1628)

Changed

• Convert the anti-entropy logic to use ImportRoaring instead of QueryNode (#1780)
• Simplify require-* logic in Makefile (#1755)
• Cleanup logging (#1748)
• Remove TravisCI, add CircleCI shield (#1740)
• Upgrade Peg dependency and regenerate grammar (#1725)
• Upgrade to protoc 3.6.1 (also updated protoc-gen-gofast) (#1724)
• Move column attrs logic to executor (#1677)
• Shrink container bit count to int32 (#1664)

Performance

• Remove bounds check (#1619)
• Improve benchmarking and performance (#1741)

Fixed

• Ensure internal client closes all response bodies (#1795)
• Allow translate log entry buffer to grow (#1787)
• Add Gopkg.lock as a dependency for vendor target (#1790)
• Cluster resize fix (#1785)
• Attempt to fix deadlock by releasing view lock before broadcasting (#1782)
• Fix bug where cluster goes into RESIZING instead of NORMAL (#1777)
• Propogate updates to node details (not just additions and deletions) (#1769)
• Fix arm64 support (#1764)
• Fix data races (#1750)
• Fix fragment checksums race condition (#1749)
• Import cmd field type flag (#1732)
• Increase the translate file size for tests/benchmarks (#1744)
• Prevent panic in Bitmap.UnmarshalBinary when there is no data (#1742)
• Remove unused rule from peg grammar (#1737)
• Improve Internal Client errors (#1729)
• Forward imports to non-coordinator shards (#1719)
• Fix double escapes in PQL grammar (#1727)
• Ensure btree comparison doesn't fail for smallish N (#1712)
• Drop now-superfluous methodNotAllowedHandler (#1711)
• Use pilosa.Logger everywhere (#1674)
• Ensure view closes fragment on broadcast error (#1675)
• Prevent closing os.Stderr (used in verbose test logging) (#1696)
• Allow holder to close/open/close without panic on closing closed channel (#1686)
• Fix bug with Range() queries with field keys (#1679)
• Sync query validation for handlers (#1676)
• Wrap translation store errors, decrease test map size to prevent failure on 32-bit (#1665)
• Fix pass-by-value issue in proto decode (#1662)
• Do not run prerelease in CI if this is a pull request (#1655)
• Ensure mutex imports unset previous columns (#1656)
• Treat import timestamps as UTC (#1651)
• Remove unused log buffers from test cluster, fixes race (#1612)
• Add --field-keys and --index-keys options to pilosa import (#1621)
• Use passed stdin, stdout, and stderr in the cmd package (#1620)
• Update Go client sample to match latest master (#1614)

This version contains 32 contributions from 5 contributors. There are 89 files changed; 2,752 insertions; and 1,013 deletions.

Added

• Add CircleCI (#1610)
• Add key translation to exports (#1608)
• Support importing key values (#1599, #1601)
• Treat coordinator as primary translate store (#1582)
• Add DEGRADED cluster state and handle gossip NodeLeave events correctly (#1584)
• Add linters to gometalinter and fix related issues (#1544, #1543, #1540, #1539, #1537, #1536, #1535, #1534, #1530, #1529, #1528, #1526, #1527)
• Add mutex field type (#1524)
• Fragment rows() and rowsForColumn() (#1532)

Fixed

• Fix race on replicationClosing channel (#1607)
• Prevent anti-entropy and cluster resize from running simultaneously (#1586)
• Require a valid port that isn't greater than 65,535 (#1603)
• Add view parameter to sync logic for syncing time fields (#1602)
• Fix translator in cluster environment (#1552)
• Use string prefix instead of equality so json error message will pass on all Go versions (#1558)

This version contains 11 contributions from 3 contributors. There are 30 files changed, 1,569 insertions, and 1,215 deletions.

Fixed

• Fix documentation (#1503, #1495, #1551)
• Fix places where empty IndexOptions were being used (#1547)
• Fix translator syncing bug in cluster environments (#1552)
• Fix race condition in translate_test (#1541)
• Add IndexOptions to IndexInfo json response (#1547)
• Add proper locking to cluster code to prevent races (#1533)
• Re-export erroneously unexported func Row.Intersect (#1502)
• Update parser to handle row keys on SetRowAttrs() (#1555)

This version contains 12 contributions from 4 contributors. There are 11 files changed, 133 insertions, and 39 deletions.

Fixed

• Use dep ensure -vendor-only for build repeatability (#1491)
• Make sure time range views are calculated correctly across months (#1485)
• Fix up error handling, add a configurable timeout to http handler closing (#1486)
• Add gossip Closer (#1483)
• Update docs references to WebUI naming (console) and installation (#1493)

This version contains 218 contributions from 7 contributors. There are 184 files changed, 21,769 insertions, and 20,275 deletions.

Added

• ID-Key Translation (#1337)
• Add CORS support to handler (#1327)

Changed

• HTTP handler updates (#1408, #1399, #1441, #1375, #1433, #1444, #1388, #1309, #1302, #1304, #1465, #1466)
• Refactor/improve tests (#1437, #1434, #1435, #1425, #1418, #1419, #1413, #1394, #1387, #1386, #1378, #1364, #1348, #1340, #1297)
• Simplify inter-node communication (#1428, #1427, #1412, #1398, #1391, #1389)
• Make gossip's interface to Pilosa the API struct (#1452)
• Rename slice to shard (#1426)
• Clearbit for time fields (#1424)
• Update docs (#1390, #1329, #1305, #1296, #1461)
• Simplify server setup (#1417, #1393,#1451)
• Refactor API (#1407)
• Rewrite PQL parser and add various improvements/simplifications (#1382, #1402, #1354, #1463)
• Rename "frame" to "field" (#1395, #1362, #1360, #1358, #1357, #1355)
• Optimize count (#1365)
• Simplify bitmap max function (#1333)
• Rename "bit" to "column" for clarity (#1326)
• Rename pilosa.Bitmap to Row (#1311)
• Invert encoding/decoding and remove internal references (#1454)

Removed

• Rename (unexport) many items to reduce public API footprint prior to 1.0 release (#1470, #1458, #1450, #1449, #1448, #1447, #1446, #1438, #1443, #1440, #1439, #1409, #1392, #1374, #1372, #1369, #1367, #1366, #1351, #1420, #1416, #1397)
• Remove dead code (#1432, #1457, #1421, #1411, #1377, #1393, #1462)
• Remove view argument from Field.SetBit and Field.ClearBit (#1396)
• Remove WebUI (now contained in a separate package) (#1363)
• Remove bench command (#1347)
• Remove "view" from API, handler, docs (#1346)
• Remove backup/restore stuff (#1339, #1341)
• Remove inverse frame functionality (#1335)
• Remove rangeEnabled option (#1332)
• Remove index and field MarshalJSON (#1468)

Fixed

• Fix a few data races (#1423)
• Fix for crash while removing containers (#1401)
• Allow dashes in frame names (#1415)
• Fix generate-config command, use single toml lib (#1350)

This version contains 10 contribution from 4 contributors. There are 5 files changed, 222 insertions, and 54 deletions.

Added

• Add Docker cluster/swarm tutorials (#1296, #1305)

Fixed

• Use errors.Cause in handler so that we return correct status codes (#1304, #1309)
• Close HTTP handler gracefully (#1310)
• Use lazy assignment for VERSION_ID so enterprise flag is set appropriately (#1308)
• Adjust cluster state tests so they aren't so dependent upon a sleep (#1306)
• Makefile enterprise build fixes (#1301)
• Re-add unintentionally removed check-clean (#1295)
• Fix syntax error and add i386 enterprise build (#1293)