Kdb Common Versions Save

This release adds the following new features:

• Code Loader (require.q)
  • Add new .require.libForce to allow libraries to be reloaded and reinitialised

NUC

• Code Loader (require.q)
  • .require.i.load now takes an additional argument (force boolean based on forced loading)
  • .require.i.init now takes an additional argument (reinit boolean based on re-initialisation of the library)
  • .require.i.markLibAsLoaded / .require.i.markLibAsInited also take an additional argument (force boolean)

Full Changelog

895fb2f require: Ensure 'libForce' behaves the same as 'lib' if library not loaded / inited
6f56708 require: Send 'force' state to library init functions
daef0aa require: Add support to force reload a library
486aa90 type: Ensure .type.isEnumeration returns true for atom [#107]
6147a71 util: Whitespace fixes
3215e68 util: Add new .util.roundSigFig
a6e6da5 type: Add GUID support to .type.ensureSymbol [#104]
44ecfed csv: Switch to '.q.csv' to allow delimeter to be overidden [#102]

This release adds the following new features:

• Code Loader (require.q)
  • Load and .*lib*.init failures will now print the full backtrace
• Random Data Generator (rand.q)
  • Ability to get random bytes and generate random numbers and GUIDs from /dev/random (.rand.dr.*)
• Terminal Management (term.q)
  • Provide the ability to specify a minimum console size via the -c command line argument
• Job Scheduler (cron.q)
  • Provide new functions for duplicate function and argument scheduling (wiki)

NUC

None in this release

Full Changelog

336bf8c cron: Add 'unique' versions of addRepeatForeverJob / addRepeatUntilJob [#100]
1a11ee3 file: Add optimised enumeration target for symbol - .file.kdb.getSymEnumerationTarget
70892b7 compress: Don't rewrite '.d' file in compression is in-place [#89]
114e9c1 terminal: Allow a minimum console size to be set with '-c' [#90]
ee7d61f require: Add backtrace support during file loading and library loading
16a2084 os: Fix .os.getTerminalSize for Mac
8cfdd89 rand: Add support for cached /dev/random handle (behind config)
57642dc rand: Add .rand.dr (dev-random) to support random data from /dev/random
a817d11 compress: Add support for Garbage Collection after a splay compression
e8bcc32 ns: Add new '.ns.getFunctionCaller' function

This release adds a number of major features and improvements:

• Process startup (boot.q)
  • A wrapper to allow easier startup of kdb-common processes
• New compression library (compress)
  • Query compression statistics for splays and partitions
  • Perform compression across HDBs
• Interprocess communication (ipc)
  • Localhost connections auto-converted to Unix Domain Sockets
  • New .ipc.connectWait to ignore timeout
• Random data generator
• Optimised length, type and attribute functions for kdb files (file.kdb)
  • Reads mimimal data from disk to retrive these values (wiki)

NUC

• type:
  • .type.isList will now return true for on-disk nested lists and anymaps
  • .type.isSplayedTable will now return false for paths that are files and true for splay tables missing / at the end of the path
• cron: .cron.add will now reject start times before 'now', configurable via .cron.cfg.historicalStartTimes (wiki)

Full Changelog

8a88986 ipc: Add support for auto-translating 'localhost' TCP/IP to UDS
bbb8b77 ipc: Add .ipc.connectWait to ignore timeout
2d6c07d file.kdb: Bug fix for sym type files falling back to 'count get'
93a3c05 type: Add support for < 3.5 on-disk nested list types
96ae6a7 env: Add .env.which
b03c101 compress: Only look for active kdb files - cols in .d / *# / *## files [#80]
70f14db compress: Fix allowing atom for single table in partition [#81]
693786e compress: Fix incorrectly attempting to copy empty files inplace [#78]
80508ac os: Fix .os.isInteractiveSession for Mac [#72]
2b81be0 file.kdb: Add support for optimised reading of attribute
77d0e88 log: Move formatter functions into configuration [#67]
34be389 compress: Fix get* errors when splay has compressed and uncompressed cols [#73]
40a069f cron: Switch 'now' calculation to second resolution [#68]
5bf14ff http: Add new HTTP status code description dictionary ('.http.status')
0798deb file.kdb: Re-write getLength to add support for "new-format" lists
a764b04 compress: Add parallel compression and fix copying nested lists
a5bf0ae Add simple boot script (boot.q) to easily initialise kdb-common in a kdb process
0f577fa compress: Improve get*Stats functions when dealing with uncompressed columns
a6099b6 compress: Compression statistics and compress for splay / partitions
d62bda8 file.hdb: Add new par.txt wrapper function
feadb0e require: Add support for slf4j-style parameterised messages in basic logger
232e2db file.kdb: Add new optimised list element size function
5a6d0f6 rand: Add random data generator
1315ec4 file: Add .file.replace
acff064 convert: Remove invalid exception line from .convert.tableToHtml

This release adds a number of major features and improvements:

• Logging (log)
  • slf4j-style parameterised logging
  • Pattern-based logging formats
  • Call line tracing for enhanced debugging (which is enabled by default if the process is started in "debug" mode)
  • Log level in text output is now padded
• Job scheduler (cron)
  • Additional mode of operation - tickless - so timer only ticks when a job needs to execute, rather on a regular interval
• HTTP client library (http)
  • Automatic deserialisation of kdb+ IPC binary format over HTTP (with content type header application/kdb-ipc)
• Namespace functions (ns)
  • .ns.getFunctionArguments now supports additional function types
  • .ns.protectedExecute now supports passing functions by value
• Conversion functions (convert)
  • .convert.tableToHtml now supports mixed list columns (due to .type.ensureString improvements)

NUC

• cron
  • .cron.cfg.runners has been renamed to .cron.runners
  • .cron.enable[] replaced with .cron.changeMode[mode]
  • Job start and end times are now rounded to the nearest millisecond
• log
  • .log.level is now accessed within the .log.current dictionary
  • .log.currentLogger is now accessed within the .log.current dictionary
  • The .log.loggers functions have been split into .log.formatter and .log.loggers functions (see adding your own logger)
• type
  • .type.isGenericList was a duplicate of .type.isMixedList so was removed

Full Changelog

1f8f1b1 cron: Log when 'repeat until' job reaches the end time
3d69e3c cron: Add 'tickless' mode
e51e9fe http: Add PATCH request type
195169a http: Fix response parsing to not split body by "\r\n"
39e2f58 http: Add support for kdb IPC decoding
43909a9 ns: .ns.getFunctionArguments / .ns.protectedExecute upgrades [fixes #4]
2c07f5b http: Fix relative redirects and support 'x-gzip' as Content-Encoding
ae3086c log: Add support for pattern-based logging and function introspection
8ae9b2c require: Prevent re-init on subsequent .require.init calls
89c15e9 type: Remove .type.isGenericList (duplicate of .type.isMixedList)
137f663 ns: Add new .ns.deleteReference
f9d26d1 log: Fix incorrect change to interface update function call
29cd6aa log: Add support for slf4j-style parameterised logging
94655c8 cargs: Cache parsed results on first call
e039e09 util: Fix .util.showTabbed
2527cd0 convert: Fix documentation formatting
959c89a type: Make .type.ensureString deal with lists better
140e615 type: Add .type.isAnymap
8d6a135 type: Performance improvements (within -> in)
d5dea8f env / so: Add missing wiki documentation link

This release introduces the following major changes:

• New env library - Environment Variable Manager
• New so library - Shared Object Function Manager
• Mac OS X (m) support within the os library

See the full changelog before for all other changes.

Full Changelog

3cfde52 Add new 'so' library - shared object function manager
9e220a7 Add new 'env' library - environment library manager
14bb59b os: Add PATH-type environment variable names and separators
9746b8b convert: Add '.convert.bytesToLong'
6143040 require: Simplify log interface functions
6d70d38 README: Remove contributors section, listed natively in GitHub
83a79f7 Added osx commands
5ffd485 type: Allow symbol lists in .type.ensureSymbol

This release introduces a new library if and makes changes to the default logging functions that all the libraries within kdb-common.

Interface Library (if)

This library has been added to allow the definition and implementation of a particular functon to be decoupled. This allows a library to reference function with knowing what the implementation of that function is.

In this version, all logging function references within all libraries within kdb-common have been updated.

NUC

The following function references have been updated:

• .log.trace -> .log.if.trace
• .log.debug -> .log.if.debug
• .log.info -> .log.if.info
• .log.warn -> .log.if.warn
• .log.error -> .log.if.error
• .log.fatal -> .log.if.fatal

Note that if you use the kdb-common logging library (log.q) then the the original functions exist as the implementation of these interfaces.

Full Changelog

d64d4c8 src: Migrate all logging to new interface versions .log -> .log.if [#41]
db73602 Add new "Interface" library [#41] 

This release includes 3 new library - http, wsc and terminal.

HTTP Query Library (http)

This library provides HTTP GET, POST, PUT and DELETE interfaces with additional features from the default GET (.Q.hg) and POST (.Q.hp):

• Additional headers can be sent with each request
• The "User-Agent" header will be sent (if configured), to identify the source process
• HTTP repsonses are parsed into a dictionary for easier processing of HTTP result
  • Includes automatically converting JSON responses with .j.k
• Automatically follow redirects
• Automatic URL encoding of query string fields and values

The following features present in .Q.hg and .Q.hp are also supported:

• Routing via proxy if HTTP_PROXY or HTTPS_PROXY set
• Bypassing proxy if target URL present in NO_PROXY
• Basic authentication supported with http://username:password@host syntax
• GZIP encoding request and decoding (only with kdb+ 4.0 or greater)

WebSocket Client Library (wsc)

Provides the ability to initiate WebSocket connections easily via .wsc.connect

Terminal (Console) Management (terminal)

This library can track changes to the current terminal window that kdb+ is running in and adjust the kdb console size (system "c") to match it.

The adjustment can be done manually (via .terminal.setToCurrentSize) or automatically via the .z.pi event hander.

Full Changelog

a82f6a9 http: Add URL encoding of query string fields and values
aeb996f http: Add PUT and DELETE requests
9bc06bc http: Add auto-parsing of JSON responses and support redirects
2b17893 Add new 'http' library as a replacement for .Q.hg / .Q.hp / .Q.hmb [#38]
5af6174 Add new WebSocket client connection library [#36]
ff3959a Add new 'terminal' library for automated console size tracking [#34]
f1bf144 util: Add new .util.isTlsAvailable

This release includes the following changes to the following libraries - type and convert.

Type Checking and Normalisation (type)

• Add: .type.is* and .type.is*List functions were added for all standard kdb types (previously inconsistently available)
• Add: .type.isEnumeratedTable which returns true if one or more columns of the specified table are enumerated
• Fix: .type.isHostPort now correctly validates the type of a port only value as long (previously accepted short, integer and longs)
• NUC: .type.isInteger now only checks for kdb integer type (-6h)
  • The previous behaviour for .type.isInteger is now .type.isWholeNumber

Type Conversions (convert)

• Add .convert.tableToHtml which will output a valid HTML <table> element for the specified kdb table

Full Changelog

258388a type: Add .type.isEnumeratedTable
18ac355 convert: Add new .convert.tableToHtml
2767bc0 type: Add is* and is*List type checking functions for all types [#29]

This release contains a single bug fix to the ipc library.

Inter-Process Communication (ipc)

• The timeout parameter in .ipc.connectWithTimeout was not used within the hopen [#27]

Full Changelog

400303a ipc: Bug fix - not using timeout in .ipc.connectWithTimeout [#27]

This release includes changes to the ipc library and a new cargs library.

This release also changes the release naming. The final part of the release name will now be the release date (matching the kdb+ versioning system).

The name will have the following scheme: kdb-common-*major*.*minor*.*patch*-*release-date*

Inter-Process Communication (ipc)

.ipc.outbound has been updated to also not log the password if .ipc.cfg.logPasswordsDuringConnect has been set to false, matching the logging behaviour.

Enhanced Command-Line Argument Parser (cargs)

This new library provides additional parsing features on top of the standard .Q.opt functionality. See https://github.com/BuaBook/kdb-common/wiki/cargs.q for more information.

Full Changelog

32f768f Add new enhanced command-line argument parser
dbf07b3 ipc: Do not log password in .ipc.outbound if .ipc.cfg.logPasswordsDuringConnect is false [#24]