R client for kdb+

Execute kdb+ queries from R for advanced high-performance analytics.

See Interfacing with R on code.kx.com.

Installation

# remove old package
if('rkdb' %in% rownames(installed.packages())) remove.packages('rkdb')
# install devtools
if(! 'devtools' %in% rownames(installed.packages())) install.packages('devtools')
library(devtools)
# install rkdb
devtools::install_github('kxsystems/rkdb', quiet=TRUE,INSTALL_opts=c("--no-multiarch"))
# to install rkdb of particular release
# devtools::install_github('kxsystems/[email protected]', quiet=TRUE)
library(rkdb)

First steps

Set up a connection

Start a kdb+ process to test the installation.

q -p 5000

Open a connection to it.

h <- open_connection('localhost',5000)

Hello kdb+

You can evaluate any kdb+ expression and its result will come back to R:

execute(h, '1+1')

    ## [1] 2

Assigning a variable in the q workspace also works:

execute(h, 'x:1+1') #assign x to 2

    ## NULL

execute(h, 'x') # get back the value

    ## [1] 2

Getting data from kdb+ to R

Kdb+ uses some basic types that might not have a direct equivalent in R. Note also that this is not a bijective operation. The conversions (from kdb to R) are:

Computing on kdb+

Rkdb provides a convenient way to retrieve computation done on the kdb+ side so you can have the best of both worlds:

kdb <- '
t: ([] x:1000#`a`b`c;y:1000#1f*til 10;z:1000#1f*til 4);
select sum y, dev z by x from t
'

execute(h, kdb)

    ##   x    y        z
    ## 1 a 1503 1.120709
    ## 2 b 1497 1.116689
    ## 3 c 1500 1.116689

One can for instance use R graphical capabilities:

kdb <- '
t: ([] x:1000#`a`b`c;y:1000#1f*til 10;z:1000#1f*til 4);
select y,z from t where x=`a
'

DF <- execute(h, kdb)
plot(DF$y, DF$z, main='scatter plot', xlab='y values', ylab='z values')

Getting data from R to kdb+

Evaluating kdb+ expressions using R objects

You can call kdb+ functions with R objects as arguments. They will be passed and converted to native kdb+ data types, and the kdb+ expression will be evaluated:

execute(h, "raze", list(c(1,2,3), c(4,5,6)))

    ## [1] 1 2 3 4 5 6

execute(h, "+", 2, 5)

    ## [1] 7

execute(h,'{`tmp set x}',data.frame(a=c(1,2,3),b=c("a","b","b")))

    ## [1] "tmp"

For example, here is how you can use the left-join function on two data frames:

DF1 <- data.frame(x=c('x','x','y','y'), y=1:4)
DF2 <- data.frame(x=c('x','y','z'), z=seq(10,30,10))
execute(h, "{[x;y] x lj `x xkey y}", DF1, DF2)

    ##   x y  z
    ## 1 x 1 10
    ## 2 x 2 10
    ## 3 y 3 20
    ## 4 y 4 20