A fast, ergonomic and portable tensor library in Nim with a deep learning focus for CPU, GPU and embedded devices via OpenMP, Cuda and OpenCL backends
This release is named after "Memories of Ice" (2001), the third book of Steven Erikson epic dark fantasy masterpiece "The Malazan Book of the Fallen".
Changes :
toUnsafeView
as replacement of dataArray
to return a ptr UncheckedArray
cumsum
, cumprod
Thanks to @Vindaar for maintaining the repo, the docs, pretty-printing and answering many many questions on Discord while I took a step back. Thanks to @filipeclduarte for the cumsum/cumprod, @Clonkk for updating raw data accesses, @struggle for finding a bug in mean square error backprop, @timotheecour for spreading new upstream requirements downstream and @anon767 for Graph Neural Network.
v0.6.0 only had v0.5.2 in nimble file
This release is named after "Windwalkers" (2004, French "La Horde du Contrevent"), by Alain Damasio.
Changes:
symeig
proc to compute eigenvectors of a symmetric matrix
now accepts an "uplo" char parameter. This allows to fill only the Upper or Lower
part of the matrix, the other half is not used in computation.svd_randomized
, a fast and accurate SVD approximation via random sampling.
This is the standard driver for large scale SVD applications as SVD on large matrices is very slow.pca
now uses the randomized SVD instead of computing the covariance matrix.
It can now efficiently deal with large scale problems.
It now accepts a center
, n_oversamples
and n_power_iters
arguments.
Note that pca
without centering is equivalent to a truncated SVD.hilbert
has been introduced. It creates the famous ill-conditioned Hilbert matrix.
The matrix is suitable to stress test decompositions.arange
procedure has been introduced. It creates evenly spaced value within a specified range
and step(y_pred, y_target)
(from (y_target, y_pred)), enabling the syntax y_pred.accuracy_score(y)
.
All existing error functions in Arraymancer were commutative w.r.t. to arguments
so existing code will keep working.solve
procedure has been added to solve linear system of equations represented as matrices.softmax
layer has been added to the autograd and neural networks
complementing the SoftmaxCrossEntropy layer which fused softmax + Negative-loglikelihood.Bug fixes:
gemm
could crash when the result was column major.(A * X) + b
could update the b matrix.$
of number types due to ambiguous call.Breaking:
symeig
, the eigenvectors
argument is now called return_eigenvectors
.symeig
with slice, the new uplo
precedes the slice argument.pca
overload that projected a data matrix on already existing principal axes
was removed. Simply multiply the mean-centered data matrix with the components instead.Deprecation:
+.
, *.
, /.
, -.
, ^.
, +.=
, *.=
, /.=
, -.=
, ^.=
instead of the previous order .+
and .+=
.
This allows the broadcasting operators to have the same precedence as the
natural operators.
This also align Arraymancer with other Nim packages: Manu and NumericalNimThanks to @dynalagreen for the SGD with Momentum, @xcokazaki for spotting the in-place division typo, @Vindaar for fixing the automatic matrix multiplication and addition fusion, @Imperator26 for the Softmax layer, @brentp for reviewing and augmenting the SVD and PCA API, @auxym for the linear equation solver and @berquist for the reordering all error functions to the new API. Thanks @b3liever for suggesting the dot change to solve the precedence issue in broadcasting and elementwise operators.
Fix packaging bug in ".nimble" and updated readme/changelog
Changes affecting backward compatibility:
Changes:
Einsum
Fix:
Thanks to @chimez for the complex support and updating for 0.20, @metasyn for the tokenizer, @xcokazaki for the image dimension fix and @Vindaar for the einsum implemention
This release is named after "Sign of the Unicorn" (1975), the third book of Roger Zelazny masterpiece "The Chronicles of Amber".
Changes affecting backward compatibility:
pca(x: Tensor, nb_components: int)
now returns a tuple
of result and principal components tensors in descending order instead of just a resultpca(x: Tensor, principal_axes: Tensor)
will project the input x
on the principal axe suppliedChanges:
Datasets:
IO:
Machine learning
Neural network and autograd:
Tensors:
index_select
toSeq
exportsEnd-to-end Examples:
Important fixes:
Special thanks to @metasyn (MNIST caching, IMDB dataset, Kmeans) and @Vindaar (HDF5 support and the example of using Arraymancer + Plot.ly) for their large contributions on this release.
Ecosystem:
Using Arraymancer + Plotly for NN training visualisation: https://github.com/Vindaar/NeuralNetworkLiveDemo
Monocle, proof-of-concept data visualisation in Nim using Vega. Hopefully allowing this kind of visualisation in the future:
and compatibility with the Vega ecosystem, especially the Tableau-like Voyager.
Agent Smith, reinforcement learning framework.
Currently it wraps the Arcade Learning Environment
for practicing reinforcement learning on Atari games.
In the future it will wrap Starcraft 2 AI bindings
and provides a high-level interface and examples to reinforcement learning algorithms.
Laser, the future Arraymancer backend which provides:
im2col
solutions can only reach 16% of matrix multiplication efficiency on the common deep learning filter sizesFuture breaking changes.
Arraymancer backend will switch to Laser
for next version.
Impact:
map_inline
, map2_inline
, map3_inline
, apply_inline
, apply2_inline
, apply3_inline
, reduce_inline
, fold_inline
, fold_axis_inline
will be removed and replace by forEach
and forEachStaged
with the following syntax:forEach x in a, y in b, z in c:
x += y * z
Both will work with an arbitrary number of tensors and will generate 2x to 3x more compact code wile being about 30% more efficient for strided iteration. Furthermore forEachStaged
will allow precise control of the parallelisation strategy including pre-loop and post-loop synchronisation with thread-local variables, locks, atomics and barriers.
The existing higer-order functions map
, map2
, apply
, apply2
, fold
, reduce
will not be impacted. For small inlinable functions it will be recommended to use the forEach
macro to remove function call overhead (Yyou can't inline a proc parameter).
The neural network domain specific language will use less magic
for the forward
proc.
Currently the neural net domain specific language only allows the type
Variable[T]
for inputs and the result.
This prevents its use with embedding layers which also requires an index input.
Furthermore this prevents using tuple[output, hidden: Variable]
result type
which is very useful to pass RNNs hidden state for generative neural networks (for example text sequence or time-series).
So unfortunately the syntax will go from the current
forward x, y:
shortcut to classic Nim proc forward[T](x, y: Variable[T]): Variable[T]
Once CuDNN GRU is implemented, the GRU layer might need some adjustments to give the same results on CPU and Nvidia's GPU and allow using GPU trained weights on CPU and vice-versa.
Thanks:
This release is named after "The Name of the Wind" (2007), the first book of Patrick Rothfuss masterpiece "The Kingkiller Chronicle".
Changes:
Core:
argmax
and argmax_max
procs.Datasets:
Linear algebra:
Machine Learning
Statistics
Neural network
Examples and tutorials
Tooling
Several updates linked to Nim rapid development and several bugfixes.
This release is named after "Wizard's First Rule" (1994), the first book of Terry Goodkind masterpiece "The Sword of Truth".
I am very excited to announce the third release of Arraymancer which includes numerous improvements, features and (unfortunately!) breaking changes.
Warning ⚠: ALL deprecated procs will be removed next release due to deprecated spam and to reduce maintenance burden.
Changes:
Very Breaking
let a = b
will share data by default and copies must be made explicitly.
unsafe
proc to avoid copies especially for slices.clone
to do copies.Breaking (?)
Documentation
Huge performance improvements
apply_inline
, map_inline
, fold_inline
and reduce_inline
templates are available.unlikely
A*B + C
and C+=A*B
are automatically fused in one operationNeural network:
linear
, sigmoid_cross_entropy
, softmax_cross_entropy
layersShapeshifting:
unsqueeze
and stack
Math:
min
, max
, abs
, reciprocal
, negate
and in-place mnegate
and mreciprocal
Statistics:
Broadcasting
.^
(broadcasted exponentiation)Cuda:
Examples
xor
function examplePrecision
ln1p
(ln(1 + x)
) and exp1m
procs (exp(1 - x)
) where appropriate to avoid catastrophic cancellationDeprecated
zeros
, ones
, newTensor
you will get a deprecated warning.newTensor
, zeros
, ones
arguments have been changed from zeros([5, 5], int)
to zeros[int]([5, 5])
unsafe
proc are now default and deprecated.This release is named after "The Colour of Magic" (1983), the first book of Terry Pratchett masterpiece "Discworld".
I am very excited to announce the second release of Arraymancer which includes numerous improvements blablabla
...
Without further ado:
Community
Breaking
shallowCopy
is now unsafeView
and accepts let
arguments.*
instead of |*|
dot
instead of .*
Deprecated
Backend
parameter deprecated.fmap
is now map
agg
and agg_in_place
are now fold
and nothing (too bad!)Initial support for Cuda !!!
Tensors
unsafe
operations that works without copy: unsafeTranspose
, unsafeReshape
, unsafebroadcast
, unsafeBroadcast2
, unsafeContiguous
,.+, .*, ./, .-
and their in-place equivalent .+=, .-=, .*=, ./=
squeeze
, at
and their unsafe
version.size
export_tensor
and toRawSeq
Ecosystem:
This release is named after "Magician: Apprentice" (1982), the first book of Raymond E. Feist masterpiece "The Riftwar Cycle".
First public release.
Include:
See README