An open-source scientific computing library for embedded systems running Zephyr OS or standalone.
The Zephyr Scientific Library (zscilib) is an attempt to provide a set of functions useful for scientific computing, data analysis and data manipulation in the context of resource constrained embedded hardware devices.
It is written entirely in C, and while the main development target for the library is the Zephyr Project, it tries to be as portable as possible, and a standalone reference project is included to use this library in non-Zephyr-based projects.
This version of zscilib has been developed and tested against Zephyr 3.3.0.
As the processing power of small, embedded MCUs increases and costs fall, more computation can be done on the endnode itself. This allows for more of the 'complex' data analysis that used to take place at the PC or server level (data aggregation, statistical analysis, etc.) to be done in less time, using less data storage, and at a lower overall processing cost on small, embedded devices.
A key goal of zscilib is to allow more data processing to happen on the endnode.
By generating scientifically-relevant data points (standard SI units,
pre-filtered data, etc.) directly on the endnode, zscilib aims to be a bridge
between raw data and more numerically complex toolkits like gsl
, numpy
or R
.
Numerous high quality, mature, open source scientific libraries already exist:
Despite the wealth of mature functions in these existing libraries, though, they tend to have the following two problems in an embedded context:
The second item is of particular importance, since the goal of embedded systems is often 'sensing' via raw data, correlating that data, and acting on the final data points or passing them on for further analysis.
CMSIS-DSP contains a number of highly efficient algorithms for filtering raw sensor data, but it doesn't offer any domain-specific assistance converting filtered accelerometer vectors into orientation data, for example, or reading a set of photodiodes and converting that data into a useful photometric value like lux. It is excellent at 'conditioning' data, but not at 'understanding' it.
zscilib aims to find a middle ground between these two, allowing for richer processing of raw data, but within the confines and limitations of the class of microcontrollers commonly used on low-cost sensor endnodes.
A few makefile-based projects are included in samples/standalone
showing
how zscilib can be used independent of Zephyr.
If you already have an appropriate GNU toolchain and build tools (make
, etc.)
installed, you can simply execute the following commands:
$ cd samples/standalone/svd_pinv
$ make
$ bin/zscilib
Hello, zscilib!
...
To run one of the sample applications using qemu, run the following commands:
Be sure to run
source zephyr/zephyr-env.sh
(OS X or Linux) or.\zephyr\zephyr-env.cmd
(Windows) before the commands below! This also assumesqemu-system-arm
is available on your local system.
$ west build -p -b mps2/an521/cpu0 \
modules/lib/zscilib/samples/matrix/mult -t run
...
*** Booting Zephyr OS build zephyr-v2.6.0-536-g89212a7fbf5f ***
zscilib matrix mult demo
mtx multiply output (4x3 * 3x4 = 4x4):
14.000000 17.000000 20.000000 23.000000
35.000000 44.000000 53.000000 62.000000
56.000000 71.000000 86.000000 101.000000
7.000000 9.000000 11.000000 13.000000
Press CTRL+A
then x
to quit qemu.
To run the unit tests for this library, run the following command:
$ twister --inline-logs -p mps2/an521/cpu0 -T modules/lib/zscilib/tests
See the tests
folder for further details.
To run compliance tests to make sure submitted code matches Zephyr PR
requirements, run this (updating HEAD~2
for the number of commits to check,
or setting it to origin/master..
to check everything):
$ ../../../zephyr/scripts/ci/check_compliance.py \
-m Gitlint -m Identity -m Nits -m pylint -m checkpatch \
-c HEAD~2..
If you wish to debug using QEMU (and with minor variation actual hardware), you can run the following commands to start a new GDB debug session.
For an overview of debugging in GDB, you may find the following presentation useful: LVC21-308: Essential ARM Cortex-M Debugging with GDB
In one terminal window, run:
$ west build -p auto -b mps2/an521/cpu0 modules/lib/zscilib/samples/matrix/pinv
Once the ELF file has been built, we can start a GDB server on the default
1234
socket, and wait for a new connection via:
$ cd build
$ ninja debugserver
In a new terminal window, connect to the GDB server via:
$ cd <zephyr_path>
$ arm-none-eabi-gdb-py \
--eval-command="target remote localhost:1234" \
--se=build/zephyr/zephyr.elf
The
-py
extension is optional, and makes use of a version of GDB from the ARM GNU toolchain releases that enables Python scripts to be used with your debug sessions. See the LVC21-308 presentation at the top of this section for details.
From here, you can start debugging with the (gdb)
prompt.
For example:
(gdb) b main
(gdb) c
Continuing.
Breakpoint 1, main () at modules/lib/zscilib/samples/matrix/pinv/src/main.c:70
70 printf("\n\nzscilib pseudo-inverse demo\n\n");
(gdb) n
72 pinv_demo();
(gdb) step
pinv_demo () at modules/lib/zscilib/samples/matrix/pinv/src/main.c:25
25 zsl_real_t vi[18 * 3] = {
(gdb) n
...
(gdb) quit
zscilib can be configured to make use of single-precision (32-bit) or
double-precision (64-bit) floating point values via the
CONFIG_ZSL_SINGLE_PRECISION
flag, which will determine the size of
zsl_real_t
used throughout the library. The default setting for this
flag is n
, meaning 64-bit values are used by default.
There is a tradeoff between the added range and precision that 64-bit (double-precision) floating point values offer, and the memory and performance gains of the smaller, less-precise but faster 32-bit (single-precision) operations.
Due to the reduced precision of single-precision values, certain complex functions in zscilib are only available when double-precision is enabled (PINV, SVD, etc.).
The sample code in this library typically has the CONFIG_FPU
option set,
meaning that floating-point support is configured for
Unshared FP registers mode. This mode is used when the application
has a single thread that uses floating point registers.
If your application makes use of multiple threads, and more than one of
these threads uses floating-point operations, you should also enable the
CONFIG_FPU_SHARING
config flag, which configures the kernel for
Shared FP registers mode. In this mode, the floating point registers are
saved and restored during each context switch, even when the associated threads
are not using them. This feature comes at the expense of an extra 72 bytes of
stack memory per stack frame (s0..s15
+ FPCSR
, plus an alignment word
to ensure that the stack pointer is double-word aligned).
Features marked with the v0.2.0 flag are in progress or planned as part of the current release cycle, and may be partially implemented or stubbed at present. v0.3.0 indicates features planned for that later release.
Feature | Func | f32 | f64 | Arm | Notes |
---|---|---|---|---|---|
Array to vector | zsl_vec_from_arr |
x | x | ||
Copy | zsl_vec_copy |
x | x | ||
Get subset | zsl_vec_get_subset |
x | x | ||
Add | zsl_vec_add |
x | x | ||
Subtract | zsl_vec_sub |
x | x | ||
Negate | zsl_vec_neg |
x | x | ||
Sum | zsl_vec_sum |
x | x | 2 or more vects | |
Scalar add | zsl_vec_scalar_add |
x | x | ||
Scalar multiply | zsl_vec_scalar_mult |
x | x | ||
Scalar divide | zsl_vec_scalar_div |
x | x | ||
Distance | zsl_vec_dist |
x | x | Between 2 vects | |
Dot product | zsl_vec_dot |
x | x | ||
Norm/abs value | zsl_vec_norm |
x | x | ||
Project | zsl_vec_project |
x | x | ||
To unit vector | zsl_vec_to_unit |
x | x | ||
Cross product | zsl_vec_cross |
x | x | ||
Sum of squares | zsl_vec_sum_of_sqrs |
x | x | ||
Comp-wise mean | zsl_vec_mean |
x | x | ||
Arithmetic mean | zsl_vec_ar_mean |
x | x | ||
Reverse | zsl_vec_rev |
x | x | ||
Zero to end | zsl_vec_zte |
x | x | 0 vals to end | |
Equality check | zsl_vec_is_equal |
x | x | ||
Non-neg check | zsl_vec_is_nonneg |
x | x | All values >= 0 | |
Contains | zsl_vec_contains |
x | x | ||
Quicksort | zsl_vec_sort |
x | x | ||
zsl_vec_print |
x | x |
Feature | Func | f32 | f64 | Arm | Notes |
---|---|---|---|---|---|
Array to matrix | zsl_mtx_from_arr |
x | x | ||
Copy | zsl_mtx_copy |
x | x | ||
Get value | zsl_mtx_get |
x | x | ||
Set value | zsl_mtx_set |
x | x | ||
Get row | zsl_mtx_get_row |
x | x | ||
Set row | zsl_mtx_set_row |
x | x | ||
Get col | zsl_mtx_get_col |
x | x | ||
Set col | zsl_mtx_set_col |
x | x | ||
Add | zsl_mtx_add |
x | x | ||
Add (d) | zsl_mtx_add_d |
x | x | Destructive | |
Sum rows | zsl_mtx_sum_rows_d |
x | x | Destructive | |
Sum rows scaled | zsl_mtx_sum_rows_scaled_d |
x | x | Destructive | |
Subtract | zsl_mtx_sub |
x | x | ||
Subtract (d) | zsl_mtx_sub_d |
x | x | Destructive | |
Multiply | zsl_mtx_mult |
x | x | ||
Multiply (d) | zsl_mtx_mult_d |
x | x | Destructive | |
Multiply sc (d) | zsl_mtx_scalar_mult_d |
x | x | Destructive | |
Multiple row sc (d) | zsl_mtx_scalar_mult_row_d |
x | x | Destructive | |
Transpose | zsl_mtx_trans |
x | x | ||
Adjoint 3x3 | zsl_mtx_adjoint_3x3 |
x | x | ||
Adjoint | zsl_mtx_adjoint |
x | x | ||
Wedge product | zsl_mtx_vec_wedge |
x | |||
Reduce | zsl_mtx_reduce |
x | x | Row+col removal | |
Reduce (iter) | zsl_mtx_reduce_iter |
x | x | Iterative ver. | |
Augment | zsl_mtx_augm_diag |
x | x | Adds row+col(s) | |
Determinant 3x3 | zsl_mtx_deter_3x3 |
x | x | ||
Determinant | zsl_mtx_deter |
x | x | ||
Gaussian El. | zsl_mtx_gauss_elim |
x | x | ||
Gaussian El. (d) | zsl_mtx_gauss_elim_d |
x | x | Destructive | |
Gaussian Rd. | zsl_mtx_gauss_reduc |
x | x | ||
Column norm. | zsl_mtx_cols_norm |
x | x | Unitary col vals | |
Gram-Schimdt | zsl_mtx_gram_schmidt |
x | x | ||
Elem. norm. | zsl_mtx_norm_elem |
x | x | Norm vals to i,j | |
Elem. norm. (d) | zsl_mtx_norm_elem_d |
x | x | Destructive | |
Invert 3x3 | zsl_mtx_inv_3x3 |
x | x | ||
Invert | zsl_mtx_inv |
x | x | ||
Balance | zsl_mtx_balance |
x | x | ||
Householder Ref. | zsl_mtx_householder |
x | x | ||
QR decomposition | zsl_mtx_qrd |
x | x | ||
QR decomp. iter. | zsl_mtx_qrd_iter |
x | |||
Eigenvalues | zsl_mtx_eigenvalues |
x | |||
Eigenvectors | zsl_mtx_eigenvectors |
x | |||
SVD | zsl_mtx_svd |
x | |||
Pseudoinverse | zsl_mtx_pinv |
x | |||
Min value | zsl_mtx_min |
x | x | ||
Max value | zsl_mtx_max |
x | x | ||
Min index | zsl_mtx_min_idx |
x | x | ||
Max index | zsl_mtx_max_idx |
x | x | ||
Equality check | zsl_mtx_is_equal |
x | x | ||
Non-neg check | zsl_mtx_is_notneg |
x | x | All values >= 0 | |
Symmetr. check | zsl_mtx_is_sym |
x | x | ||
zsl_mtx_print |
x | x |
The following component-wise unary operations can be executed on a matrix
using the zsl_mtx_unary_op
function:
++
)--
)-
)!
)The following component-wise binary operations can be executed on a pair
of symmetric matrices using the zsl_mtx_binary_op
function:
a + b
)a - b
)a * b
)a / b
)mean(a, b
)a^b
)min(a, b)
)max(a, b)
)a == b
)a != b
)a < b
)a > b
)a <= b
)a >= b
)NOTE: Component-wise unary and binary matrix operations can also make use of user-defined functions at the application level if the existing operand list is not sufficient. See
zsl_mtx_unary_func
andzsl_mtx_binary_func
for details.
[1] Only available in double-precision
The zsl_measurement
struct is a proof of concept attempt at representing
measurements in a concise but unambiguous manner.
It consists of:
There is an option to adjust the measurement's scale in +/- 10^n steps (Scale Factor) from the default SI unit and scale indicated by the SI Unit Type. For example, if 'Ampere' is indicated as the SI unit, the measurement could indicate that the value is in uA by setting the scale factor to -6.
zsl_measurement
)Help is welcome on the following planned or desirable features.
Basic tooling has been added to allow for optimised architecture-specific implementations of key functions in assembly.
At present, this feature isn't being actively used or developed, but an aim of zscilib is to add optimised versions of key functions to try to get the best possible performance out of limited resources.
Initial optimisation will target the Arm Cortex-M family of devices and the Thumb and Thumb-2 instruction sets, though other architectures can be accommodated if necessary or useful.
Since the primary target of this codebase is running as a module in Zephyr OS, it follows the same coding style, which is itself based on the Linux kernel coding style.
You can format the source code to match this style automatically using the uncrustify command line tool, which has plugins available for many common text editors (Atom Beautify, for example).
If you wish to contribute to this library, you can raise a PR as follows:
git clone
your forked repository.Also have a look at the Issues page to see if there is any outstanding work or issues that you might be able to help with!
Apache 2.0.