Nnom Versions Save

Notes

It has been a year since the last release. Thanks to the people that contributed to this project, there are many accumulated bug fixing, and changes. It is now the time to make a final release on version 0.4.x. The main branch will become stable-0.4.x.

It does not mean that the project will end here. on the contrary, the new 0.5.x will come and there will be major changes in interfaces. This is how the version tag works. 0.5.x will not fully compatible with 0.4.x. So we can drop deprecated and clean up the lib.

Major updates:

• Set NNOM_TRUNCATE off by default https://github.com/majianjia/nnom/issues/100
• Add static buffer interface. Users can allocate a large chunk of memory for nnom to use. Therefore to avoid the need of malloc()
• new api nnom_set_static_buf() to be call before any model initialization.
• new macro NNOM_USING_STATIC_MEMORY in nnom_port.h to control memory type.
• Add reshape layer.

bug fixed:

• fixed ROUND() perform incorrectly, see https://github.com/ARM-software/CMSIS_5/issues/1047

minor

• update SCopnstruct to support linux build. tested on ubuntu WSL 18.04 @idog-ceva
• replace "acc" by "accuracy" for TF2.3.x
• Add gaussian noise as a skipable layer in inference mode @mle-invn
• export nnom_memcpy() and nnom_memset() in nnom_port.h
• Update mfcc related example by @HadeelMabrouk

Major updates:

Calibrations changes because of RNN layers.

• The calibration data size will no longer cut to 100 automatically.
• It is no longer shuffle the data for the user.
• User must cut the calibration sample to the size (i.e. 100) and shuffle the data when needed.
• In quantisation with max-min, allow the very small number to be saturated, such as 1.00001

Support model with multiple outputs:

• currently, the output data buff naming are nnom_output_data[], nnom_output_data1[], nnom_output_data2[]...

Add RNNoise like Voice Enhancement example:

• With well documented and demo

Depthwise Conv layers are now supported depth_multiplier arguments.

• Simply use it in Keras.

Bugs fixed:

• Update which solved the issue conv2d 1*1 with strides!=1 and cmsis-nn #84.
• RNN not passing correct Q format to the next layer.
• Deleting model causes Segment Fault
• Compiler stuck at some points with multiple output model.
• DW conv and Conv cannot calculate the correct kernel range near border of the image with padding.

Minors:

• fixed hard sigmoid, fixed compiling warning of multiple outputs
• update the KWS example's MFCC C code to align with python's MFCC. Accuracy should improve a lot.
• Improve performance of local backends (DW Conv)

The major change for this version is to support the RNN layers.

New RNN layers

• Support options: return_sequence, stateful and go_backewards
• Support RNN cells: Simple RNN, LSTM and GRU
• GRU and LSTM run on 16bit internally.

New Activations

• AdvanceReLU -> is an equivalence to Keras ReLU with any argment(s) i.e. slope, threshold and max. Also those predefine ReLU such as ReLU6
• Hard TanH and Hard Sigmoid -> currently only in backend.

New Examples

• New uci-har-rnn demonstrate the usage of new RNN layers.
• Add an RNN model in kws example.

Minor:

• Completed the Q format calculation for input/output tensor. Add new method model_io_format() to print the layer io info.

Bugs:

• Fixed concatenate on 2D input.

Important: This update will not support your previous weights.h generated by the old scripts, only because an argument, dilation was added to Conv2D and DW_Conv2D. You shall use the new script to regenerate your old model.

Major changes

• Add a new type of API "structured API", marked with _s suffix. This is a set of API taking a C structure as arguments for a layer.
• Add a tools nnom.py to generate the nnom model based on NNoM structured API.
• Add supports for Channel-wise quantization in Convolutional Layers
• Add a new data type: Tensor. All operations are now based on the tensor.
• Weights and bias are now stored in tensor.
• Scripts have switched from pure Keras to Tensorflow.Keras. Support TF1.14+ and TF2.1+

Layer updates

• Add dilation supports for Conv2D and DepthwiseConv2D
• Add Transpose Convolution (Not working yet)
• Add supports for LeakyReLU.

Others

• nnom_shape_t to nnom_3d_shaped_t
• change LAYER_BUF_* macros to NNOM_TENSOR_BUF_* for clarity.

Bug fixed:

• Fix bugs in CHW mode flatten tensor_chw2hwc()
• Fix bugs in upsampling_build()
• Fix dimention calculation error in global_pooling_builtd()

• Re-organized files structure. Now we have layer-based files.
• nnom_out_shape.c and all *_out_shape() now changed to *_build() and placed in each layer-based files.
• Add a tensor type. Now all operators (activation/conv, dense...) will operate directly on the tensor.
• Now tensor will be created separately. A tensor is shared between layers (more or less similar to the previous memory block does).

• Backend fully supports CHW format. Ready for MCU with hardware accelerator.
• Flatten layer now flatten CHW format to HWC.
• Input layer auto-convert HWC image to CHW when using CHW format.
• Implement layer-wise KLD quantisation. (KLD only apply for convolution)
• Activation quantisation now uses 1000 samples for calibration instead of the whole dataset.
• nnom_predict() now provide probability as output. It now supports single neural output.
• fixed pointer address cut-off in Win64 platform
• Add a new example auto-test, for Travis CI and all PC platforms.
• Update other examples for new APIs.
• Fixed bugs in TanH, Sigmoid when the integer bit width is smaller than 0.
• Documentation

Known issues:

• Batch normalisation after depthwise convolution is not working properly. Temporary solution: use batch normalisation after the pointwise convolution (in a depthwise-pointwise structure).

• The script does not support implicitly defined activation. e.g. Dense(32, activation='relu'). Temporary solution: use explicitly activation. e.g.

Dense(32)
ReLU()

Note: KLD quantisation. Ref: http://on-demand.gputechconf.com/gtc/2017/presentation/s7310-8-bit-inference-with-tensorrt.pdf

• Add docs for examples
• Rebuilt most of the examples
• Add Zero Padding Layer and Cropping Layer
• Add new properties border_t
• Bugs fixed.

Update Log

• License is now changed to Apache-2.0 from this version.
• Add more examples. (Octave conv, keyword spotting...)
• Update compiling interfaces.
• Change porting interface. (LOG -> NNOM_LOG)
• Add an up sampling layer.
• Add, Sub and Mult are now supported by nnom_utils.py to convert from Keras directly.
• Sigmoid and TanH are now working properly (related to https://github.com/majianjia/nnom/issues/13).
• Add a layer_callback() interface, which called after every layer has finished its operation.
• Bugs fixed.

Fixed memory buffer calculation mistakes.

• Bugs fixed since the Alpha release.
• Improve documentations
• Add up-sampling layers