CRNN-with-STN

implement CRNN in Keras with Spatial Transformer Network (STN) for Optical Character Recognition(OCR)

The model is easy to start a trainning, but the performance of recognition is not better than the original CRNN without STN.

You can run CRNN individually by just remove the STN components, and connect batchnorm_7 to x_shape. The CRNN can reach 90% of recognition accuracy.

Train on Synthetic Word Dataset realsed by M. Jaderberg et al. You can download the dataset HERE

How to Run

Just run the CRNN_with_STN.py script. My environment is Tensorflow 1.4.0, Keras 2.0.9. If you don't know how to install frameworks, please check Installing Deep Learning Frameworks on Ubuntu with CUDA support.

Reference

CRNN(Convolutional Recurrent Neural Network)

Shi, Baoguang, X. Bai, and C. Yao. "An End-to-End Trainable Neural Network for Image-based Sequence Recognition and Its Application to Scene Text Recognition." IEEE Transactions on Pattern Analysis & Machine Intelligence PP.99(2016):1-1.[arxiv]

STN(Spatial Transformer Network)

Max Jaderberg, Karen Simonyan,Andrew Zisserman and Koray Kavukcuoglu. "Spatial Transformer Network" [arxiv]

2019.03.13 UPDATE

I have tried to put STN part in the front of the network, between batchnorm_1 and conv_2, than the network didn't converge at all. Maybe I used STN in a wrong way?

Weights Download, 2019.05.31 UPDATE

1. ou can download trained weights CRNN_without_STN, and CRNN_with_STN. (Google Drive)

2. If you can't access google, here is the Baidu Netdisk share link, with password "89vw".

3. Added a DEMO for model predicting