Exploring Font-independent Features for Scene Text Recognition
This is the official Tensorflow implementation of the paper:
Yizhi Wang and Zhouhui Lian. Exploring Font-independent Features for Scene Text Recognition. ACM Multimedia. 2020.
Novel Font Scene Text (NFST) Dataset
As scene texts in novel font styles only make up a small proportion in existing benchmarks, we collect 100 text images with novel or unusual font styles to form a new dataset named as the Novel Font Scene Text (NFST) dataset (download link).
We compare our method with other two state-of-the-art methods (ASTER and SAR) whose codes are publicly available. Our method significantly outperforms others on this dataset (see the following table), whose robustness to font style variance is proved.
| Training data | Ours | SAR | ASETR |
|---|---|---|---|
| 90K+ST | 55% | 45% | 44% |
| 90K+ST+SA+R | 71% | 63% | 58% |
Installation
Requirement
- python 3.6
- TensorFlow 1.15
Please use Anaconda to build the environment (otherwise, some errors may occur):
conda create -n efifstr python=3.6
source activate efifstr
pip install editdistance edit_distance
conda install tensorflow-gpu=1.15 pillow matplotlib
We reuse some code from ASTER, which is derived from Tensorflow Object Detection API.
Build the custom operators and protobuf files
- Go to
c_ops/and runbuild.shto build the custom operators - Execute
protoc protos/*.proto --python_out=.to build the protobuf files
Pretrained-model
The pretrained model files are available on this link (Update: new link has better performance). Download pretrained-model.zip and extract it under experiments/demo/log/.
To test the preformance of the pretrained-model, simply execute:
python eval.py
This will output the recognition result of NFSF dataset.
To test on a single image (data/demo.jpg), simply execute:
python demo.py
Training and on-the-fly evaluation
Data preparation scripts for several popular scene text datasets are located under tools. See their source code for usage.
The multi-font glyph images are available on this link. Download glyphs-325-fonts.npy and put it under data/.
To run the example training, run
python train.py
Change the configuration in experiments/demo/trainval.prototxt to configure your own training process.
During the training, you can run a separate program to repeatedly evaluates the produced checkpoints.
python eval.py
Evaluation configuration is also in trainval.prototxt.