We introduce a novel loss function, Layout and Character Oriented Focal Loss (LCOFL), which considers factors such as resolution, texture, and structural details, as well as the performance of the License Plate Recognition (LPR) task itself. We enhance character feature learning using deformable convolutions and shared weights in an attention module and employ a GAN-based training approach with an Optical Character Recognition (OCR) model as the discriminator to guide the super-resolution process. Our results show significant improvements in character reconstruction quality, outperforming two state-of-the-art methods in both quantitative and qualitative measures.
The High-Resolution (HR) images used in our experiments were generated as follows. For each image from the RodoSol-ALPR dataset, we first cropped the License Plate (LP) region using the annotations provided by the creators of the dataset. We then used the same annotations to rectify each LP image, making it more horizontal, tightly bounded, and easier to recognize. The rectified images serve as the HR images.
We generated Low-Resolution (LR) versions of each HR image by simulating the effects of an optical system with lower resolution. This was achieved by iteratively applying random Gaussian noise to each HR image until we reached the desired degradation level for a given LR image (i.e., SSIM < 0.1). We padded the LR and HR images to maintain the aspect ratio before resizing. This process was described in our previous work.
Here are some HR-LR image pairs created:
The RodoSol-SR dataset is released for academic research only and is free to researchers from educational or research institutes for non-commercial purposes.
To be able to download the dataset, please read this license agreement carefully, fill it out and send it back to the second author (rblsantos@inf.ufpr.br) (who also manages access to the RodoSol-ALPR dataset). Your e-mail must be sent from a valid university account (.edu, .ac or similar).
In general, you will receive a download link within 3-5 business days. Failure to follow the instructions may result in no response.
This section provides instructions on testing the model, training it from scratch, and fine-tuning it on a custom dataset. Follow the steps below to set up and run the model. Additionally, the Optical Character Reader (OCR) used in this work for training was sourced from GP_LPR by Liu et al.
To test the model, ensure that the config file specifies the path to the .pth file, as shown in the example below:
model:
name: cgnetV2_deformable
load: ./save/_cgnetV2_deformable_test/best_model_cgnetV2_deformable_Epoch_82.pth
args:
in_channels: 3
out_channels: 3
Then, run the following command:
python3 test.py --config ./config/Sibgrapi_ablation/cgnetV2_deformable.yaml --save True
To train the model from scratch, set the following variables in the config file to null, as shown below:
LOAD_PRE_TRAINED_OCR: null
resume: null
Then, execute the following command:
python3 ParallelNetTrain.py --config ./config/Sibgrapi_ablation/cgnetV2_deformable_test.yaml --save True
To train or fine-tune the model on a custom dataset, ensure that a .txt file containing the paths to the cropped and rectified images is formatted as shown below:
path/to/HR1_images.jpg;path/to/LR1_images.jpg;training
path/to/HR2_images.jpg;path/to/LR2_images.jpg;validation
path/to/HR3_images.jpg;path/to/LR3_images.jpg;testing
Then, modify the config file to include the path to the .txt file:
train_dataset:
dataset:
name: parallel_training
args:
path_split: ./path/to/file.txt
phase: training
wrapper:
name: parallel_images_lp
args:
imgW: 48
imgH: 16
aug: True
image_aspect_ratio: 3
background: (127, 127, 127)
batch: 2
val_dataset:
dataset:
name: parallel_training
args:
path_split: ./path/to/file.txt
phase: validation
wrapper:
name: parallel_images_lp
args:
imgW: 48
imgH: 16
aug: False
image_aspect_ratio: 3
background: (127, 127, 127)
batch: 2
If you use our code in your research, please cite:
- V. Nascimento, R. Laroca, R. O. Ribeiro, W. R. Schwartz, D. Menotti, “Enhancing License Plate Super-Resolution: A Layout-Aware and Character-Driven Approach,” in Conference on Graphics, Patterns and Images (SIBGRAPI), pp. 1-6, Sept. 2024. [IEEE Xplore] [arXiv]
@article{nascimento2024enhancing,
title = {Enhancing License Plate Super-Resolution: A Layout-Aware and Character-Driven Approach},
author = {V. {Nascimento} and R. {Laroca} and R. O. {Ribeiro} and W. R. {Schwartz} and D. {Menotti}},
year = {2024},
journal = {Conference on Graphics, Patterns and Images (SIBGRAPI)},
volume = {},
number = {},
pages = {1-6},
doi = {10.1109/SIBGRAPI62404.2024.10716303},
issn = {1530-1834},
}
You may also be interested in our previous work. If you use the LR-HR image pairs we created for our experiments, please cite it:
- V. Nascimento, R. Laroca, J. A. Lambert, W. R. Schwartz, D. Menotti, “Super-Resolution of License Plate Images Using Attention Modules and Sub-Pixel Convolution Layers,” in Computers & Graphics, vol. 113, pp. 69-76, 2023. [Science Direct] [arXiv]
A list of all our papers on vehicle identification can be seen here.
Please contact Valfride Nascimento (vwnascimento@inf.ufpr.br) with questions or comments.