摘要：
合集：AI案例-CV-交通
数据集：中国车牌字符数据集
数据集价值：提高车牌字符识别准确性
AI问题：目标检测和OCR
解决方案：Tensorflow1.x和Mask R-CNN模型
一、问题描述
中国车牌字符数据集是一个专门用于车牌识别系统的数据集，它包含了来自中国31个省份的车牌中的字符的图像数据，涵盖了中文字符、数字以及英文字母。
中国车牌字符数据集在多个领域具有显著的应用价值，特别是在车牌识别系统的开发、优化和应用中。数据集对车牌识别技术的影响包括：
- 提高识别准确性：数据集包含多种格式、大小和质量的车牌字符图片，有助于训练模型以适应不同的字符样式和场景，从而提高识别准确性。
- 支持多种应用场景：车牌识别系统广泛应用于交通监控、车辆管理、停车场自动化等领域，数据集的应用使得这些系统能够更准确地自动识别机动车号牌信息。
数据集对智能交通系统发展的贡献
- 推动技术进步：通过提供丰富的标注数据，数据集促进了基于深度学习的车牌识别方法的研究和发展，推动了智能交通系统的技术进步。
- 提升系统性能：数据集的多样性和复杂性使得研究人员能够训练出更强大的模型，这些模型在处理复杂场景下的车牌识别时表现更佳，从而提升了整个智能交通系统的性能。
中国车牌字符数据集对于推动车牌识别技术的发展、提升智能交通系统的性能以及促进相关应用场景的实现具有不可替代的价值。
二、数据集内容
单个车牌字符数据集
数据集包含了数字0到9、英文字母A到Z的图片，以及中国31个省份的简称汉字图片。每张图片的尺寸为20×20像素，适合用于深度学习模型的训练。数据集总计包含约16,151张数字和字母图像。
文件目录：dataset/ann
字符列表：
0
1
2
3
4
5
6
7
8
9
A
B
C
D
E
F
G
H
J
K
L
M
N
P
Q
R
S
T
U
V
W
X
Y
Z
zh_cuan
zh_e
zh_gan
zh_gan1
zh_gui
zh_gui1
zh_hei
zh_hu
zh_ji
zh_jin
zh_jing
zh_jl
zh_liao
zh_lu
zh_meng
zh_min
zh_ning
zh_qing
zh_qiong
zh_shan
zh_su
zh_sx
zh_wan
zh_xiang
zh_xin
zh_yu
zh_yu1
zh_yue
zh_yun
zh_zang
zh_zhe
车牌目标检测数据集
车牌目标检测数据集包括250张车辆图片，200张用于训练，50张用于测试，然后在这些图片上标注出车牌区域。
文件目录：dataset\carplate
样例：via_region_data.json
{
"川A09X20.jpg45148": {
"filename": "川A09X20.jpg",
"size": 45148,
"regions": [{
"shape_attributes": {
"name": "polygon",
"all_points_x": [320, 442, 447, 323],
"all_points_y": [281, 261, 310, 336]
},
"region_attributes": {}
}],
"file_attributes": {}
},
"川A019W2.jpg51190": {
"filename": "川A019W2.jpg",
"size": 51190,
"regions": [{
"shape_attributes": {
"name": "polygon",
"all_points_x": [322, 390, 397, 328, 328],
"all_points_y": [267, 190, 237, 317, 317]
},
"region_attributes": {}
}],
"file_attributes": {}
},
数据集版权许可协议
Deed – CC0 1.0 通用 – Creative Commons
三、方案样例
解决方案
已知一车牌图片，例如：
解决方案包括三个步骤：
1）使用Mask_RCNN算法进行定位车牌。
Mask R-CNN（Mask Region-based Convolutional Neural Network）是一种用于实例分割的深度学习算法。它扩展了Faster R-CNN算法，不仅能够检测图像中的物体，还能为每个检测到的物体生成高质量的分割掩码。
源码目录：Mask_RCNN
2）车牌字符分割
源码：character_segmentation.ipynb
操作步骤：
2.1）转换为灰度图像； 2.2）Canny边缘检测算：
Canny边缘检测算法是一种广泛使用的边缘检测技术，由John Canny于1986年提出。它以其低错误率、良好的定位能力和对噪声的鲁棒性而闻名。 2.3）标记并筛选区域 2.4）提取车牌中每个单个字符图像
3）使用CNN算法进行单个字符识别
安装tensorflow1.14
参考AI基础专栏中的《安装深度学习框架TensorFlow》文章中的安装方法。
conda config --add channels anaconda
conda create -n tensorflow1-14-p36 python=3.6
conda activate tensorflow1-14-p36
conda install tensorflow-gpu=1.14
验证：执行python
import tensorflow as tf
print(tf.__version__)
检查 TensorFlow 是否可以访问 GPU
import tensorflow as tf
# 检查 TensorFlow 是否可以访问 GPU
print("GPU 是否可用: ", tf.test.is_gpu_available())
# 获取 TensorFlow 可用的 GPU 设备列表
gpus = tf.test.gpu_device_name()
if gpus:
print("可用的 GPU 设备: ", gpus)
else:
print("没有检测到 GPU 设备")
输出：
2024-10-30 12:06:31.004756: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1326] Created TensorFlow device (/device:GPU:0 with 1350 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce MX150, pci bus id: 0000:02:00.0, compute capability: 6.1)
GPU 是否可用: True
可用的 GPU 设备: /device:GPU:0
安装scikit-image
scikit-image 是一个开源的 Python 图像处理库，提供了大量的图像处理算法和工具，可以用于图像分析、计算机视觉、机器学习等领域。
conda install -c conda-forge scikit-image
conda install matplotlib
安装Pillow6.2.2
Pillow 是一个 Python Imaging Library (PIL) 的分支，它提供了广泛的文件格式支持和强大的图像处理功能。为了适用于 TensorFlow 1.14 和 Python 3.6，推荐Pillow版本6.2.2。
conda install Pillow==6.2.2
Mask_RCNN配置
输入的图片大小为640*480，GPU内存为2G，配置IMAGE_RESIZE_MODE= square和IMAGE_SHAPE=[256 256 3]。训练前程序下载mask_rcnn_coco.h5模型文件。
配置类：Mask_RCNN\mrcnn\config.py
class Config(object):
"""Base configuration class. For custom configurations, create a
sub-class that inherits from this one and override properties
that need to be changed.
"""
# Name the configurations. For example, 'COCO', 'Experiment 3', ...etc.
# Useful if your code needs to do things differently depending on which
# experiment is running.
NAME = None # Override in sub-classes
...
class CarplateConfig(Config):
"""Configuration for training on the toy dataset.
Derives from the base Config class and overrides some values.
"""
# Give the configuration a recognizable name
NAME = "carplate"
# We use a GPU with 12GB memory, which can fit two images.
# Adjust down if you use a smaller GPU.
# IMAGES_PER_GPU = 2
IMAGES_PER_GPU = 1
# Number of classes (including background)
NUM_CLASSES = 1 + 1 # Background + carplate
# Number of training steps per epoch
STEPS_PER_EPOCH = 100
# Skip detections with < 90% confidence
DETECTION_MIN_CONFIDENCE = 0.9
配置值：
Using TensorFlow backend.
Weights: coco
Dataset: ../dataset/carplate
Logs: D:\AiSelect2-CV\Transportation\AiS-CV-Trp-DataSet-Automobile-License-Char-Detect-2020\simple-car-plate-recognition\Mask_RCNN\logs
Configurations:
BACKBONE resnet50
BACKBONE_STRIDES [4, 8, 16, 32, 64]
BATCH_SIZE 1
BBOX_STD_DEV [0.1 0.1 0.2 0.2]
COMPUTE_BACKBONE_SHAPE None
DETECTION_MAX_INSTANCES 100
DETECTION_MIN_CONFIDENCE 0.9
DETECTION_NMS_THRESHOLD 0.3
FPN_CLASSIF_FC_LAYERS_SIZE 1024
GPU_COUNT 1
GRADIENT_CLIP_NORM 5.0
IMAGES_PER_GPU 1
IMAGE_MAX_DIM 256
IMAGE_META_SIZE 14
IMAGE_MIN_DIM 256
IMAGE_MIN_SCALE 0
IMAGE_RESIZE_MODE square
IMAGE_SHAPE [256 256 3]
LEARNING_MOMENTUM 0.9
LEARNING_RATE 0.001
LOSS_WEIGHTS {'rpn_class_loss': 1.0, 'rpn_bbox_loss': 1.0, 'mrcnn_class_loss': 1.0, 'mrcnn_bbox_loss': 1.0, 'mrcnn_mask_loss': 1.0}
MASK_POOL_SIZE 14
MASK_SHAPE [28, 28]
MAX_GT_INSTANCES 100
MEAN_PIXEL [123.7 116.8 103.9]
MINI_MASK_SHAPE (56, 56)
NAME carplate
NUM_CLASSES 2
POOL_SIZE 7
POST_NMS_ROIS_INFERENCE 1000
POST_NMS_ROIS_TRAINING 2000
ROI_POSITIVE_RATIO 0.33
RPN_ANCHOR_RATIOS [0.5, 1, 2]
RPN_ANCHOR_SCALES (32, 64, 128, 256, 512)
RPN_ANCHOR_STRIDE 1
RPN_BBOX_STD_DEV [0.1 0.1 0.2 0.2]
RPN_NMS_THRESHOLD 0.7
RPN_TRAIN_ANCHORS_PER_IMAGE 128
STEPS_PER_EPOCH 100
TOP_DOWN_PYRAMID_SIZE 256
TRAIN_BN False
TRAIN_ROIS_PER_IMAGE 100
USE_MINI_MASK True
USE_RPN_ROIS True
VALIDATION_STEPS 50
WEIGHT_DECAY 0.0001
Mask R-CNN建模
这段代码是一个用于训练或应用Mask R-CNN模型以检测车牌的Python脚本。它使用了argparse库来解析命令行参数，并根据提供的参数执行相应的操作。下面是对代码的详细解释：
- 导入argparse库用于解析命令行参数。
- 创建一个ArgumentParser对象，并添加所需的命令行参数。这些参数包括：
command：要执行的操作，可以是’train’（训练模型）或’splash’（应用颜色闪烁效果）。dataset：车牌数据集的目录路径，仅在训练时需要。weights：预训练权重文件的路径，或指定为’coco’（使用COCO数据集预训练的权重）。logs：日志和检查点文件的目录，默认为’logs/’。image：要应用颜色闪烁效果的图像路径。video：要应用颜色闪烁效果的视频路径。
- 解析命令行参数，并根据参数执行相应的验证。
- 打印出一些参数的值，以便于调试。
- 根据命令行参数创建相应的配置对象。如果是训练操作，则使用
CarplateConfig配置；如果是应用颜色闪烁效果，则使用InferenceConfig配置，它是CarplateConfig的一个子类，用于推理任务。 - 创建Mask R-CNN模型实例。如果是训练操作，则使用训练模式；如果是应用颜色闪烁效果，则使用推理模式。
- 根据提供的权重参数选择要加载的权重文件。可以选择预训练的COCO权重、最后一次训练的权重、ImageNet预训练权重，或指定一个自定义的权重文件路径。
- 加载选定的权重文件。如果是COCO预训练权重，则排除最后的几层，因为它们需要与数据集中的类别数量匹配。
- 根据命令行参数执行相应的操作。如果是训练操作，则调用
train函数训练模型；如果是应用颜色闪烁效果，则调用detect_and_color_splash函数处理图像或视频。 - 如果提供了未知的命令，则打印一条错误消息。
源码：Mask_RCNN\carplate.py
# Root directory of the project
ROOT_DIR = os.path.abspath("./")
# Import Mask RCNN
sys.path.append(ROOT_DIR) # To find local version of the library
from mrcnn.config import Config
from mrcnn import model as modellib, utils
# Path to trained weights file
COCO_WEIGHTS_PATH = os.path.join(ROOT_DIR, "mask_rcnn_coco.h5")
# Directory to save logs and model checkpoints, if not provided
# through the command line argument --logs
DEFAULT_LOGS_DIR = os.path.join(ROOT_DIR, "logs")
if __name__ == '__main__':
import argparse
# Parse command line arguments
parser = argparse.ArgumentParser(
description='Train Mask R-CNN to detect carplates.')
parser.add_argument("command",
metavar="<command>",
help="'train' or 'splash'")
parser.add_argument('--dataset', required=False,
metavar="/path/to/carplate/dataset/",
help='Directory of the carplate dataset')
parser.add_argument('--weights', required=True,
metavar="/path/to/weights.h5",
help="Path to weights .h5 file or 'coco'")
parser.add_argument('--logs', required=False,
default=DEFAULT_LOGS_DIR,
metavar="/path/to/logs/",
help='Logs and checkpoints directory (default=logs/)')
parser.add_argument('--image', required=False,
metavar="path or URL to image",
help='Image to apply the color splash effect on')
parser.add_argument('--video', required=False,
metavar="path or URL to video",
help='Video to apply the color splash effect on')
args = parser.parse_args()
# Validate arguments
if args.command == "train":
assert args.dataset, "Argument --dataset is required for training"
elif args.command == "splash":
assert args.image or args.video,\
"Provide --image or --video to apply color splash"
print("Weights: ", args.weights)
print("Dataset: ", args.dataset)
print("Logs: ", args.logs)
# Configurations
if args.command == "train":
config = CarplateConfig()
else:
class InferenceConfig(CarplateConfig):
# Set batch size to 1 since we'll be running inference on
# one image at a time. Batch size = GPU_COUNT * IMAGES_PER_GPU
GPU_COUNT = 1
IMAGES_PER_GPU = 1
config = InferenceConfig()
config.display()
# Create model
if args.command == "train":
model = modellib.MaskRCNN(mode="training", config=config,
model_dir=args.logs)
else:
model = modellib.MaskRCNN(mode="inference", config=config,
model_dir=args.logs)
# Select weights file to load
if args.weights.lower() == "coco":
weights_path = COCO_WEIGHTS_PATH
# Download weights file
if not os.path.exists(weights_path):
utils.download_trained_weights(weights_path)
elif args.weights.lower() == "last":
# Find last trained weights
weights_path = model.find_last()
elif args.weights.lower() == "imagenet":
# Start from ImageNet trained weights
weights_path = model.get_imagenet_weights()
else:
weights_path = args.weights
# Load weights
print("Loading weights ", weights_path)
if args.weights.lower() == "coco":
# Exclude the last layers because they require a matching
# number of classes
model.load_weights(weights_path, by_name=True, exclude=[
"mrcnn_class_logits", "mrcnn_bbox_fc",
"mrcnn_bbox", "mrcnn_mask"])
else:
model.load_weights(weights_path, by_name=True)
# Train or evaluate
if args.command == "train":
train(model)
elif args.command == "splash":
detect_and_color_splash(model, image_path=args.image,
video_path=args.video)
else:
print("'{}' is not recognized. "
"Use 'train' or 'splash'".format(args.command))
训练函数定义train()：
def train(model):
"""Train the model."""
# Training dataset.
dataset_train = CarplateDataset()
dataset_train.load_carplate(args.dataset, "train")
dataset_train.prepare()
# Validation dataset
dataset_val = CarplateDataset()
dataset_val.load_carplate(args.dataset, "val")
dataset_val.prepare()
# *** This training schedule is an example. Update to your needs ***
# Since we're using a very small dataset, and starting from
# COCO trained weights, we don't need to train too long. Also,
# no need to train all layers, just the heads should do it.
print("Training network heads")
model.train(dataset_train, dataset_val,
learning_rate=config.LEARNING_RATE,
epochs=30,
layers='heads')
执行训练
python carplate.py train –dataset=../dataset/carplate –weights=coco
1/100 [..............................] - ETA: 16:42 - loss: 3.6735 - rpn_class_loss_loss: 0.0563 - rpn_bbox_loss_loss: 2/100 [..............................] - ETA: 9:26 - loss: 3.6583 - rpn_class_loss_loss: 0.0487 - rpn_bbox_loss_loss: 3/100 [..............................] - ETA: 6:53 - loss: 3.1196 - rpn_class_loss_loss: 0.0434 - rpn_bbox_loss_loss: 4/100 [>.............................] - ETA: 5:35 - loss: 2.8602 - rpn_class_loss_loss: 0.0431 - rpn_bbox_loss_loss: 5/100 [>.............................] - ETA: 4:48 - loss: 2.9143 - rpn_class_loss_loss: 0.0477 - rpn_bbox_loss_loss: 6/100 [>.............................] - ETA: 4:16 - loss: 3.0159 - rpn_class_loss_loss: 0.0448 - rpn_bbox_loss_loss: 7/100 [=>............................] - ETA: 3:53 - loss: 2.9172 - rpn_class_loss_loss: 0.0441 - rpn_bbox_loss_loss: 8/100 [=>............................] - ETA: 3:36 - loss: 2.9926 - rpn_class_loss_loss: 0.0442 - rpn_bbox_loss_loss: 9/100 [=>............................] - ETA: 3:24 - loss: 3.1194 - rpn_class_loss_loss: 0.0593 - rpn_bbox_loss_loss: 10/100 [==>...........................] - ETA: 3:13 - loss: 3.0196 - rpn_class_loss_loss: 0.0565 - rpn_bbox_loss_loss: 11/100 [==>...........................] - ETA: 3:04 - loss: 3.3052 - rpn_class_loss_loss: 0.0870 - rpn_bbox_loss_loss: 12/100 [==>...........................] - ETA: 2:56 - loss: 3.2931 - rpn_class_loss_loss: 0.0825 - rpn_bbox_loss_loss: 13/100 [==>...........................] - ETA: 2:48 - loss: 3.1649 - rpn_class_loss_loss: 0.0802 - rpn_bbox_loss_loss: 14/100 [===>..........................] - ETA: 2:42 - loss: 3.0171 - rpn_class_loss_loss: 0.0766 - rpn_bbox_loss_loss: 15/100 [===>..........................] - ETA: 2:37 - loss: 2.9281 - rpn_class_loss_loss: 0.0761 - rpn_bbox_loss_loss: 16/100 [===>..........................] - ETA: 2:32 - loss: 2.9766 - rpn_class_loss_loss: 0.0776 - rpn_bbox_loss_loss: 17/100 [====>.........................] - ETA: 2:27 - loss: 2.9117 - rpn_class_loss_loss: 0.0751 - rpn_bbox_loss_loss: 18/100 [====>.........................] - ETA: 2:24 - loss: 3.0615 - rpn_class_loss_loss: 0.0902 - rpn_bbox_loss_loss: 19/100 [====>.........................] - ETA: 2:19 - loss: 3.0260 - rpn_class_loss_loss: 0.0872 - rpn_bbox_loss_loss: 20/100 [=====>........................] - ETA: 2:16 - loss: 3.0309 - rpn_class_loss_loss: 0.0907 - rpn_bbox_loss_loss: 21/100 [=====>........................] - ETA: 2:13 - loss: 2.9667 - rpn_class_loss_loss: 0.0881 - rpn_bbox_loss_loss: 22/100 [=====>........................] - ETA: 2:09 - loss: 2.8531 - rpn_class_loss_loss: 0.0866 - rpn_bbox_loss_loss: 23/100 [=====>........................] - ETA: 2:06 - loss: 2.8218 - rpn_class_loss_loss: 0.0843 - rpn_bbox_loss_loss: 24/100 [======>.......................] - ETA: 2:03 - loss: 2.8211 - rpn_class_loss_loss: 0.0841 - rpn_bbox_loss_loss: 25/100 [======>.......................] - ETA: 2:01 - loss: 2.7371 - rpn_class_loss_loss: 0.0825 - rpn_bbox_loss_loss: 26/100 [======>.......................] - ETA: 1:58 - loss: 2.7106 - rpn_class_loss_loss: 0.0800 - rpn_bbox_loss_loss: 27/100 [=======>......................] - ETA: 1:55 - loss: 2.6715 - rpn_class_loss_loss: 0.0776 - rpn_bbox_loss_loss: 28/100 [=======>......................] - ETA: 1:53 - loss: 2.6142 - rpn_class_loss_loss: 0.0762 - rpn_bbox_loss_loss: 29/100 [=======>......................] - ETA: 1:51 - loss: 2.6069 - rpn_class_loss_loss: 0.0753 - rpn_bbox_loss_loss: 30/100 [========>.....................] - ETA: 1:49 - loss: 2.6103 - rpn_class_loss_loss: 0.0748 - rpn_bbox_loss_loss: 31/100 [========>.....................] - ETA: 1:47 - loss: 2.6402 - rpn_class_loss_loss: 0.0734 - rpn_bbox_loss_loss: 32/100 [========>.....................] - ETA: 1:45 - loss: 2.6475 - rpn_class_loss_loss: 0.0739 - rpn_bbox_loss_loss: 33/100 [========>.....................] - ETA: 1:43 - loss: 2.6097 - rpn_class_loss_loss: 0.0724 - rpn_bbox_loss_loss: 34/100 [=========>....................] - ETA: 1:41 - loss: 2.5997 - rpn_class_loss_loss: 0.0712 - rpn_bbox_loss_loss: 35/100 [=========>....................] - ETA: 1:39 - loss: 2.6122 - rpn_class_loss_loss: 0.0708 - rpn_bbox_loss_loss: 36/100 [=========>....................] - ETA: 1:37 - loss: 2.5497 - rpn_class_loss_loss: 0.0692 - rpn_bbox_loss_loss: 37/100 [==========>...................] - ETA: 1:35 - loss: 2.5502 - rpn_class_loss_loss: 0.0678 - rpn_bbox_loss_loss: 38/100 [==========>...................] - ETA: 1:33 - loss: 2.5400 - rpn_class_loss_loss: 0.0667 - rpn_bbox_loss_loss:
...
训练完后，在logs文件夹中找到最后一轮的h5模型文件，比如mask_rcnn_carplate_0030.h5。
源码开源协议
The MIT License (MIT)
Copyright (c) 2017 Matterport, Inc.