In this tutorial, we will introduce how to quantize a pre-trained MobileNetV2 model using ESP-PPQ and deploy the quantized MobileNetV2 model using ESP-DL.
Preparationï Model quantizationï Pre-trained modelïLoad the pre-trained model of MobileNet_v2 from torchvision. You can also download it from ONNX models or TensorFlow models:
import torchvision from torchvision.models.mobilenetv2 import MobileNet_V2_Weights model = torchvision.models.mobilenet.mobilenet_v2(weights=MobileNet_V2_Weights.IMAGENET1K_V1)Calibration datasetï
The calibration dataset needs to be consistent with your model input format. The calibration dataset needs to cover all possible situations of your model input as much as possible to better quantize the model. Here we take the ImageNet dataset as an example to demonstrate how to prepare the calibration dataset.
Use torchvision to load the ImageNet dataset:
import torchvision.datasets as datasets
from torch.utils.data.dataset import Subset
dataset = datasets.ImageFolder(
CALIB_DIR,
transforms.Compose(
[
transforms.Resize(256),
transforms.CenterCrop(224),
transforms.ToTensor(),
transforms.Normalize(
mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225]
),
]
),
)
dataset = Subset(dataset, indices=[_ for _ in range(0, 1024)])
dataloader = DataLoader(
dataset=dataset,
batch_size=BATCH_SIZE,
shuffle=False,
num_workers=4,
pin_memory=False,
collate_fn=collate_fn1,
)
8bit default configuration quantizationï
Quantization settings
target="esp32p4" num_of_bits=8 batch_size=32 quant_setting = QuantizationSettingFactory.espdl_setting() # default setting
Quantization results
Analysing Graphwise Quantization Error:: Layer | NOISE:SIGNAL POWER RATIO /features/features.16/conv/conv.2/Conv: | ââââââââââââââââââââ | 48.831% /features/features.15/conv/conv.2/Conv: | âââââââââââââââââââ | 45.268% /features/features.17/conv/conv.2/Conv: | ââââââââââââââââââ | 43.112% /features/features.18/features.18.0/Conv: | âââââââââââââââââ | 41.586% /features/features.14/conv/conv.2/Conv: | âââââââââââââââââ | 41.135% /features/features.13/conv/conv.2/Conv: | ââââââââââââââ | 35.090% /features/features.17/conv/conv.0/conv.0.0/Conv: | âââââââââââââ | 32.895% /features/features.16/conv/conv.1/conv.1.0/Conv: | ââââââââââââ | 29.226% /features/features.12/conv/conv.2/Conv: | ââââââââââââ | 28.895% /features/features.16/conv/conv.0/conv.0.0/Conv: | âââââââââââ | 27.808% /features/features.7/conv/conv.2/Conv: | âââââââââââ | 27.675% /features/features.10/conv/conv.2/Conv: | âââââââââââ | 26.292% /features/features.11/conv/conv.2/Conv: | âââââââââââ | 26.085% /features/features.6/conv/conv.2/Conv: | âââââââââââ | 25.892% /classifier/classifier.1/Gemm: | ââââââââââ | 25.591% /features/features.15/conv/conv.0/conv.0.0/Conv: | ââââââââââ | 25.323% /features/features.4/conv/conv.2/Conv: | ââââââââââ | 24.787% /features/features.15/conv/conv.1/conv.1.0/Conv: | ââââââââââ | 24.354% /features/features.14/conv/conv.1/conv.1.0/Conv: | ââââââââ | 20.207% /features/features.9/conv/conv.2/Conv: | ââââââââ | 19.808% /features/features.14/conv/conv.0/conv.0.0/Conv: | ââââââââ | 18.465% /features/features.5/conv/conv.2/Conv: | âââââââ | 17.868% /features/features.12/conv/conv.1/conv.1.0/Conv: | âââââââ | 16.589% /features/features.13/conv/conv.1/conv.1.0/Conv: | âââââââ | 16.143% /features/features.11/conv/conv.1/conv.1.0/Conv: | ââââââ | 15.382% /features/features.3/conv/conv.2/Conv: | ââââââ | 15.105% /features/features.13/conv/conv.0/conv.0.0/Conv: | ââââââ | 15.029% /features/features.10/conv/conv.1/conv.1.0/Conv: | ââââââ | 14.875% /features/features.2/conv/conv.2/Conv: | ââââââ | 14.869% /features/features.11/conv/conv.0/conv.0.0/Conv: | ââââââ | 14.552% /features/features.9/conv/conv.1/conv.1.0/Conv: | ââââââ | 14.050% /features/features.8/conv/conv.1/conv.1.0/Conv: | ââââââ | 13.929% /features/features.8/conv/conv.2/Conv: | ââââââ | 13.833% /features/features.12/conv/conv.0/conv.0.0/Conv: | ââââââ | 13.684% /features/features.7/conv/conv.0/conv.0.0/Conv: | âââââ | 12.942% /features/features.6/conv/conv.1/conv.1.0/Conv: | âââââ | 12.765% /features/features.10/conv/conv.0/conv.0.0/Conv: | âââââ | 12.251% /features/features.5/conv/conv.1/conv.1.0/Conv: | âââââ | 11.186% /features/features.17/conv/conv.1/conv.1.0/Conv: | ââââ | 11.070% /features/features.9/conv/conv.0/conv.0.0/Conv: | ââââ | 10.371% /features/features.4/conv/conv.1/conv.1.0/Conv: | ââââ | 10.356% /features/features.6/conv/conv.0/conv.0.0/Conv: | ââââ | 10.149% /features/features.4/conv/conv.0/conv.0.0/Conv: | ââââ | 9.472% /features/features.8/conv/conv.0/conv.0.0/Conv: | ââââ | 9.232% /features/features.3/conv/conv.1/conv.1.0/Conv: | ââââ | 9.187% /features/features.1/conv/conv.1/Conv: | ââââ | 8.770% /features/features.5/conv/conv.0/conv.0.0/Conv: | âââ | 8.408% /features/features.7/conv/conv.1/conv.1.0/Conv: | âââ | 8.151% /features/features.2/conv/conv.1/conv.1.0/Conv: | âââ | 7.156% /features/features.3/conv/conv.0/conv.0.0/Conv: | âââ | 6.328% /features/features.2/conv/conv.0/conv.0.0/Conv: | ââ | 5.392% /features/features.1/conv/conv.0/conv.0.0/Conv: | | 0.875% /features/features.0/features.0.0/Conv: | | 0.119% Analysing Layerwise quantization error:: 100%|âââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââ| 53/53 [08:44<00:00, 9.91s/it] Layer | NOISE:SIGNAL POWER RATIO /features/features.1/conv/conv.0/conv.0.0/Conv: | ââââââââââââââââââââ | 14.303% /features/features.0/features.0.0/Conv: | â | 0.844% /features/features.1/conv/conv.1/Conv: | â | 0.667% /features/features.2/conv/conv.1/conv.1.0/Conv: | â | 0.574% /features/features.3/conv/conv.1/conv.1.0/Conv: | â | 0.419% /features/features.15/conv/conv.1/conv.1.0/Conv: | | 0.272% /features/features.9/conv/conv.1/conv.1.0/Conv: | | 0.238% /features/features.17/conv/conv.1/conv.1.0/Conv: | | 0.214% /features/features.4/conv/conv.1/conv.1.0/Conv: | | 0.180% /features/features.11/conv/conv.1/conv.1.0/Conv: | | 0.151% /features/features.12/conv/conv.1/conv.1.0/Conv: | | 0.148% /features/features.16/conv/conv.1/conv.1.0/Conv: | | 0.146% /features/features.14/conv/conv.2/Conv: | | 0.136% /features/features.13/conv/conv.1/conv.1.0/Conv: | | 0.105% /features/features.6/conv/conv.1/conv.1.0/Conv: | | 0.105% /features/features.8/conv/conv.1/conv.1.0/Conv: | | 0.083% /features/features.7/conv/conv.2/Conv: | | 0.076% /features/features.5/conv/conv.1/conv.1.0/Conv: | | 0.076% /features/features.3/conv/conv.2/Conv: | | 0.075% /features/features.16/conv/conv.2/Conv: | | 0.074% /features/features.13/conv/conv.0/conv.0.0/Conv: | | 0.072% /features/features.15/conv/conv.2/Conv: | | 0.066% /features/features.4/conv/conv.2/Conv: | | 0.065% /features/features.11/conv/conv.2/Conv: | | 0.063% /classifier/classifier.1/Gemm: | | 0.063% /features/features.2/conv/conv.0/conv.0.0/Conv: | | 0.054% /features/features.13/conv/conv.2/Conv: | | 0.050% /features/features.10/conv/conv.1/conv.1.0/Conv: | | 0.042% /features/features.17/conv/conv.0/conv.0.0/Conv: | | 0.040% /features/features.2/conv/conv.2/Conv: | | 0.038% /features/features.4/conv/conv.0/conv.0.0/Conv: | | 0.034% /features/features.17/conv/conv.2/Conv: | | 0.030% /features/features.14/conv/conv.0/conv.0.0/Conv: | | 0.025% /features/features.16/conv/conv.0/conv.0.0/Conv: | | 0.024% /features/features.10/conv/conv.2/Conv: | | 0.022% /features/features.11/conv/conv.0/conv.0.0/Conv: | | 0.021% /features/features.9/conv/conv.2/Conv: | | 0.021% /features/features.14/conv/conv.1/conv.1.0/Conv: | | 0.020% /features/features.7/conv/conv.1/conv.1.0/Conv: | | 0.020% /features/features.5/conv/conv.2/Conv: | | 0.019% /features/features.8/conv/conv.2/Conv: | | 0.018% /features/features.12/conv/conv.2/Conv: | | 0.017% /features/features.6/conv/conv.2/Conv: | | 0.014% /features/features.7/conv/conv.0/conv.0.0/Conv: | | 0.014% /features/features.3/conv/conv.0/conv.0.0/Conv: | | 0.013% /features/features.12/conv/conv.0/conv.0.0/Conv: | | 0.009% /features/features.15/conv/conv.0/conv.0.0/Conv: | | 0.008% /features/features.5/conv/conv.0/conv.0.0/Conv: | | 0.006% /features/features.6/conv/conv.0/conv.0.0/Conv: | | 0.005% /features/features.9/conv/conv.0/conv.0.0/Conv: | | 0.003% /features/features.18/features.18.0/Conv: | | 0.002% /features/features.10/conv/conv.0/conv.0.0/Conv: | | 0.002% /features/features.8/conv/conv.0/conv.0.0/Conv: | | 0.002% * Prec@1 60.500 Prec@5 83.275*
Quantization error analysis
The top1 accuracy after quantization is only 60.5%, which is far from the accuracy of the float model (71.878%). The quantization model has a large loss in accuracy, including:
Graphwise Error
The last layer of the model is /classifier/classifier.1/Gemm, and the cumulative error of this layer is 25.591%. In experience, the cumulative error of the last layer is less than 10%, and the accuracy loss of the quantization model is small.
Layerwise error
Observing the Layerwise error, it is found that the errors of most layers are less than 1%, indicating that the quantization errors of most layers are small, and only a few layers have large errors. We can choose to quantize the layers with large errors using int16. For details, please see mixed precision quantization.
esp-dl supports mixed precision quantization, which can be used to quantize some layers using int16 and some layers using int8. The quantization error of the model can be reduced by using mixed precision quantization.
Quantization settings
from esp_ppq.api import get_target_platform
target="esp32p4"
num_of_bits=8
batch_size=32
# The following layers are quantized using int16
quant_setting = QuantizationSettingFactory.espdl_setting()
quant_setting.dispatching_table.append("/features/features.1/conv/conv.0/conv.0.0/Conv", get_target_platform(TARGET, 16))
quant_setting.dispatching_table.append("/features/features.1/conv/conv.0/conv.0.2/Clip", get_target_platform(TARGET, 16))
Quantization results
Layer | NOISE:SIGNAL POWER RATIO /features/features.16/conv/conv.2/Conv: | ââââââââââââââââââââ | 31.585% /features/features.15/conv/conv.2/Conv: | âââââââââââââââââââ | 29.253% /features/features.17/conv/conv.0/conv.0.0/Conv: | ââââââââââââââââ | 25.077% /features/features.14/conv/conv.2/Conv: | ââââââââââââââââ | 24.819% /features/features.17/conv/conv.2/Conv: | ââââââââââââ | 19.546% /features/features.13/conv/conv.2/Conv: | ââââââââââââ | 19.283% /features/features.16/conv/conv.0/conv.0.0/Conv: | ââââââââââââ | 18.764% /features/features.16/conv/conv.1/conv.1.0/Conv: | ââââââââââââ | 18.596% /features/features.18/features.18.0/Conv: | ââââââââââââ | 18.541% /features/features.15/conv/conv.0/conv.0.0/Conv: | ââââââââââ | 15.633% /features/features.12/conv/conv.2/Conv: | âââââââââ | 14.784% /features/features.15/conv/conv.1/conv.1.0/Conv: | âââââââââ | 14.773% /features/features.14/conv/conv.1/conv.1.0/Conv: | âââââââââ | 13.700% /features/features.6/conv/conv.2/Conv: | ââââââââ | 12.824% /features/features.10/conv/conv.2/Conv: | âââââââ | 11.727% /features/features.14/conv/conv.0/conv.0.0/Conv: | âââââââ | 10.612% /features/features.11/conv/conv.2/Conv: | ââââââ | 10.262% /features/features.9/conv/conv.2/Conv: | ââââââ | 9.967% /classifier/classifier.1/Gemm: | ââââââ | 9.117% /features/features.5/conv/conv.2/Conv: | ââââââ | 8.915% /features/features.7/conv/conv.2/Conv: | âââââ | 8.690% /features/features.3/conv/conv.2/Conv: | âââââ | 8.586% /features/features.4/conv/conv.2/Conv: | âââââ | 7.525% /features/features.13/conv/conv.1/conv.1.0/Conv: | âââââ | 7.432% /features/features.12/conv/conv.1/conv.1.0/Conv: | âââââ | 7.317% /features/features.13/conv/conv.0/conv.0.0/Conv: | ââââ | 6.848% /features/features.8/conv/conv.2/Conv: | ââââ | 6.711% /features/features.10/conv/conv.1/conv.1.0/Conv: | ââââ | 6.100% /features/features.8/conv/conv.1/conv.1.0/Conv: | ââââ | 6.043% /features/features.11/conv/conv.1/conv.1.0/Conv: | ââââ | 5.962% /features/features.9/conv/conv.1/conv.1.0/Conv: | ââââ | 5.873% /features/features.12/conv/conv.0/conv.0.0/Conv: | ââââ | 5.833% /features/features.7/conv/conv.0/conv.0.0/Conv: | ââââ | 5.832% /features/features.11/conv/conv.0/conv.0.0/Conv: | ââââ | 5.736% /features/features.6/conv/conv.1/conv.1.0/Conv: | ââââ | 5.639% /features/features.5/conv/conv.1/conv.1.0/Conv: | âââ | 5.017% /features/features.10/conv/conv.0/conv.0.0/Conv: | âââ | 4.963% /features/features.17/conv/conv.1/conv.1.0/Conv: | âââ | 4.870% /features/features.3/conv/conv.1/conv.1.0/Conv: | âââ | 4.655% /features/features.2/conv/conv.2/Conv: | âââ | 4.650% /features/features.4/conv/conv.0/conv.0.0/Conv: | âââ | 4.648% /features/features.1/conv/conv.1/Conv: | âââ | 4.318% /features/features.9/conv/conv.0/conv.0.0/Conv: | ââ | 3.849% /features/features.6/conv/conv.0/conv.0.0/Conv: | ââ | 3.712% /features/features.4/conv/conv.1/conv.1.0/Conv: | ââ | 3.394% /features/features.8/conv/conv.0/conv.0.0/Conv: | ââ | 3.391% /features/features.7/conv/conv.1/conv.1.0/Conv: | ââ | 2.713% /features/features.2/conv/conv.1/conv.1.0/Conv: | ââ | 2.637% /features/features.2/conv/conv.0/conv.0.0/Conv: | ââ | 2.602% /features/features.5/conv/conv.0/conv.0.0/Conv: | â | 2.397% /features/features.3/conv/conv.0/conv.0.0/Conv: | â | 1.759% /features/features.1/conv/conv.0/conv.0.0/Conv: | | 0.433% /features/features.0/features.0.0/Conv: | | 0.119% Analysing Layerwise quantization error:: 100%|âââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââ| 53/53 [08:27<00:00, 9.58s/it] * Layer | NOISE:SIGNAL POWER RATIO /features/features.1/conv/conv.1/Conv: | ââââââââââââââââââââ | 1.096% /features/features.0/features.0.0/Conv: | âââââââââââââââ | 0.844% /features/features.2/conv/conv.1/conv.1.0/Conv: | ââââââââââ | 0.574% /features/features.3/conv/conv.1/conv.1.0/Conv: | ââââââââ | 0.425% /features/features.15/conv/conv.1/conv.1.0/Conv: | âââââ | 0.272% /features/features.9/conv/conv.1/conv.1.0/Conv: | ââââ | 0.238% /features/features.17/conv/conv.1/conv.1.0/Conv: | ââââ | 0.214% /features/features.4/conv/conv.1/conv.1.0/Conv: | âââ | 0.180% /features/features.11/conv/conv.1/conv.1.0/Conv: | âââ | 0.151% /features/features.12/conv/conv.1/conv.1.0/Conv: | âââ | 0.148% /features/features.16/conv/conv.1/conv.1.0/Conv: | âââ | 0.146% /features/features.14/conv/conv.2/Conv: | ââ | 0.136% /features/features.13/conv/conv.1/conv.1.0/Conv: | ââ | 0.105% /features/features.6/conv/conv.1/conv.1.0/Conv: | ââ | 0.105% /features/features.8/conv/conv.1/conv.1.0/Conv: | â | 0.083% /features/features.5/conv/conv.1/conv.1.0/Conv: | â | 0.076% /features/features.3/conv/conv.2/Conv: | â | 0.075% /features/features.16/conv/conv.2/Conv: | â | 0.074% /features/features.13/conv/conv.0/conv.0.0/Conv: | â | 0.072% /features/features.7/conv/conv.2/Conv: | â | 0.071% /features/features.15/conv/conv.2/Conv: | â | 0.066% /features/features.4/conv/conv.2/Conv: | â | 0.065% /features/features.11/conv/conv.2/Conv: | â | 0.063% /classifier/classifier.1/Gemm: | â | 0.063% /features/features.13/conv/conv.2/Conv: | â | 0.059% /features/features.2/conv/conv.0/conv.0.0/Conv: | â | 0.054% /features/features.10/conv/conv.1/conv.1.0/Conv: | â | 0.042% /features/features.17/conv/conv.0/conv.0.0/Conv: | â | 0.040% /features/features.2/conv/conv.2/Conv: | â | 0.038% /features/features.4/conv/conv.0/conv.0.0/Conv: | â | 0.034% /features/features.17/conv/conv.2/Conv: | â | 0.030% /features/features.14/conv/conv.0/conv.0.0/Conv: | | 0.025% /features/features.16/conv/conv.0/conv.0.0/Conv: | | 0.024% /features/features.10/conv/conv.2/Conv: | | 0.022% /features/features.11/conv/conv.0/conv.0.0/Conv: | | 0.021% /features/features.9/conv/conv.2/Conv: | | 0.021% /features/features.14/conv/conv.1/conv.1.0/Conv: | | 0.020% /features/features.7/conv/conv.1/conv.1.0/Conv: | | 0.020% /features/features.5/conv/conv.2/Conv: | | 0.019% /features/features.8/conv/conv.2/Conv: | | 0.018% /features/features.12/conv/conv.2/Conv: | | 0.017% /features/features.1/conv/conv.0/conv.0.0/Conv: | | 0.017% /features/features.6/conv/conv.2/Conv: | | 0.014% /features/features.7/conv/conv.0/conv.0.0/Conv: | | 0.014% /features/features.3/conv/conv.0/conv.0.0/Conv: | | 0.013% /features/features.12/conv/conv.0/conv.0.0/Conv: | | 0.009% /features/features.15/conv/conv.0/conv.0.0/Conv: | | 0.008% /features/features.5/conv/conv.0/conv.0.0/Conv: | | 0.006% /features/features.6/conv/conv.0/conv.0.0/Conv: | | 0.005% /features/features.9/conv/conv.0/conv.0.0/Conv: | | 0.003% /features/features.18/features.18.0/Conv: | | 0.002% /features/features.10/conv/conv.0/conv.0.0/Conv: | | 0.002% /features/features.8/conv/conv.0/conv.0.0/Conv: | | 0.002% * Prec@1 69.550 Prec@5 88.450*
Quantization Error Analysis
After replacing the layer with the largest error with 16-bit quantization, it can be observed that the model accuracy is significantly improved. The top1 accuracy after quantization is 69.550%, which is close to the accuracy of the float model (71.878%). The cumulative error of the last layer of the model /classifier/classifier.1/Gemm is 9.117%.
This method is proposed in the paper Data-Free Quantization Through Weight Equalization and Bias Correction. When using this method, the original ReLU6 in the MobilenetV2 model needs to be replaced with ReLU.
Quantization Settings
import torch.nn as nn
def convert_relu6_to_relu(model):
for child_name, child in model.named_children():
if isinstance(child, nn.ReLU6):
setattr(model, child_name, nn.ReLU())
else:
convert_relu6_to_relu(child)
return model
# Replace ReLU6 with ReLU
model = convert_relu6_to_relu(model)
# Use layerwise equalization
quant_setting = QuantizationSettingFactory.espdl_setting()
quant_setting.equalization = True
quant_setting.equalization_setting.iterations = 4
quant_setting.equalization_setting.value_threshold = .4
quant_setting.equalization_setting.opt_level = 2
quant_setting.equalization_setting.interested_layers = None
Layer | NOISE:SIGNAL POWER RATIO /features/features.16/conv/conv.2/Conv: | ââââââââââââââââââââ | 34.497% /features/features.15/conv/conv.2/Conv: | ââââââââââââââââââ | 30.813% /features/features.14/conv/conv.2/Conv: | âââââââââââââââ | 25.876% /features/features.17/conv/conv.0/conv.0.0/Conv: | ââââââââââââââ | 24.498% /features/features.17/conv/conv.2/Conv: | ââââââââââââ | 20.290% /features/features.13/conv/conv.2/Conv: | ââââââââââââ | 20.177% /features/features.16/conv/conv.0/conv.0.0/Conv: | ââââââââââââ | 19.993% /features/features.18/features.18.0/Conv: | âââââââââââ | 19.536% /features/features.16/conv/conv.1/conv.1.0/Conv: | ââââââââââ | 17.879% /features/features.12/conv/conv.2/Conv: | ââââââââââ | 17.150% /features/features.15/conv/conv.0/conv.0.0/Conv: | âââââââââ | 15.970% /features/features.15/conv/conv.1/conv.1.0/Conv: | âââââââââ | 15.254% /features/features.1/conv/conv.1/Conv: | âââââââââ | 15.122% /features/features.10/conv/conv.2/Conv: | âââââââââ | 14.917% /features/features.6/conv/conv.2/Conv: | ââââââââ | 13.446% /features/features.11/conv/conv.2/Conv: | âââââââ | 12.533% /features/features.9/conv/conv.2/Conv: | âââââââ | 11.479% /features/features.14/conv/conv.1/conv.1.0/Conv: | âââââââ | 11.470% /features/features.5/conv/conv.2/Conv: | ââââââ | 10.669% /features/features.3/conv/conv.2/Conv: | ââââââ | 10.526% /features/features.14/conv/conv.0/conv.0.0/Conv: | ââââââ | 9.529% /features/features.7/conv/conv.2/Conv: | âââââ | 9.500% /classifier/classifier.1/Gemm: | âââââ | 8.965% /features/features.4/conv/conv.2/Conv: | âââââ | 8.674% /features/features.12/conv/conv.1/conv.1.0/Conv: | âââââ | 8.349% /features/features.13/conv/conv.1/conv.1.0/Conv: | âââââ | 8.068% /features/features.8/conv/conv.2/Conv: | âââââ | 7.961% /features/features.13/conv/conv.0/conv.0.0/Conv: | ââââ | 7.451% /features/features.10/conv/conv.1/conv.1.0/Conv: | ââââ | 6.714% /features/features.9/conv/conv.1/conv.1.0/Conv: | ââââ | 6.399% /features/features.8/conv/conv.1/conv.1.0/Conv: | ââââ | 6.369% /features/features.11/conv/conv.1/conv.1.0/Conv: | ââââ | 6.222% /features/features.2/conv/conv.2/Conv: | âââ | 5.867% /features/features.5/conv/conv.1/conv.1.0/Conv: | âââ | 5.719% /features/features.12/conv/conv.0/conv.0.0/Conv: | âââ | 5.546% /features/features.6/conv/conv.1/conv.1.0/Conv: | âââ | 5.414% /features/features.10/conv/conv.0/conv.0.0/Conv: | âââ | 5.093% /features/features.17/conv/conv.1/conv.1.0/Conv: | âââ | 4.951% /features/features.11/conv/conv.0/conv.0.0/Conv: | âââ | 4.941% /features/features.2/conv/conv.1/conv.1.0/Conv: | âââ | 4.825% /features/features.7/conv/conv.0/conv.0.0/Conv: | ââ | 4.330% /features/features.2/conv/conv.0/conv.0.0/Conv: | ââ | 4.299% /features/features.3/conv/conv.1/conv.1.0/Conv: | ââ | 4.283% /features/features.4/conv/conv.0/conv.0.0/Conv: | ââ | 3.477% /features/features.4/conv/conv.1/conv.1.0/Conv: | ââ | 3.287% /features/features.8/conv/conv.0/conv.0.0/Conv: | ââ | 2.787% /features/features.9/conv/conv.0/conv.0.0/Conv: | ââ | 2.774% /features/features.6/conv/conv.0/conv.0.0/Conv: | ââ | 2.705% /features/features.7/conv/conv.1/conv.1.0/Conv: | ââ | 2.636% /features/features.5/conv/conv.0/conv.0.0/Conv: | â | 1.846% /features/features.3/conv/conv.0/conv.0.0/Conv: | â | 1.170% /features/features.1/conv/conv.0/conv.0.0/Conv: | | 0.389% /features/features.0/features.0.0/Conv: | | 0.025% Analysing Layerwise quantization error:: 100%|ââââââââââ| 53/53 [07:46<00:00, 8.80s/it] Layer | NOISE:SIGNAL POWER RATIO /features/features.1/conv/conv.0/conv.0.0/Conv: | ââââââââââââââââââââ | 0.989% /features/features.0/features.0.0/Conv: | âââââââââââââââââ | 0.845% /features/features.16/conv/conv.2/Conv: | âââââ | 0.238% /features/features.17/conv/conv.2/Conv: | ââââ | 0.202% /features/features.14/conv/conv.2/Conv: | ââââ | 0.198% /features/features.1/conv/conv.1/Conv: | ââââ | 0.192% /features/features.15/conv/conv.2/Conv: | âââ | 0.145% /features/features.4/conv/conv.2/Conv: | ââ | 0.120% /features/features.2/conv/conv.2/Conv: | ââ | 0.111% /features/features.2/conv/conv.1/conv.1.0/Conv: | ââ | 0.079% /classifier/classifier.1/Gemm: | â | 0.062% /features/features.13/conv/conv.2/Conv: | â | 0.050% /features/features.3/conv/conv.2/Conv: | â | 0.050% /features/features.12/conv/conv.2/Conv: | â | 0.050% /features/features.5/conv/conv.1/conv.1.0/Conv: | â | 0.047% /features/features.3/conv/conv.1/conv.1.0/Conv: | â | 0.046% /features/features.7/conv/conv.2/Conv: | â | 0.045% /features/features.5/conv/conv.2/Conv: | â | 0.030% /features/features.11/conv/conv.2/Conv: | â | 0.028% /features/features.6/conv/conv.2/Conv: | â | 0.027% /features/features.6/conv/conv.1/conv.1.0/Conv: | â | 0.026% /features/features.4/conv/conv.0/conv.0.0/Conv: | | 0.025% /features/features.15/conv/conv.1/conv.1.0/Conv: | | 0.023% /features/features.8/conv/conv.1/conv.1.0/Conv: | | 0.021% /features/features.10/conv/conv.2/Conv: | | 0.020% /features/features.11/conv/conv.1/conv.1.0/Conv: | | 0.020% /features/features.16/conv/conv.1/conv.1.0/Conv: | | 0.017% /features/features.14/conv/conv.0/conv.0.0/Conv: | | 0.016% /features/features.4/conv/conv.1/conv.1.0/Conv: | | 0.012% /features/features.13/conv/conv.1/conv.1.0/Conv: | | 0.012% /features/features.13/conv/conv.0/conv.0.0/Conv: | | 0.012% /features/features.12/conv/conv.1/conv.1.0/Conv: | | 0.012% /features/features.17/conv/conv.0/conv.0.0/Conv: | | 0.011% /features/features.12/conv/conv.0/conv.0.0/Conv: | | 0.011% /features/features.2/conv/conv.0/conv.0.0/Conv: | | 0.010% /features/features.9/conv/conv.2/Conv: | | 0.008% /features/features.8/conv/conv.2/Conv: | | 0.008% /features/features.10/conv/conv.1/conv.1.0/Conv: | | 0.008% /features/features.16/conv/conv.0/conv.0.0/Conv: | | 0.008% /features/features.7/conv/conv.0/conv.0.0/Conv: | | 0.008% /features/features.10/conv/conv.0/conv.0.0/Conv: | | 0.006% /features/features.15/conv/conv.0/conv.0.0/Conv: | | 0.005% /features/features.3/conv/conv.0/conv.0.0/Conv: | | 0.004% /features/features.11/conv/conv.0/conv.0.0/Conv: | | 0.004% /features/features.18/features.18.0/Conv: | | 0.003% /features/features.5/conv/conv.0/conv.0.0/Conv: | | 0.003% /features/features.9/conv/conv.1/conv.1.0/Conv: | | 0.003% /features/features.6/conv/conv.0/conv.0.0/Conv: | | 0.003% /features/features.7/conv/conv.1/conv.1.0/Conv: | | 0.003% /features/features.17/conv/conv.1/conv.1.0/Conv: | | 0.002% /features/features.14/conv/conv.1/conv.1.0/Conv: | | 0.002% /features/features.8/conv/conv.0/conv.0.0/Conv: | | 0.001% /features/features.9/conv/conv.0/conv.0.0/Conv: | | 0.001% * Prec@1 69.800 Prec@5 88.550
Quantization Error Analysis
Note that applying layerwise equalization to 8-bit quantization helps reduce quantization loss. The cumulative error of the last layer of the model /classifier/classifier.1/Gemm is 8.965%. The top1 accuracy after quantization is 69.800%, which is closer to the accuracy of the float model (71.878%) and higher than the quantization accuracy of mixed precision quantization.
Note
To further reduce the quantization error, you can try using QAT (Auantization Aware Training). For specific methods, please refer to PPQ QAT example.
Model deploymentï Image classification base classï Pre-processïImagePreprocessor class contains the common pre-precoess pipeline, color conversion, crop, resize, normalization, quantizeã
RetroSearch is an open source project built by @garambo | Open a GitHub Issue
Search and Browse the WWW like it's 1997 | Search results from DuckDuckGo
HTML:
3.2
| Encoding:
UTF-8
| Version:
0.7.4