Model Optimization & Tuning

Role in the Project

Ensures models are optimized for speed and accuracy, particularly for deployment on edge devices.

Strengths & Weaknesses

Strengths:

• Model quantization reduces computational requirements.
• Pruning removes unnecessary parameters for efficiency.

Weaknesses:

• Requires extensive hyperparameter tuning.
• Quantization may reduce model accuracy.

Available Technologies & Comparison

• TensorRT (Chosen for NVIDIA optimization) vs. OpenVINO (Optimized for Intel) vs. ONNX Runtime (Broader model compatibility).

Chosen Approach

• Model quantization and pruning using TensorRT.
• Hyperparameter tuning via Optuna.

Example of TensorRT conversion:

import tensorrt as trt
logger = trt.Logger(trt.Logger.WARNING)
builder = trt.Builder(logger)
network = builder.create_network()
parser = trt.OnnxParser(network, logger)