What efficient edge detection actually requires
Efficient real-time object detection on low-power hardware is a systems problem, not simply a matter of selecting the smallest YOLO checkpoint. A production device must capture frames, decode video, resize and normalise images, run inference, apply post-processing, and deliver an action within a predictable latency and power budget.
For an Indian field deployment, those constraints are concrete: a solar-powered camera may operate with intermittent connectivity; a traffic device may face dust, glare, and monsoon rain; an agricultural drone may have limited battery capacity; and a factory gateway may need to run several camera streams without an expensive GPU. The right target is therefore useful detections per watt at a defined latency, rather than a headline FPS number.
Start with a measurable deployment target
Before choosing a model, write down the operating envelope:
- Input: camera resolution, frame rate, field of view, codec, and lighting conditions.
- Detection scope: classes, minimum object size, acceptable false positives, and required recall.
- Latency: end-to-end response time and its 95th or 99th percentile—not only average inference time.
- Throughput: streams per device, whether every frame must be processed, and whether frame skipping is acceptable.
- Power and thermal limits: battery life, sustained wattage, enclosure temperature, and cooling.
- Connectivity and privacy: what must run offline and which events, if any, may be uploaded.
Measure the complete pipeline. A model that takes 18 ms for inference can still produce a 100 ms result if camera buffering, CPU-based preprocessing, memory copies, and post-processing dominate the path.
Choose the smallest model that meets the task
Compact one-stage detectors are usually the best starting point for edge deployments. Nano and small variants of current YOLO-style families, MobileNet-SSD, and hardware-vendor models can provide a useful accuracy-speed trade-off. Do not select solely by COCO benchmark mAP: a model trained on generic images may miss Indian licence plates, two-wheelers, crop pests, safety helmets, or low-light workers.
Build a representative validation set from the actual camera positions. Include small objects, occlusion, motion blur, backlighting, regional clothing, signage, and seasonal changes. If your application concerns infrastructure, a specialist reference such as automated defect detection for railway track safety illustrates why domain-specific data and failure analysis matter more than a generic benchmark.
Useful architecture choices include:
- Depthwise separable convolutions, which reduce multiply-accumulate operations and parameter movement.
- Compact feature pyramids, which preserve small-object performance without simply increasing input resolution.
- Efficient necks and detection heads, reducing feature-map memory traffic.
- Hardware-friendly operators, avoiding layers that fall back to slow CPU execution.
A smaller input image can improve throughput dramatically, but test its effect on the smallest object that matters. In many deployments, a modest resolution increase is more valuable than a larger model.
Optimise precision: FP16, INT8, and QAT
FP16 is often the lowest-risk acceleration step on GPUs and many NPUs. It reduces memory use and can increase throughput with little or no measurable accuracy loss. INT8 can deliver a larger improvement on supported accelerators, but calibration quality is critical.
For post-training quantisation, use a calibration set that reflects production conditions: day and night scenes, different cameras, crowded frames, and difficult backgrounds. Compare class-wise precision and recall before and after conversion. If accuracy drops materially, use quantisation-aware training (QAT) so the model learns to tolerate reduced precision during training.
Validate the converted model on the target runtime. An INT8 model is not automatically faster if unsupported operators are silently executed in FP32 or transferred between the CPU and accelerator. Record model size, accelerator utilisation, memory bandwidth, latency percentiles, and energy per processed frame.
Pruning, distillation, and training improvements
Unstructured pruning can make a model sparse without making it faster on ordinary hardware. Prefer structured channel or block pruning when the deployment runtime can exploit the resulting shape. Fine-tune after pruning and verify that the speed gain survives export.
Knowledge distillation is often more dependable than aggressive pruning. Train a compact student against a stronger teacher, using both ground-truth labels and teacher predictions. This can improve performance on hard examples while keeping the student architecture compatible with TensorRT, TFLite, ONNX Runtime, OpenVINO, or an NPU SDK.
Training improvements also reduce deployment cost. Use targeted augmentation for motion blur, exposure changes, rain, dust, and camera vibration. Avoid augmentations that create unrealistic scenes and increase false positives. Track errors by class, size, lighting, and camera—not only one aggregate mAP value.
Match the model to the hardware stack
The hardware decision should include its compiler and runtime, not just its TOPS rating.
- Raspberry Pi and ARM CPU devices: use TFLite, NCNN, or ONNX Runtime with a compact model; add an accelerator when sustained multi-stream performance is required.
- NVIDIA Jetson: export to TensorRT, enable FP16 or INT8 where validated, and use the DeepStream stack for multi-camera pipelines.
- Intel CPU, iGPU, and VPU deployments: evaluate OpenVINO and confirm that layers are supported by the intended device.
- Mobile and embedded NPUs: use the vendor converter and inspect operator coverage, tensor layouts, and supported quantisation modes.
- Industrial gateways: prioritise long-term software support, secure updates, storage endurance, and thermal design alongside inference speed.
Always benchmark the exact board, camera, driver, compiler version, and enclosure. A desktop result is not evidence of field performance.
Engineer the video pipeline, not just inference
Use hardware video decoding when available and avoid unnecessary frame copies. A zero-copy path between capture buffers and the accelerator can save substantial CPU time and memory bandwidth. Keep preprocessing on the accelerator or in a fused runtime where possible.
Batch size one is normally appropriate for interactive detection. For multiple streams, asynchronous queues can improve utilisation, but uncontrolled buffering increases stale detections. Set a maximum queue depth and measure age-of-frame as well as processing latency. Frame skipping should be policy-driven: track objects between detections when the application permits it, but do not skip frames during safety-critical events.
Post-processing can also become a bottleneck. Use efficient non-maximum suppression, cap candidate detections, and avoid converting large tensors back to the CPU merely to draw overlays. For privacy-sensitive deployments, transmit event metadata or blurred regions instead of raw video wherever possible.
Build a production evaluation plan
A credible edge benchmark should report:
- End-to-end latency at p50, p95, and p99.
- Sustained FPS over at least 30–60 minutes, including thermal behaviour.
- Accuracy by class, object size, lighting, and camera location.
- Power draw, energy per frame, and battery-life estimate where relevant.
- Startup time, crash recovery, offline operation, and storage use.
- Performance after model, driver, and firmware updates.
Run tests with the production camera and enclosure. Monitor throttling, dropped frames, memory leaks, accelerator fallbacks, and clock changes. A field pilot should include remote logs, signed model updates, rollback support, and a clear policy for uncertain detections.
Indian deployment opportunities
Low-power vision is especially useful where connectivity, power, or privacy limits cloud inference. Traffic systems can detect congestion and helmet use locally; farms can identify crop stress or pests from drones; factories can flag safety equipment violations; and public infrastructure teams can monitor bridges and roads. For location-aware operations, edge detections can feed real-time location intelligence platforms in India without continuously uploading video.
The strongest designs keep the raw stream local, send only actionable events, and degrade gracefully when networks fail. That approach lowers bandwidth costs and makes deployments more practical outside major urban data centres.
A practical implementation sequence
1. Collect representative video and define latency, accuracy, and power targets.
2. Train or fine-tune two or three compact detector families.
3. Export each model to the target runtime before extensive optimisation.
4. Benchmark FP32, FP16, and INT8 end to end.
5. Apply distillation or structured pruning only where profiling identifies a real benefit.
6. Optimise capture, decode, memory transfer, preprocessing, and post-processing.
7. Pilot in the production enclosure and monitor thermal and accuracy drift.
8. Add secure updates, observability, and rollback before scaling.
The winning system is rarely the model with the highest benchmark score. It is the one that maintains reliable detections, predictable latency, and acceptable energy use after weeks in the field.