Edge deployment turns a computer vision model into a product that must work with limited compute, unreliable connectivity, changing light, and real maintenance constraints. For an Indian factory, farm, clinic, or retail site, local inference can reduce latency and cloud bills while keeping sensitive footage on the premises. But success depends on the complete pipeline—not just frames per second reported on a developer laptop.
This guide explains how to deploy computer vision on edge devices, from defining the workload and selecting hardware to optimizing models, building an inference pipeline, and monitoring production performance.
Start with the workload, not the device
Write down the operating requirements before comparing Jetson, Raspberry Pi, Coral, or any other platform. At minimum, specify:
- Task: detection, classification, segmentation, pose estimation, OCR, tracking, or anomaly detection.
- Input: camera resolution, frame rate, number of streams, codec, and lighting conditions.
- Latency target: the maximum acceptable time from capture to action.
- Throughput: required frames per second and whether every frame must be analysed.
- Availability: expected uptime, boot behaviour, and what the system should do offline.
- Power and enclosure: mains power, battery, solar, heat, dust, vibration, and ingress protection.
- Data policy: whether footage may leave the site and how long events should be retained.
A safety camera that needs one decision per second has very different requirements from a robotic arm requiring deterministic responses. Define accuracy using field data as well: precision, recall, false alarms per hour, missed events, and performance by camera or location.
Choose hardware by total deployment cost
The edge is a spectrum. Microcontrollers can handle tiny classification or sensor-fusion models, but are rarely suitable for full video analytics. Raspberry Pi-class single-board computers are useful for low-rate inference, gateways, and prototypes. AI accelerators such as Coral TPUs can deliver efficient INT8 inference when the model uses supported operators. NVIDIA Jetson modules are a strong choice for GPU-accelerated, multi-stream workloads, especially when CUDA and TensorRT support simplify the stack. Industrial PCs with Intel CPUs or integrated accelerators may be preferable where serviceability and long availability matter more than size.
Compare devices using the complete system cost:
- sustained, not peak, inference performance;
- power draw under load;
- camera-input and video-decoding capacity;
- memory headroom for preprocessing and tracking;
- driver and runtime support;
- secure boot, storage, and remote management;
- availability of replacement units in India; and
- enclosure, cooling, installation, and field-service costs.
Do not select hardware from a benchmark that uses a different input size, batch size, or model. Test the exact camera pipeline on the target device.
Pick and export an edge-friendly model
Begin with a compact architecture when possible. MobileNet, EfficientNet-Lite, YOLO nano variants, and task-specific lightweight networks can be better choices than shrinking a large model after training. If your model is still under development, building computer vision models on GitHub can help you compare reproducible training and export workflows.
Export to an intermediate format such as ONNX when it improves portability, then validate outputs against the original framework. Check preprocessing carefully: RGB versus BGR ordering, resize method, normalisation, letterboxing, and output decoding are common sources of silent accuracy loss.
Optimize systematically
Quantization
FP16 often provides a straightforward speed and memory improvement on GPUs. INT8 can reduce memory use and power further, but requires representative calibration data. Include footage from every important camera, time of day, product type, and operating condition. If post-training quantization damages recall, use quantization-aware training rather than immediately changing hardware.
Pruning and distillation
Structured pruning is more likely to produce real speed gains than removing arbitrary individual weights, because supported runtimes can execute the smaller structure efficiently. Knowledge distillation can transfer performance from a large teacher to a deployable student. Measure end-to-end latency after every optimization; a smaller file does not automatically mean faster inference.
For mobile and embedded constraints, use the principles in this AI model optimization guide for mobile devices, while remembering that an edge camera adds decoding, memory movement, and post-processing costs.
Build a production inference pipeline
A reliable pipeline separates capture, decode, preprocessing, inference, post-processing, tracking, and event delivery. Use bounded queues so a slow downstream stage does not consume unlimited memory. Decide whether to drop old frames or process every frame; for many monitoring tasks, analysing the latest frame is better than building a stale queue.
Use the runtime suited to the accelerator:
- TensorRT: optimized NVIDIA GPU inference, with engine builds tied to model and hardware details.
- ONNX Runtime: a useful cross-platform baseline with hardware-specific execution providers.
- OpenVINO: practical for Intel CPUs, integrated GPUs, and supported accelerators.
- TensorFlow Lite or LiteRT: suitable for many mobile, embedded, and TPU workflows.
- Vendor SDKs: often provide efficient camera capture, decoding, tracking, and multi-stream scheduling.
Benchmark cold-start time, warm-up time, p50 and p95 latency, sustained FPS, memory use, dropped frames, and temperature—not only average inference time. Profile the entire pipeline with realistic streams.
Design for offline operation and fleet management
Indian deployments may face intermittent links, power cuts, and difficult physical access. The device should continue making local decisions when disconnected, buffer only necessary events, and synchronise them when connectivity returns. Keep raw video local by default; upload thumbnails, embeddings, metadata, or low-confidence samples only when policy allows.
Use signed versioned releases, staged rollouts, health checks, rollback support, and remote logs. An update system should report model version, runtime version, device temperature, storage, uptime, camera status, and recent inference errors. For larger fleets, container-based management can help, but containers do not replace hardware-aware drivers and resource limits.
Secure the device and protect video
Treat the edge unit as exposed infrastructure. Enable secure boot where available, encrypt local storage, restrict shell access, rotate credentials, and sign both application and model artifacts. Store secrets in a hardware-backed keystore when the platform supports one. Minimise retention and document who can access recordings.
If the system supports clinical, worker-safety, or public-space use, define human review and escalation rules before launch. For healthcare teams, the deployment considerations in integrating computer vision in healthcare apps are especially relevant because accuracy, auditability, and consent matter as much as latency.
Validate in the field
A lab test rarely captures dust on a lens, monsoon glare, nighttime illumination, camera vibration, crowded scenes, or regional product variation. Create a field validation set and report results by site, camera, weather, time of day, and object category. Test degraded connectivity, device restarts, full storage, thermal throttling, corrupted frames, and power recovery.
Monitor drift using sampled frames, confidence distributions, false-alert rates, and operator feedback. Send uncertain or novel cases for review rather than uploading indiscriminate video. Retrain only after confirming that the data represents a real production failure.
A practical deployment checklist
- Define task, latency, throughput, privacy, and power requirements.
- Select hardware using sustained benchmarks and total installed cost.
- Export and numerically validate the model on the target runtime.
- Compare FP16, INT8, pruning, and distillation with field-representative data.
- Profile capture, decoding, preprocessing, inference, and post-processing together.
- Add watchdogs, bounded queues, local buffering, health metrics, and rollback.
- Test heat, dust, lighting, connectivity loss, storage limits, and power recovery.
- Secure boot, credentials, model files, updates, and retained footage.
The best edge deployment is not the one with the highest headline FPS. It is the one that maintains useful accuracy, predictable latency, and operational reliability after months in the field. Start with one representative site, measure the complete system, and expand only after the failure modes are understood.