TinyML is not simply a smaller version of cloud machine learning. It is a product discipline in which the model, sensor pipeline, firmware, power budget, and update process must work together on constrained hardware. For Indian builders, that can mean detecting machine faults in a factory, classifying crops in the field, monitoring cold-chain equipment, or recognising local speech and environmental events without a permanent internet connection.
The most important decision is to define the device constraint before choosing an architecture. A model that is accurate on a laptop but exceeds an MCU’s RAM, flash, latency, or battery budget is not a deployable model.
What “training on edge hardware” really means
Most TinyML projects use a train-on-host, run-on-edge workflow. Training generally happens on a workstation, cloud GPU, or development server because backpropagation needs substantially more memory and compute than a microcontroller provides. The finished model is then converted, compressed, compiled, and tested on the target device.
Some edge computers can perform limited fine-tuning, but this is the exception rather than the default. Distinguish three stages:
- Training: learn parameters from labelled examples.
- Conversion and optimisation: make the model compatible with the target runtime.
- Inference: execute predictions on the device, often offline.
This distinction prevents a common planning error: selecting hardware capable of inference and assuming it can also train the model.
Start with a measurable device specification
Write a deployment specification before collecting data. Record:
- MCU or processor, clock speed, accelerator, and supported operators
- Available flash and peak RAM during inference
- Input type, sampling rate, resolution, and window length
- Maximum acceptable latency and prediction rate
- Battery capacity, sleep schedule, and energy per inference
- Connectivity, security requirements, and update mechanism
- Expected temperature, vibration, lighting, noise, and network conditions
Boards such as ESP32-class devices, Arm Cortex-M microcontrollers, Raspberry Pi-class computers, and specialist sensor platforms have very different capabilities. A small CNN may fit comfortably on an edge computer but fail on a microcontroller because of activation memory. Measure the complete application, including buffers and firmware, not just the model file.
For vision projects, clarify whether the device needs continuous video, periodic snapshots, or event-triggered images. For audio, define the microphone, sample rate, noise profile, and acceptable false-trigger rate. If you are building a computer-vision pipeline, the workflow in How to Build Computer Vision Models on GitHub is useful for organising datasets, experiments, and reproducible code.
Build data that matches the field
Tiny models cannot compensate for a dataset that excludes real operating conditions. Collect data from the actual sensors and hardware where possible. A phone microphone, laboratory camera, and production sensor often produce materially different inputs.
Use a dataset plan that includes:
- Different users, devices, locations, seasons, and operating states
- Positive, negative, and ambiguous examples
- Fault conditions and normal variation
- Class balance and the cost of each error
- Device-specific noise, motion blur, lighting, and calibration drift
Split data by person, site, session, or machine, rather than randomly splitting adjacent samples. Random splits can leak nearly identical windows into training and validation, producing an inflated score. Keep a final test set untouched until model selection is complete.
Preprocessing must be identical during training and inference. Document scaling, windowing, resizing, normalisation, feature extraction, and label timing. For Indian deployments, test regional accents, scripts, weather, power conditions, and connectivity assumptions where relevant. Language projects may also benefit from Low-Resource Language Datasets for AI Training in India, especially when a narrow local vocabulary matters more than a generic benchmark.
Choose the smallest model that meets the target
Begin with a simple baseline: thresholds, linear models, decision trees, or a compact fully connected network. A baseline reveals whether a neural network is necessary and provides a useful comparison for power and latency.
For neural models, prefer architectures with predictable memory use and supported operators. Common choices include:
- Small 1D CNNs for sensor and audio windows
- Compact 2D CNNs for low-resolution image classification
- Depthwise-separable convolutions to reduce multiply-accumulate operations
- Tiny recurrent or temporal-convolution layers for short sequences
- Feature-based classical models when the signal has a strong domain representation
Do not optimise for accuracy alone. Track parameter count, peak RAM, flash size, inference latency, energy per inference, and performance by class. A model with one percentage point less accuracy may be the better product if it doubles battery life or fits on a lower-cost board.
Transfer learning can help with image and audio tasks, but the base model must be compatible with the target budget. Freeze early layers, fine-tune selectively, and validate on field data rather than relying on the source dataset.
Train, quantise, and validate correctly
Train the model in a normal ML environment, then export it through the runtime supported by your device, such as TensorFlow Lite for Microcontrollers or another vendor-compatible inference engine. Confirm that every operation is supported before investing in deployment work.
Quantisation is usually the highest-impact optimisation. Post-training integer quantisation can reduce storage and improve MCU execution, while quantisation-aware training often preserves accuracy better when 8-bit weights and activations materially change predictions. Compare the floating-point and quantised models on the untouched test set.
Also consider:
- Structured pruning when the runtime can exploit the reduced architecture
- Knowledge distillation from a larger teacher model
- Lower input resolution or shorter sensor windows
- Operator fusion and compiler-specific optimisation
- Feature extraction that reduces raw input volume
Inspect errors after every optimisation step. Quantisation may disproportionately harm a minority class or a low-amplitude fault. Report a confusion matrix, precision, recall, F1 score, false-positive rate, and false-negative rate—not only overall accuracy. For safety-critical alerts, define an operating threshold and measure detection delay.
Deploy on the actual board
A successful desktop conversion is not a successful edge deployment. Compile the model into the device firmware or supported runtime, reserve tensor memory, and test the complete sensor-to-prediction path.
Measure:
- Cold-start and steady-state latency
- Peak RAM and flash consumption
- Energy per inference and energy during sleep
- Throughput under realistic sampling conditions
- Thermal behaviour and stability over long runs
- Results with missing, corrupted, or out-of-range sensor data
Keep preprocessing on-device whenever possible so the production input path matches evaluation. Add confidence thresholds, debouncing, temporal voting, or a fallback state to reduce unstable alerts. Log compact diagnostics rather than raw personal data when privacy and bandwidth matter.
For larger edge computers, local deployment patterns can overlap with How to Deploy Large Language Models Locally, but TinyML requires stricter attention to static memory, operator support, and energy. Hardware selection should follow the workload; guides such as Best AI Hardware for Interactive Desk Pets illustrate why sensors, accelerators, power, and physical interaction must be evaluated together.
Plan for monitoring and updates
A model can degrade without its code changing. Sensor ageing, new environments, seasonal conditions, and user behaviour create distribution shift. Define what can be measured safely in the field: confidence histograms, trigger rates, battery impact, reset frequency, and sampled error cases where consent permits.
Use signed firmware and model updates, version every model with its preprocessing configuration, and retain a rollback path. Avoid sending raw audio, images, or health data by default. If retraining is required, aggregate and review data before adding it to the training set.
A practical release checklist is:
- Reproduce the training run and record dataset and code versions.
- Validate the quantised model on a held-out, field-like test set.
- Confirm memory, latency, energy, and thermal budgets on production hardware.
- Test power loss, corrupted input, network absence, and update rollback.
- Run a pilot across representative Indian locations and operating conditions.
- Monitor post-launch errors and define a retraining trigger.
FAQ
Can a microcontroller train a TinyML model?
Usually not. Microcontrollers are normally used for inference, while training happens on a workstation or cloud machine. Some capable edge computers support limited on-device adaptation, but this requires a separate memory, privacy, and safety design.
Is quantisation always necessary?
No, but integer quantisation is often the most practical route to lower memory, latency, and power consumption. Test accuracy and class-level errors after conversion rather than assuming the compressed model behaves like the original.
What is a good first TinyML project?
Choose a narrow classification task with measurable labels, such as keyword detection, vibration anomaly detection, or a simple environmental event. Start with a small dataset collected from the target sensor and set a strict latency and power budget.
How should Indian startups choose hardware?
Choose against the complete bill of materials and operating environment, not a development-board specification. Compare availability, replacement cost, power supply, enclosure, connectivity, security, local support, and long-term procurement alongside model performance.
Apply for AI Grants India
Building an on-device AI product in India? Apply for the AI Grants India programme to explore funding and support for experimentation, pilots, and responsible deployment.