Apply for AI Grants India

Mandi prices rarely move because of one variable. Harvest arrivals, rainfall, irrigation, storage, transport costs, export rules, procurement, festivals, and local demand can all affect the price reported for a commodity in a particular market. A useful analysis therefore needs more than a chart of monthly averages.

Karpathy’s Autoresearch approach is best treated as an automated experiment loop: define a measurable objective, let an agent modify or test an analysis, evaluate it against a fixed metric, and retain only improvements. It is not a magic mandi-data platform, and it does not remove the need for careful data sourcing or domain judgment. Used properly, it can help a researcher compare seasonal features, forecasting methods, and validation strategies without manually rebuilding every experiment.

This guide explains how to use Karpathy Autoresearch to analyze mandi price fluctuations across seasonal cycles in India, with an emphasis on reproducibility and decisions that farmers, aggregators, traders, and researchers can actually use.

Define the question before opening the tools

Start with one decision and one forecast horizon. For example:

• Estimate next week’s modal price for onion in Lasalgaon.
• Measure whether prices typically rise between harvest arrival and the lean season.
• Compare seasonal volatility across three mandis for the same commodity.
• Test whether rainfall, arrivals, or storage indicators improve a baseline forecast.

Avoid combining all of these into one experiment. A narrow objective produces a clearer dataset, metric, and output. If the project involves building a lightweight local inference system, the workflow can later connect with quantized models for mandi price queries.

Build a trustworthy mandi dataset

Choose consistent price fields

Mandi records may include minimum, maximum, and modal prices, along with arrivals and a reporting date. Select one target field and document its unit. The modal price is often a practical starting point, but it should not be treated as interchangeable with a farmer’s realised net price after commission, transport, grading, and wastage.

Keep these columns where available:

• Commodity and variety
• State, district, market, and market identifier
• Date of quotation
• Minimum, maximum, and modal price
• Arrivals or traded quantity
• Unit and quality grade
• Weather observations or rainfall totals
• Distance, freight, storage, or procurement indicators

Use official or clearly documented sources such as agricultural marketing records, state portals, weather datasets, and procurement announcements. Preserve the raw files separately from cleaned data. Record download dates, source URLs, schema changes, and any transformations.

Clean without erasing market behaviour

Do not automatically replace every extreme value. A sharp price increase may be a genuine supply shock. Instead:

• Standardise dates and convert all prices to a common unit.
• Map commodity and market names to stable identifiers.
• Remove duplicate records using a documented rule.
• Flag missing observations rather than silently filling them.
• Separate zero arrivals from missing arrivals.
• Investigate sudden price changes against news, rainfall, closures, or arrival data.
• Mark periods affected by lockdowns, strikes, floods, or policy changes.

For missing prices, compare forward fill, interpolation, and model-based imputation in separate experiments. Never let future observations leak into a historical training window.

Represent seasonal cycles correctly

A calendar month is an imperfect proxy for agricultural seasonality. Create features that reflect the crop and region:

• Month, week of year, and day of year
• Kharif, Rabi, or Zaid season where relevant
• Days since the local harvest window began
• Days since a major festival or procurement period
• Rolling averages and volatility over 7, 30, and 90 days
• Arrival changes compared with the previous week or year
• Rainfall totals and deviations from normal
• Lagged prices and lagged arrivals

Encode cyclical calendar variables with sine and cosine transformations so December and January remain close in feature space. For multi-mandi analysis, retain both local features and regional aggregates. A price rise in one market may reflect diversion of arrivals from a neighbouring market rather than a commodity-wide trend.

If the project needs a model trained specifically on Indian price records, review the workflow for fine-tuning a model with Indian mandi price data, but keep forecasting experiments separate from language-model fine-tuning unless there is a clear reason to combine them.

Set up the Autoresearch experiment loop

Create a compact, reproducible repository containing:

• data/ for versioned raw and processed files
• prepare.py for cleaning and feature generation
• train.py for the forecast or seasonal analysis
• evaluate.py for fixed metrics and comparison tables
• config files for target market, commodity, horizon, and date ranges
• A short README explaining assumptions and commands

Define the experiment contract before automation begins. The agent may change feature selection, model parameters, lag windows, or seasonal encodings, but it should not change the test period, target definition, or evaluation metric without an explicit review.

Start with defensible baselines:

1. Last observed price.
2. Same market and same week in the previous year.
3. Seven-day or 30-day moving average.
4. Seasonal median for the relevant market and commodity.

Then allow experiments with linear regression, gradient boosting, random forests, or time-series models. A complex model is useful only if it beats a simple baseline consistently and remains explainable enough for the intended user.

Validate across time, not random rows

Random train-test splits are usually misleading for price forecasting because they allow future patterns into training data. Use rolling or expanding-window validation:

• Train on an initial historical period.
• Validate on the next block of weeks or months.
• Expand the training window and repeat.
• Reserve the latest period as a final untouched test set.

Report more than one metric. MAE is easy to interpret in rupees per unit; RMSE penalises large misses; MAPE can become unstable when prices are low. Add directional accuracy if the decision depends on whether prices will rise or fall. For seasonal volatility, compare standard deviation, interquartile range, and the frequency of unusually large moves.

Ask Autoresearch to produce an experiment table with the commit or configuration, features changed, validation period, metrics, runtime, and failure notes. This makes improvements auditable rather than anecdotal. For policy-sensitive work, the same disciplined approach can be applied to RBI policy trend analysis with Karpathy Autoresearch.

Interpret results for Indian market users

A forecast is not automatically a recommendation to delay selling. Translate outputs into scenarios:

• Base case: expected price and uncertainty range.
• Upside case: lower arrivals or stronger demand.
• Downside case: heavy arrivals, transport disruption, or weak demand.
• Action trigger: a threshold at which storage, transport, or sale timing should be reviewed.

Show confidence intervals and the historical error for each market. A model that performs well in a large, liquid mandi may fail in a smaller market with irregular reporting. Explain which variables drove the result, but do not claim causation from correlation alone. Weather may affect both arrivals and prices, while government announcements can create abrupt regime changes that historical data cannot anticipate.

Common failure modes and safeguards

• Data leakage: enforce date-based splits and inspect feature timestamps.
• Overfitting: cap the number of experiments and retain a final holdout period.
• Market mismatch: do not pool mandis without market identifiers and local validation.
• Unit errors: verify quintal, kilogram, tonne, and commodity-grade conversions.
• Survivorship bias: include discontinued markets or document exclusions.
• False precision: round outputs sensibly and show uncertainty.
• Automation risk: require human review before publishing or acting on findings.

Practical deliverable

A useful final report should include the data sources, cleaning decisions, seasonal definitions, baseline comparison, rolling-validation results, error by market and season, key drivers, limitations, and an exportable CSV of forecasts. Add a simple chart of actual versus predicted prices and a separate chart of arrivals and volatility. That package is more valuable than a single headline prediction because it lets another researcher reproduce the work and challenge its assumptions.

As of 2026, the strongest use of Autoresearch for mandi analysis is not replacing agricultural expertise. It is shortening the cycle between a well-defined question, a tested hypothesis, and an evidence-backed decision while keeping the data, evaluation, and limitations visible.