Reinforcement learning (RL) can produce impressive backtests, but a high return on historical data is not evidence of a useful trading system. A credible evaluation must show whether an agent adds value after costs, survives different market regimes, and beats simple alternatives on the same information set.
For Indian builders, Maharashtra-linked blue-chip companies provide a practical universe for experimentation. The state is home to major financial, technology, industrial, pharmaceutical, consumer, and infrastructure businesses listed on Indian exchanges. However, “Maharashtra blue chip” is not a formal index category. Define the universe transparently—for example, companies headquartered or substantially operating in Maharashtra, with strong liquidity and large market capitalisation—and record the membership and selection date.
This guide explains how to benchmark RL agents without confusing a stock basket with a risk-free standard or allowing accidental look-ahead bias.
Define the benchmark before training
Start by writing a short benchmark specification. It should state:
- Universe: NSE- or BSE-listed securities meeting your Maharashtra and liquidity criteria.
- Rebalancing rule: Equal weight, market-cap weight, or another fixed method.
- Rebalance frequency: Monthly or quarterly is easier to reproduce than daily discretionary changes.
- Data frequency: Daily bars for a swing strategy; intraday data only when you can model timestamps, latency, and liquidity properly.
- Evaluation dates: Fixed train, validation, test, and walk-forward periods.
- Currency and return convention: INR total returns, including dividends where possible.
Do not select today’s winners and apply them to the full historical period. That creates survivorship bias. Use historical constituents if available, or clearly label a current-universe study as limited. Adjust for stock splits, bonuses, rights issues, and dividends, and preserve delisted securities where your data provider supports them.
A useful benchmark suite includes more than one portfolio. Compare the agent with a buy-and-hold portfolio, equal-weight buy-and-hold, a broad Indian index such as Nifty 50, and a simple risk-controlled strategy. This makes it easier to determine whether the agent has learned a genuine policy or merely benefited from market direction.
Build a realistic trading environment
The environment should represent what a trader could have known and executed at each decision point. A typical state may include lagged returns, rolling volatility, moving averages, volume features, cash, current holdings, and portfolio weights. Every feature must be shifted so that information from day *t* is used only for an order executed at *t+1*, unless you explicitly model an end-of-day execution process.
Use portfolio weights or target allocations instead of unconstrained buy/sell actions when the goal is asset allocation. Add constraints for:
- Maximum position and sector exposure
- Cash allocation and leverage
- Turnover per rebalance
- Short selling, if permitted by the experiment
- Minimum trade size and lot conventions
Transaction costs should include brokerage, exchange charges, Securities Transaction Tax, GST, SEBI charges, stamp duty, and an estimate of slippage. The exact treatment depends on the instrument and execution model. A simple percentage cost is acceptable for an initial study, but run sensitivity tests with higher costs. An agent that works only at zero cost is not a credible result.
Reward design requires equal care. Raw profit encourages risk-taking and makes comparisons difficult across periods. Consider a net portfolio return minus transaction costs, with penalties for drawdown, excessive turnover, leverage, or constraint violations. Keep the training reward separate from the final score: a reward shaped for learning should not replace standard financial reporting.
Select algorithms and control randomness
PPO, SAC, and actor-critic methods are common choices for continuous portfolio actions. DQN is more natural for a small, discrete action space, but it can become unwieldy when the agent selects among many securities and position sizes. Algorithm choice matters less than disciplined experiment design.
Run multiple random seeds and report the mean, median, and dispersion of results. Save the environment version, feature definitions, hyperparameters, seed values, code revision, and data snapshot. A beginner building a reproducible baseline can also use the principles in machine learning portfolio projects for beginners in India, especially around documentation and evaluation splits.
Do not tune repeatedly on the final test set. Use training data for learning, validation data for model selection, and a locked test period for the final report. For non-stationary markets, walk-forward evaluation is usually more informative: train on an expanding or rolling window, validate on the next period, then advance the window and repeat.
Metrics that reveal real performance
Report portfolio-level and trade-level results. At minimum include:
- Cumulative and annualised return: State whether returns are price or total returns.
- Volatility: Annualise consistently using the data frequency.
- Sharpe ratio: Specify the INR risk-free rate, sampling frequency, and annualisation method.
- Sortino ratio: Useful when downside volatility matters more than upside variation.
- Maximum drawdown: Include drawdown duration and recovery time.
- Calmar ratio: Annualised return divided by maximum drawdown.
- Turnover and costs: Show gross and net performance separately.
- Hit rate and profit factor: Helpful, but insufficient on their own.
- Exposure statistics: Cash, leverage, concentration, and sector weights.
Use confidence intervals or bootstrap analysis where appropriate. A small difference in Sharpe ratio may be noise, particularly when the test period is short or returns are highly autocorrelated. Also compare the agent’s performance during bull, bear, sideways, high-volatility, and crisis-like periods rather than reporting only one aggregate number.
A reproducible evaluation workflow
1. Freeze the universe and data snapshot. Record source, download date, corporate-action treatment, and missing-value policy.
2. Create chronological splits. Never shuffle time-series observations.
3. Implement baselines first. Verify that buy-and-hold, equal weight, and index comparisons produce sensible results.
4. Test the environment. Check cash conservation, position limits, reward timing, and cost calculations with hand-worked examples.
5. Train several seeds. Store checkpoints and validation results without inspecting the final test period.
6. Run walk-forward tests. Refit only with information available at each historical decision point.
7. Stress the assumptions. Increase slippage, delay execution, remove selected features, and vary the rebalancing schedule.
8. Publish an audit table. Include dates, universe, costs, constraints, metrics, seeds, and baseline results.
For larger experiments, separate data ingestion, feature generation, simulation, training, and reporting into versioned modules. This is the same engineering discipline needed when building distributed systems with AI agents: clear interfaces and observable runs reduce silent errors.
Common failure modes
Look-ahead bias appears when adjusted prices, future index membership, or same-day closing data leaks into the action. Survivorship bias appears when failed or delisted companies are excluded. Reward hacking occurs when the agent exploits simulator bugs, unlimited liquidity, or an improperly calculated reward. Overfitting appears when features, costs, and hyperparameters are repeatedly changed until one test period looks excellent.
Feature importance does not prove causality. A model may rely on unstable correlations, and a strategy that performs well on large-cap shares may fail after liquidity or sector conditions change. Include a simpler model and a no-skill baseline. If the RL agent cannot beat these after costs with stable risk, the result is still valuable: it tells you that added complexity is not justified.
What a credible result looks like
A strong report does not promise profits. It demonstrates that the agent was evaluated fairly, beats relevant baselines net of realistic costs, remains viable across seeds and periods, and has understandable exposure and drawdown behaviour. Treat the output as research, not investment advice, and obtain appropriate compliance and risk review before deploying capital.
The best next step is usually a paper-trading phase with live data, logged decisions, rejected orders, turnover, and slippage. Compare live paper results with the pre-registered backtest assumptions. Only after that should you consider limited production capital, strict exposure limits, and a kill switch.