How to Create a WebMCP Tool for Agents to Scrape BSE and NSE Market Data

AI agents can answer market questions far more reliably when they use structured tools instead of scraping arbitrary pages. A WebMCP-style tool can expose carefully defined capabilities—such as fetching a quote, reading an index snapshot, or retrieving historical candles—while keeping authentication, rate limits, provenance, and exchange licensing under your control.

For BSE and NSE data, the key challenge is not merely writing a scraper. It is designing an agent-facing interface that uses authorised data sources, handles dynamic web applications, distinguishes delayed from real-time prices, and prevents an agent from making unsupported financial claims. This guide explains how to create a WebMCP tool for agents to scrape BSE and NSE market data using a safer API-first architecture.

What is a WebMCP tool?

WebMCP is best understood as a model-facing tool layer for web capabilities. Instead of asking an AI model to browse pages and infer values from HTML, you expose typed operations with clear inputs and outputs. The model selects a tool; your server validates the request, obtains data, normalises it, and returns structured evidence.

A market-data tool might expose operations such as:

get_quote: latest permitted quote for an equity or index
get_ohlcv: historical open, high, low, close, and volume candles
search_instrument: resolve a company name to an exchange symbol
get_market_status: determine whether an exchange is open, closed, or in a special session
get_corporate_actions: retrieve dividends, splits, bonuses, or rights information

The tool should not expose unrestricted browsing. Narrow operations make responses more predictable, easier to audit, and safer for financial use cases.

Check data rights before writing code

BSE and NSE data may be subject to exchange terms, redistribution restrictions, licensing requirements, and vendor-specific conditions. A publicly visible webpage is not automatically an authorised source for automated extraction or commercial redistribution.

Before deployment:

1. Review the relevant exchange website terms and robots directives.
2. Confirm whether your use case allows automated access.
3. Obtain a licensed feed or use an authorised market-data provider where required.
4. Check whether the data is real-time, delayed, end-of-day, indicative, or derived.
5. Verify redistribution rights for AI-agent responses and customer applications.
6. Store source, timestamp, entitlement, and attribution metadata.

For a prototype, use synthetic fixtures or a provider that explicitly permits development access. Do not bypass CAPTCHAs, bot protections, authentication controls, paywalls, or technical restrictions. If a provider offers an official API, use it instead of scraping its frontend.

Recommended architecture

A robust implementation separates the agent protocol from the exchange or vendor integration:

AI agent
   |
   | WebMCP tool call
   v
Tool gateway
   |-- schema validation
   |-- authentication and tenant policy
   |-- rate limiting
   |-- audit logging
   v
Market-data adapter
   |-- authorised API or licensed feed
   |-- optional permitted HTML adapter
   v
Normalizer and cache
   v
Structured response with provenance

This design prevents provider-specific details from leaking into the agent contract. If you change vendors, the agent still calls get_quote with the same schema.

Use a backend service rather than placing API keys in browser JavaScript. Recommended components include:

Tool server: Node.js, Python, Go, or another runtime that supports your WebMCP integration.
Schema validator: JSON Schema, Zod, Pydantic, or equivalent.
Provider adapters: One adapter per authorised source.
Cache: Redis or a database with TTLs appropriate to each data type.
Observability: Structured logs, metrics, traces, and failed-request alerts.
Secrets manager: Environment-level or cloud secret storage, never source code.

Define a narrow tool contract

A model should not need to understand exchange-specific URL formats. It should provide an instrument identifier, exchange, date range, and requested fields. For example:

{
  "name": "get_quote",
  "description": "Retrieve an authorised quote for an instrument listed on BSE or NSE. Returns the observation timestamp and data status.",
  "inputSchema": {
    "type": "object",
    "required": ["exchange", "symbol"],
    "properties": {
      "exchange": {
        "type": "string",
        "enum": ["BSE", "NSE"]
      },
      "symbol": {
        "type": "string",
        "minLength": 1,
        "maxLength": 30
      },
      "market": {
        "type": "string",
        "enum": ["equity", "index", "etf"]
      }
    },
    "additionalProperties": false
  }
}

For production, consider accepting a canonical instrument identifier rather than relying only on a symbol. The same company can have different identifiers across exchanges, and names can be ambiguous. A search operation should return a stable instrument ID, exchange, symbol, security name, ISIN where permitted, and instrument type.

A quote response should be explicit about freshness:

{
  "instrument": {
    "exchange": "NSE",
    "symbol": "INFY",
    "name": "Example Instrument"
  },
  "quote": {
    "last": 1500.25,
    "open": 1492.00,
    "high": 1511.40,
    "low": 1488.10,
    "previousClose": 1490.50,
    "volume": 1234567,
    "currency": "INR"
  },
  "asOf": "2026-09-03T10:15:24+05:30",
  "status": "delayed",
  "source": "authorised-provider-name",
  "sourceUrl": "https://provider.example/record/123",
  "disclaimer": "Market data may be delayed and is not investment advice."
}

Never allow the model to infer that a value is live if the provider marks it delayed. Return status values such as real_time, delayed, end_of_day, indicative, or unknown.

Build the provider adapter

The adapter should be the only layer that knows how to obtain data. Prefer an official API or licensed feed. If an authorised source provides JSON, parse JSON directly instead of extracting text from rendered pages.

A simplified Python-style adapter might look like this:

from datetime import datetime, timezone

class MarketDataError(Exception):
    pass

async def get_quote(provider, exchange: str, symbol: str):
    if exchange not in {"BSE", "NSE"}:
        raise MarketDataError("Unsupported exchange")

    normalized_symbol = symbol.strip().upper()
    if not normalized_symbol or len(normalized_symbol) > 30:
        raise MarketDataError("Invalid symbol")

    raw = await provider.quote(exchange=exchange, symbol=normalized_symbol)
    if not raw:
        raise MarketDataError("No quote returned")

    return {
        "instrument": {
            "exchange": exchange,
            "symbol": normalized_symbol,
            "name": raw.get("name")
        },
        "quote": {
            "last": raw.get("last"),
            "open": raw.get("open"),
            "high": raw.get("high"),
            "low": raw.get("low"),
            "previousClose": raw.get("previous_close"),
            "volume": raw.get("volume"),
            "currency": "INR"
        },
        "asOf": raw.get("as_of") or datetime.now(timezone.utc).isoformat(),
        "status": raw.get("status", "unknown"),
        "source": provider.name,
        "disclaimer": "Verify data status and consult a qualified adviser."
    }

In a real service, add strict numeric validation. Reject impossible values, unexpected nulls, malformed timestamps, and provider responses that do not match the expected schema. Preserve the raw provider payload only when your licence and privacy policy permit it.

Handling HTML pages safely

If you have explicit permission to automate a web page, treat HTML extraction as a fallback, not your primary data strategy. Exchange pages can change markup, use client-side rendering, or return different content based on session state.

A permitted HTML adapter should:

Use a clear, rate-limited request policy.
Identify itself appropriately where required.
Respect terms, robots directives, and access restrictions.
Avoid login circumvention, CAPTCHA solving, and anti-bot evasion.
Parse semantic attributes or embedded structured data when documented.
Validate that the page’s exchange, symbol, and timestamp match the request.
Fail closed when selectors break instead of returning guessed values.
Record retrieval time and page provenance.

Do not ask an agent to “scrape any page containing the price.” That creates prompt-injection, symbol-confusion, and stale-data risks. Your server should select approved domains and endpoints through an allowlist.

Add caching and exchange-aware freshness

Caching improves resilience and reduces unnecessary provider requests. TTLs should reflect the data’s entitlement and use case:

Instrument search: hours or days, with invalidation for corporate changes.
Market status: seconds to a minute during sessions.
Intraday quotes: provider-defined freshness, often seconds or minutes.
Historical candles: minutes to hours after publication, depending on corrections.
End-of-day data: cache until the next expected publication or correction window.

Include the cache state in internal logs and consider returning retrievedAt separately from asOf. These timestamps answer different questions: when the market observation occurred and when your system obtained it.

India-aware handling should account for the Asia/Kolkata timezone, exchange holidays, trading sessions, special sessions, auction windows, and corporate-action adjustments. Do not simply label a missing quote as zero when the exchange is closed.

Security and agent guardrails

An agent-facing financial tool needs stronger controls than a normal public endpoint.

Input controls

Allow only BSE and NSE values from an enum.
Normalise and length-limit symbols.
Reject control characters, URLs, HTML, and unexpected fields.
Use canonical instrument lookup to prevent ambiguous names.
Limit historical date ranges and maximum candle counts.

Request controls

Require application authentication and tenant-level authorisation.
Apply per-user, per-tenant, and provider rate limits.
Use outbound domain allowlists and network egress controls.
Set connection, read, and total request timeouts.
Prevent server-side request forgery by never accepting arbitrary URLs.

Output controls

Return structured data, not unverified prose.
Include source and observation timestamps.
Preserve missing values as null; do not fabricate them.
Attach a data-status and financial-information disclaimer.
Prevent the tool from issuing buy, sell, or target-price recommendations unless a separately governed product supports that function.

Prompt injection can occur in page content, company names, news text, or provider error messages. Treat all retrieved text as untrusted data. The tool server, not the webpage, defines the operation and response format.

Error design and fallback behaviour

Use machine-readable errors so agents can recover without hallucinating:

{
  "error": {
    "code": "DATA_UNAVAILABLE",
    "message": "The authorised provider did not return a quote.",
    "retryable": true,
    "details": {
      "exchange": "BSE",
      "symbol": "ABC"
    }
  }
}

Useful error codes include INVALID_SYMBOL, UNSUPPORTED_EXCHANGE, ENTITLEMENT_REQUIRED, RATE_LIMITED, MARKET_CLOSED, DATA_STALE, PROVIDER_TIMEOUT, and DATA_UNAVAILABLE. Never silently fall back from a licensed real-time feed to an unknown webpage. If a fallback is permitted, label it clearly and preserve the source.

Testing your WebMCP market tool

Test at four levels:

1. Schema tests: invalid exchanges, empty symbols, extra properties, oversized date ranges.
2. Adapter tests: provider fixtures, malformed numbers, missing timestamps, changed field names, throttling, and timeouts.
3. Contract tests: verify every tool response matches the published schema.
4. Agent evaluations: ask realistic questions and check whether the agent selects the correct tool, reports freshness, and avoids unsupported conclusions.

Create fixtures for NSE and BSE instruments, indices, ETFs, suspended securities, closed-market periods, and corporate actions. Include cases where the two exchanges have different symbols or last-traded timestamps.

Monitor production with metrics such as provider latency, cache-hit rate, error rate, stale-response count, schema failures, and calls by tenant. Alert when a provider suddenly returns null prices, a page structure changes, or timestamp freshness exceeds your policy.

Compliance and user disclosure in India

Market-data tooling is not automatically investment advice, but the surrounding application may create regulatory and consumer-protection obligations. Define whether your product only displays factual data or also analyses, ranks, recommends, or executes trades. Obtain advice from qualified Indian legal and compliance professionals for your specific model.

At minimum, document:

Data source and licence or permission basis.
Real-time versus delayed status.
Data retention and redistribution rules.
User consent and authentication practices.
Correction and incident-response procedures.
Whether responses are informational and not investment advice.

Do not claim that a quote is official, live, or complete unless your provider entitlement and technical controls support that claim.

A practical implementation checklist

Before connecting an AI agent to BSE or NSE data, confirm that you can answer “yes” to these questions:

Do we have permission or a licence for the selected data source?
Is the tool API-first and restricted to approved providers?
Are exchange, symbol, date, and field inputs schema-validated?
Do responses include asOf, retrieval time, source, currency, and freshness status?
Are stale, missing, and conflicting values handled explicitly?
Are API keys stored server-side and rotated securely?
Are rate limits, audit logs, timeouts, and domain allowlists enabled?
Can the agent distinguish facts from recommendations?
Do automated tests cover provider failures and exchange-specific cases?
Is there a documented process for data corrections and source changes?

A narrow, well-documented tool will generally outperform a broad “browse the exchange website” instruction. It is easier for agents to call, easier for engineers to test, and safer to operate at scale.

FAQ

Can I scrape NSE or BSE pages for free?

Public accessibility does not necessarily grant permission for automated extraction or redistribution. Check exchange terms and use an authorised API, licensed feed, or provider that explicitly permits your use case.

Should a WebMCP tool return real-time prices?

Only if your data entitlement, provider, infrastructure, and disclosure support real-time delivery. Otherwise, label values as delayed, end-of-day, indicative, or unknown.

What is better for agents: HTML or JSON?

Structured JSON from an authorised API is usually more reliable. HTML extraction should be limited to permitted sources and protected with validation, provenance, and fail-closed behaviour.

Can the agent place trades through the same tool?

Trading requires a separate, strongly authenticated and authorised workflow with confirmations, limits, auditability, and compliance review. Keep read-only market data separate from execution tools.

How do I support both BSE and NSE symbols?

Use exchange-qualified identifiers and, where possible, a canonical instrument ID. Do not assume that a company name or symbol uniquely identifies the same security on both exchanges.

Apply for AI Grants India

Building an agent-native market-data product in India? Apply to AI Grants India for support, visibility, and opportunities to develop your AI startup responsibly.