AI agents increasingly need more than general web search: they need structured, verifiable access to benchmark data. If you are evaluating models for Indian-language understanding, IndicGlue is a useful benchmark family—but agents need a reliable interface for querying tasks, languages, scores, and experiment metadata. WebMCP tools can provide that interface by exposing benchmark operations as typed, permissioned functions that an agent can discover and call.
This guide explains how to build WebMCP tools for agents to query IndicGlue benchmarks for model evaluation. It covers API design, data modeling, tool schemas, validation, security, reproducibility, and production deployment for India-focused AI systems.
What WebMCP Tools Do in Agentic Evaluation
WebMCP refers to a model-context protocol pattern for exposing web capabilities to AI agents through structured tools. Instead of asking an agent to scrape a benchmark webpage or infer query parameters from unstructured documentation, you provide explicit operations such as:
- Listing available IndicGlue tasks and language pairs
- Retrieving benchmark metadata and dataset versions
- Querying leaderboard results for a model
- Comparing two or more models across tasks
- Filtering results by language, metric, split, or evaluation date
- Fetching reproducibility details such as prompts, checkpoints, and code references
A good WebMCP tool should behave like a stable API contract. The agent supplies validated JSON arguments, the server performs an authorized operation, and the response returns concise, machine-readable evidence. Human-readable explanations can be included, but the underlying result should remain structured.
For model evaluation, this distinction matters. An agent should not claim that one model is better based on a loosely interpreted webpage. It should receive the exact task, metric, split, score, sample count, and benchmark version needed to support the conclusion.
Understand IndicGlue Before Designing the Tool Layer
IndicGlue is designed around natural-language understanding evaluation for Indian languages. Its tasks and datasets may cover capabilities such as natural language inference, paraphrase or semantic similarity, named entity recognition, sentiment analysis, question answering, and other language-understanding settings. The exact task inventory, dataset versions, language coverage, and metric definitions should always be obtained from the authoritative benchmark documentation and repository.
Before writing a tool, create a benchmark registry with fields such as:
benchmark_id: stable identifier, for exampleindicglueversion: dataset or evaluation releasetask_id: canonical task namelanguage: ISO-style language code where possiblelanguage_name: display name such as Hindi, Bengali, Tamil, or Marathidataset_split:train,validation,test, or benchmark-specific splitprimary_metric: accuracy, F1, Pearson correlation, exact match, or another metrichigher_is_better: Boolean metric directionsample_count: number of evaluated exampleslicense: dataset licensing informationsource_url: canonical documentation or repository linklast_verified_at: timestamp for metadata verification
Do not assume that scores from different tasks are directly comparable. Accuracy on one classification task and F1 on another measure different properties. Your tool should return metric names and definitions with every result, rather than returning a bare number.
Define the Core WebMCP Tool Surface
Start with a small set of narrowly scoped tools. Narrow tools are easier for agents to understand, validate, secure, and test than one general-purpose query function.
1. List benchmark tasks
{
"name": "indicglue_list_tasks",
"description": "List IndicGlue tasks, supported languages, splits, and primary metrics.",
"inputSchema": {
"type": "object",
"properties": {
"version": { "type": "string" },
"language": { "type": "string" },
"include_deprecated": { "type": "boolean", "default": false }
},
"additionalProperties": false
}
}This tool helps an agent discover valid task identifiers before issuing a score query. It should return canonical IDs, not only display labels.
2. Get benchmark metadata
{
"name": "indicglue_get_metadata",
"description": "Return authoritative metadata for an IndicGlue task or benchmark release.",
"inputSchema": {
"type": "object",
"properties": {
"task_id": { "type": "string" },
"version": { "type": "string" }
},
"required": ["task_id"],
"additionalProperties": false
}
}Metadata should include the task definition, language coverage, metric formula or reference, split semantics, dataset version, and source citation.
3. Query evaluation results
{
"name": "indicglue_query_results",
"description": "Query verified IndicGlue model evaluation results using task, language, metric, and model filters.",
"inputSchema": {
"type": "object",
"properties": {
"model_ids": {
"type": "array",
"items": { "type": "string" },
"maxItems": 50
},
"task_ids": {
"type": "array",
"items": { "type": "string" },
"maxItems": 50
},
"languages": {
"type": "array",
"items": { "type": "string" },
"maxItems": 30
},
"version": { "type": "string" },
"split": { "type": "string" },
"limit": { "type": "integer", "minimum": 1, "maximum": 100 },
"cursor": { "type": "string" }
},
"additionalProperties": false
}
}The response should contain one result object per evaluation, including model_id, task_id, language, metric, value, version, split, sample_count, evaluated_at, and evidence.
4. Compare models
A dedicated comparison tool improves agent reliability because it can enforce consistent filters:
{
"name": "indicglue_compare_models",
"description": "Compare models on identical IndicGlue tasks, languages, versions, splits, and metrics.",
"inputSchema": {
"type": "object",
"properties": {
"model_ids": {
"type": "array",
"minItems": 2,
"maxItems": 10,
"items": { "type": "string" }
},
"task_ids": {
"type": "array",
"items": { "type": "string" }
},
"languages": {
"type": "array",
"items": { "type": "string" }
},
"version": { "type": "string" }
},
"required": ["model_ids"],
"additionalProperties": false
}
}The server should reject comparisons that mix incompatible benchmark versions or metrics unless the response explicitly marks them as non-comparable.
Use a Canonical Result Schema
Agents need predictable output. A practical result schema might look like this:
{
"benchmark": "indicglue",
"benchmark_version": "2024.1",
"task_id": "task_example",
"language": "hi",
"split": "test",
"model_id": "org/model-name",
"metric": {
"name": "f1",
"value": 0.8421,
"scale": "0_to_1",
"higher_is_better": true
},
"sample_count": 1250,
"evaluation": {
"evaluated_at": "2026-01-20T10:30:00Z",
"code_commit": "abc123",
"prediction_artifact": "sha256:..."
},
"evidence": {
"source_url": "https://example.org/benchmark",
"record_id": "result-123"
}
}Avoid ambiguous fields such as score without a metric name or language without a documented code system. Return both raw and display-friendly values when useful. For example, store F1 internally as 0.8421, and optionally include display_value: "84.21" with display_scale: "percentage".
Use explicit null values when information is unavailable. Do not silently substitute zero, an average, or a different split. An agent must be able to distinguish “not evaluated” from “evaluated with a score of zero.”
Build the Backend Around Reproducible Data
A WebMCP endpoint is only as trustworthy as the data behind it. Store benchmark records in a normalized database or versioned files rather than generating responses from manually edited HTML.
A relational model could contain:
benchmarks: benchmark ID, release version, citation, and sourcetasks: task definition, metric, language coverage, and licensemodels: model ID, organization, architecture, parameter information, and revisionevaluations: model, task, language, split, score, and evaluation timestampartifacts: prediction files, logs, configuration, and content hashesprovenance: commit IDs, container image digests, hardware, and evaluator version
For frequently queried data, PostgreSQL works well. For a read-heavy public service, add Redis caching keyed by a normalized hash of the query and benchmark version. Never cache unversioned responses indefinitely; benchmark corrections and metadata updates need an invalidation strategy.
If you ingest results from external leaderboards, keep the original payload, retrieval timestamp, parser version, and source URL. Mark imported results separately from results generated by your own evaluation pipeline. This prevents an agent from presenting third-party scores as independently verified measurements.
Validate Agent Queries Strictly
Agent-generated arguments can be incomplete, contradictory, or malicious. Apply validation at several layers:
- Validate JSON Schema before application logic.
- Resolve task and language IDs against a registry.
- Enforce maximum page size, array length, and query complexity.
- Reject unsupported combinations instead of guessing.
- Normalize model IDs while preserving the original identifier for display.
- Require a benchmark version for historical or comparative queries.
- Return structured errors with an actionable correction.
Example error response:
{
"error": {
"code": "UNSUPPORTED_FILTER",
"message": "Language 'xx' is not available for task 'task_example'.",
"field": "languages",
"allowed_values": ["hi", "ta", "te"]
}
}Do not allow arbitrary SQL, filesystem paths, remote URLs, or shell commands as tool arguments. Tool inputs should represent domain concepts, not backend implementation details.
Add Evidence and Citation Fields by Default
Model-evaluation agents often produce reports, procurement recommendations, or research summaries. Each returned score should therefore include enough evidence for a downstream system or human reviewer to verify it.
Useful evidence fields include:
- Canonical benchmark page or repository URL
- Dataset and benchmark version
- Model repository and immutable revision
- Evaluation script version or commit
- Prediction artifact hash
- Hardware and software environment, where relevant
- Timestamp and evaluator identity
- Whether the score was self-reported, imported, or independently reproduced
You can also expose a separate indicglue_get_evidence tool that accepts a result ID and returns complete provenance. Keep the default query response compact, but make deep verification available without requiring an agent to scrape another website.
Design for Indian-Language Evaluation Realities
India-focused evaluation introduces practical issues that should be represented in the schema and documentation.
First, language labels can be inconsistent across datasets. Use stable language codes and retain script information. Hindi in Devanagari, transliterated Hindi in Latin script, and code-mixed Hinglish should not be silently grouped together.
Second, tokenization and normalization can affect metrics. Record whether Unicode normalization, punctuation handling, whitespace normalization, transliteration, or script conversion was applied. For generative tasks, specify exact-match normalization and any accepted aliases.
Third, regional and domain variation matters. A score on formal news text may not predict performance on conversational, educational, legal, agricultural, or customer-support content. Return domain metadata when available, and avoid presenting a single aggregate score as universal language capability.
Fourth, privacy and data governance require care. Do not expose benchmark examples containing personal information through a broad query tool. Use aggregate results by default, apply access controls to raw predictions, and follow applicable Indian privacy and security requirements.
Implement the Tool Server
The server can be implemented in Python, TypeScript, Go, or another language with strong JSON validation. A typical request lifecycle is:
1. Authenticate the caller or apply anonymous rate limits.
2. Parse the tool name and JSON arguments.
3. Validate the schema.
4. Resolve identifiers through the benchmark registry.
5. Construct a parameterized database query.
6. Apply authorization, pagination, and result limits.
7. Attach version and provenance fields.
8. Return structured JSON and tracing metadata.
Use typed models such as Pydantic in Python or Zod in TypeScript. Keep tool descriptions specific: state what the function returns, what filters mean, and what the tool does not do. Clear descriptions improve agent tool selection and reduce unnecessary calls.
For example, a Python service might separate concerns as follows:
class QueryResults(BaseModel):
model_ids: list[str] = Field(default_factory=list, max_length=50)
task_ids: list[str] = Field(default_factory=list, max_length=50)
languages: list[str] = Field(default_factory=list, max_length=30)
version: str | None = None
split: str | None = None
limit: int = Field(default=25, ge=1, le=100)
async def query_results(args: QueryResults) -> dict:
filters = registry.validate(args)
rows = await repository.find_results(filters)
return serializer.to_agent_response(rows, filters)The repository layer should use parameterized queries and enforce access policies independently of the agent-facing layer.
Test Agent Behavior, Not Just API Correctness
Traditional unit tests are necessary but insufficient. Test whether agents can use the tools correctly.
Create evaluation scenarios such as:
- “Which model has the highest Hindi F1 on task X in version Y?”
- “Compare models A and B only on shared Tamil tasks.”
- “Show whether this score was independently reproduced.”
- “Find all tasks where model A was evaluated on the test split.”
- “Explain why these two scores cannot be compared.”
Measure:
- Tool-selection accuracy
- Argument validity rate
- Unsupported-claim rate
- Citation completeness
- Number of tool calls per answer
- Latency and error recovery
- Correct handling of missing data
Include adversarial cases: unknown language codes, mixed benchmark versions, duplicate model names, pagination abuse, prompt injection inside imported metadata, and requests for restricted prediction data.
Security, Rate Limits, and Operations
Public benchmark tools should be treated as production APIs. Apply HTTPS, authentication for write or private operations, per-client quotas, request logging, and timeouts. Add response-size limits so a broad query cannot exhaust memory or context windows.
Keep imported benchmark text isolated from tool instructions. A dataset description or model card can contain text designed to influence an agent. Treat it as untrusted content and label it as data in the response.
Monitor query volume, error codes, cache hit rates, p95 latency, and unusual access patterns. Publish a status page or health endpoint for availability. For regulated or research-sensitive deployments, retain audit logs showing which result version was returned to which client.
A Practical Rollout Plan
A staged implementation reduces risk:
1. Metadata phase: expose task, language, metric, and version discovery.
2. Read-only results phase: provide paginated verified scores with citations.
3. Comparison phase: add strict cross-model and cross-task comparisons.
4. Evidence phase: expose artifacts, code revisions, and reproducibility status.
5. Private evaluation phase: allow authenticated users to submit model results.
6. Continuous evaluation phase: integrate CI jobs that publish immutable records after validation.
Begin with a small, well-documented subset of IndicGlue tasks. Expand only after schema stability, metric correctness, and agent behavior have been tested.
Common Mistakes to Avoid
- Returning scores without metric names or benchmark versions
- Mixing validation and test results in one field
- Treating language and script as interchangeable
- Computing averages across incompatible metrics
- Letting agents pass arbitrary SQL or URLs
- Omitting source citations and provenance
- Returning HTML when the agent needs structured JSON
- Hiding missing or non-comparable results
- Publishing raw examples without privacy review
- Assuming leaderboard values are independently verified
The goal is not merely to make benchmark data searchable. It is to make evaluation claims precise, reproducible, and auditable.
FAQ
What is the best first WebMCP tool for IndicGlue?
Start with a read-only task-discovery tool, followed by a versioned results query. Agents need valid task IDs, language coverage, and metric definitions before they can compare models safely.
Should the tool return an overall IndicGlue score?
Only if the benchmark defines a valid aggregation method. Otherwise, return per-task and per-language results and explain why metrics should not be averaged.
How can I prevent hallucinated benchmark claims?
Require the agent to call the results tool, return immutable evidence fields, and instruct downstream report generation to cite the returned result IDs and benchmark versions.
Can WebMCP tools run private model evaluations?
Yes, but private submission and raw prediction access should use authentication, authorization, malware and privacy checks, quotas, and separate write-capable tools from public read-only tools.
Apply for AI Grants India
Building a trustworthy WebMCP layer for IndicGlue can accelerate Indian-language AI research, evaluation, and deployment. Apply to AI Grants India for support, funding opportunities, and ecosystem guidance for your AI project.