List a website's pages from any URL on it
List a website's pages from any URL on it: reads robots.txt and XML sitemaps (indexes and .xml.gz included), or the home page's links when there is no sitemap. Returns URLs with last-modified dates; filter by search words or path prefix, up to 1,000 URLs. Find pricing, docs, blog or careers pages before scraping them. Not charged when nothing is found. No account or API key. $0.002 USDC per call.
Answeringour last check, 2026-10-10
1 of 1checks answered this week
468 msmedian answer time
$0.002listed price per call
$0.002price it asked us
Paid test badge: not yet. The checks above are free: we call the tool without paying and read the payment request it sends back. The Verified badge needs paid calls whose answers match the promised output, and nobody can buy a badge.
Endpoint
POST https://x402-agent-api.onrender.com/api/v1/site-map
| Category | Search and research |
|---|---|
| Provider host | x402-agent-api.onrender.com |
| Networks | eip155:8453 |
| Payment schemes | exact |
| Self-reported calls, 30 days | 1 from 1 payers (the provider's figure, not ours) |
Our checks, last 30 days
| Day | Result | HTTP | Asked | Time |
|---|---|---|---|---|
| 2026-10-10 | valid payment request | 402 | $0.002 | 468 ms |
Example input (from the provider)
{
"body": {
"limit": 3,
"path_prefix": "/blog/",
"url": "https://example.com"
},
"bodyType": "json",
"method": "POST",
"type": "http"
}
Promised output schema (from the provider)
{
"$schema": "https://json-schema.org/draft/2020-12/schema",
"properties": {
"input": {
"additionalProperties": false,
"properties": {
"body": {
"additionalProperties": false,
"properties": {
"limit": {
"default": 200,
"description": "How many URLs to return (1-1,000, default 200).",
"maximum": 1000,
"minimum": 1,
"type": "integer"
},
"path_prefix": {
"description": "Only URLs whose path starts with this, e.g. \"/blog/\" or \"/docs\".",
"maxLength": 500,
"minLength": 1,
"type": "string"
},
"search": {
"description": "Only URLs containing every word of this text (case-insensitive) in the URL, or in the title or lastmod when known, e.g. \"pricing\" or \"blog 2026\".",
"maxLength": 200,
"minLength": 1,
"type": "string"
},
"url": {
"description": "The website, or any page on it, e.g. https://example.com. Its origin (scheme and host) is mapped.",
"format": "uri",
"maxLength": 2048,
"type": "string"
}
},
"required": [
"url"
],
"type": "object"
},
"bodyType": {
"enum": [
"json",
"form-data",
"text"
],
"type": "string"
},
"method": {
"enum": [
"POST"
],
"type": "string"
},
"type": {
"const": "http",
"type": "string"
}
},
"required": [
"type",
"method",
"bodyType",
"body"
],
"type": "object"
},
"output": {
"properties": {
"example": {
"type": "object"
},
"type": {
"type": "string"
}
},
"required": [
"type"
],
"type": "object"
}
},
"required": [
"input"
],
"type": "object"
}