Turn a small website, or one section of it, into clean Markdown
Turn a small website, or one section of it, into clean Markdown: starts from your URL, finds pages from the sitemap or by following links on the same site, obeys robots.txt, and returns up to 10 pages (title, Markdown, address) in one call. Great for docs, blogs and company sites. ?url=https://example.com/docs/&maxPages=10&path=/docs/
Answeringour last check, 2026-09-29
1 of 1checks answered this week
20 msmedian answer time
$0.03listed price per call
$0.03price it asked us
Paid test badge: not yet. The checks above are free: we call the tool without paying and read the payment request it sends back. The Verified badge needs paid calls whose answers match the promised output, and nobody can buy a badge.
Endpoint
GET https://aayatai.com/crawl
| Category | Company and compliance |
|---|---|
| Provider host | aayatai.com |
| Networks | eip155:137, eip155:42161, eip155:8453, solana:5eykt4UsFv8P8NJdTREpY1vzqKqZKvdp |
| Payment schemes | exact |
| Self-reported calls, 30 days | 1 from 1 payers (the provider's figure, not ours) |
Our checks, last 30 days
| Day | Result | HTTP | Asked | Time |
|---|---|---|---|---|
| 2026-09-29 | valid payment request | 402 | $0.03 | 20 ms |
Example input (from the provider)
{
"method": "GET",
"queryParams": {
"maxPages": 2,
"url": "https://example.org"
},
"type": "http"
}
Promised output schema (from the provider)
{
"$schema": "https://json-schema.org/draft/2020-12/schema",
"properties": {
"input": {
"additionalProperties": false,
"properties": {
"method": {
"enum": [
"GET"
],
"type": "string"
},
"queryParams": {
"properties": {
"maxPages": {
"default": 5,
"description": "Most pages to return (1-10).",
"maximum": 10,
"minimum": 1,
"type": "integer"
},
"path": {
"description": "Only crawl pages under this path (default: the start page's folder).",
"maxLength": 300,
"type": "string"
},
"url": {
"description": "Start page, e.g. https://example.com/docs/.",
"format": "uri",
"maxLength": 2048,
"type": "string"
}
},
"required": [
"url"
],
"type": "object"
},
"type": {
"const": "http",
"type": "string"
}
},
"required": [
"type",
"method"
],
"type": "object"
},
"output": {
"properties": {
"example": {
"properties": {
"complete": {
"description": "False if more matching pages were found than returned.",
"type": "boolean"
},
"count": {
"type": "integer"
},
"crawledAt": {
"type": "string"
},
"discoveredFrom": {
"enum": [
"sitemap",
"links"
],
"type": "string"
},
"pages": {
"items": {
"properties": {
"chars": {
"type": "integer"
},
"markdown": {
"type": "string"
},
"title": {
"type": "string"
},
"truncated": {
"type": "boolean"
},
"url": {
"type": "string"
}
},
"type": "object"
},
"type": "array"
},
"robots": {
"description": "robots.txt status: ok, none or unreachable.",
"type": "string"
},
"skipped": {
"description": "Pages not read and why (robots.txt, error, not HTML).",
"items": {
"type": "object"
},
"type": "array"
},
"start": {
"type": "string"
}
},
"required": [
"start",
"pages",
"count",
"discoveredFrom",
"robots",
"complete"
],
"type": "object"
},
"type": {
"type": "string"
}
},
"required": [
"type"
],
"type": "object"
}
},
"required": [
"input"
],
"type": "object"
}