XML Sitemap Generator Online
Crawl a website from its home page and build sitemap.xml: same host only, robots.txt respected, noindex and canonical honoured, lastmod from headers.
Result
Press “Run” to see the result.
The sitemap generator crawls a site from the url you give, following links on the same host (after the first redirect, so http → https and www are handled), six pages at a time. It respects robots.txt (rules for APIBoxBot or *), skips pages marked noindex (meta robots or X-Robots-Tag), does not follow nofollow links, and when a page declares a canonical URL elsewhere, the canonical page is listed instead. Images, PDFs, scripts and other files are ignored.
Shape the crawl with max_pages, max_depth (0–10, default 5), path_prefix (only /blog/…), exclude (a regular expression), keep_query=true for sites whose pages differ by query string, and ignore_robots for your own site. Each URL gets its lastmod (from the Last-Modified header), an optional changefreq and a priority that decreases with link depth (priority=false drops it).
The JSON answer lists the pages with their titles and depth, what was skipped and why, the sitemaps declared in robots.txt and the XML itself; format=xml returns the sitemap.xml file directly. Pages per call depend on your plan (Free 50, Starter 200, Pro 500, online tool 20), and a crawl stops after about 25 seconds.
Use this in your code
curl "https://apibox.io.vn/v1/sitemap-generate?url=https://www.sitemaps.org/&max_pages=10" \
-H "Authorization: Bearer YOUR_API_KEY"const res = await fetch('https://apibox.io.vn/v1/sitemap-generate?url=https://www.sitemaps.org/&max_pages=10', {
method: 'GET',
headers: {
Authorization: 'Bearer YOUR_API_KEY',
},
});
const data = await res.json();
console.log(data);import requests
res = requests.get(
"https://apibox.io.vn/v1/sitemap-generate?url=https://www.sitemaps.org/&max_pages=10",
headers={"Authorization": "Bearer YOUR_API_KEY"},
timeout=30,
)
print(res.json())<?php
$ch = curl_init('https://apibox.io.vn/v1/sitemap-generate?url=https://www.sitemaps.org/&max_pages=10');
curl_setopt_array($ch, [
CURLOPT_RETURNTRANSFER => true,
CURLOPT_HTTPHEADER => ['Authorization: Bearer YOUR_API_KEY'],
]);
$response = curl_exec($ch);
print_r(json_decode($response, true));API documentation → Get a free API key
FAQ
Why are some pages missing?
They are not linked from crawled pages, are blocked by robots.txt, noindex or a canonical elsewhere, or the page limit was reached: see skipped and stopped_by.
Can it crawl JavaScript-only sites?
No, links are read from the HTML returned by the server; pages rendered only in the browser are not discovered.
How many pages can it crawl?
Up to 50 (Free), 200 (Starter) or 500 (Pro) pages per call, 20 on the online tool.