SPA catch-all returns HTTP 200 for unknown routes, /robots.txt, and /sitemap.xml
What was observed
Requests to nonexistent paths, /robots.txt, and /sitemap.xml all return the single-page app's homepage document with HTTP 200 and Content-Type text/html, instead of a 404 or the expected text/XML files. The catch-all route serves index.html for everything, including paths that should resolve to static files.
Why it matters
Search engines and crawlers cannot read the crawl rules or the true page inventory, and social platforms cannot preview pages. Unknown routes silently return valid-looking content, which can index duplicates and error pages, inflate crawl budget, and hide real defects. Neither bots nor users get any signal that a page does not exist.
Fix or reference
# Serve the real static files directly; let only known SPA routes and assets
# fall through to the app shell. Everything else gets a real 404.
/robots.txt 200
/sitemap.xml 200
# Force a true 404 for anything that is not a real route or asset (SPA fallback).const SPA_PATHS = new Set(["/", "/check", "/architecture"]);
export default {
async fetch(request, env, ctx) {
const url = new URL(request.url);
const { pathname } = url;
// Let real static assets through untouched.
if (/^\/assets\/|^\/(robots\.txt|sitemap\.xml)$/.test(pathname)) {
return fetch(request);
}
// Unknown paths must not get the shell: return a true 404.
if (!SPA_PATHS.has(pathname)) {
return new Response("Not Found", { status: 404 });
}
return fetch(request); // serve the app shell for known routes
},
};User-agent: *
Allow: /
Sitemap: https://deal212.sites.tab.bot/sitemap.xml<?xml version="1.0" encoding="UTF-8"?>
<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
<url>
<loc>https://deal212.sites.tab.bot/</loc>
<lastmod>2026-10-22</lastmod>
</url>
<url>
<loc>https://deal212.sites.tab.bot/check</loc>
<lastmod>2026-10-22</lastmod>
</url>
</urlset>