An AI-generated site looks done. It has a hero, sections, a color palette, and copy that reads well in a screenshot. Then we open the page source, and the production work is missing. Not some of it. The same eighteen things, every time. None of them change what a human sees in a browser. All of them change what a crawler, a link preview, or a cache does with the page. Here is the list we run before we call anything live.
Crawlability and indexing
This is where the gap is widest, because a client-rendered single-page app hands crawlers an empty div and expects them to run JavaScript to fill it. Many will not. We fix that with static work.
- Prerendered static HTML per route, so the first paint is real content and not a loading spinner.
- A sitemap.xml generated from a single route manifest, so it lists every page and no page twice.
- A robots.txt that points at that sitemap and does not accidentally disallow the whole site.
- A canonical URL on every page, because a screenshot cannot show you a missing canonical tag.
- A meta title and description written per page, not one template repeated across the whole site.
- Structured data as JSON-LD for the page types that support it.
- IndexNow submission on deploy, so search engines learn about changes without waiting for a crawl.
- An llms.txt file describing the site for the AI crawlers that now read it.
Sharing and presentation
A link is content too. When someone pastes the URL into Slack or iMessage, the site is representing itself, and the defaults are usually blank.
- Open Graph tags for the title, description, and image.
- Twitter card tags, which are close to Open Graph but not identical.
- A per-page share image at 1200x630 in PNG. WebP renders unreliably in LinkedIn and iMessage previews, so we ship PNG here even though we prefer WebP elsewhere.
- Descriptive alt text on images, which helps both accessibility and indexing.
Performance
Fast is a feature Google measures. Core Web Vitals are a ranking input, and they punish the exact patterns AI scaffolds produce by default.
- Largest Contentful Paint coming from the static paint, not from a component that mounts after hydration.
- Deferred analytics, so a tracking script never blocks the first render.
- Immutable cache headers on hashed assets and short cache with revalidation on HTML, so browsers reuse what has not changed.
- Responsive images that serve a size appropriate to the device.
Security and correctness
These are one-line headers that never appear in the design, so they never get added.
- A Content Security Policy, plus X-Content-Type-Options and Referrer-Policy.
- HTTPS redirects and trailing-slash and cleanURL consistency, so the same page is not served at two different URLs that then compete with each other.
Operations
The last item is the one that keeps the other seventeen from rotting. We want a deploy pipeline that fails loudly, environment config kept out of the bundle, and a build that regenerates the sitemap on every deploy so it cannot drift away from the routes that actually exist. This is the foundation we lay under every MVP we build, and it is why we wrote our post on encoding these standards as reusable skills instead of remembering them by hand.
None of these eighteen items is hard on its own. Any one of them is an afternoon at most. They get skipped for a single reason: they are invisible in the artifact you are reviewing. You approve a demo by looking at it, and looking at it cannot tell you the canonical tag is missing, the share image is broken, or the whole page is an empty div to a crawler. The demo was never the product. This checklist is the difference.