Skip to content

feat(blog-app): prerender the blog so crawlers see the content (BAPP-13) - #50

Open
nakomis wants to merge 1 commit into
mainfrom
bapp-13-prerender-blog-for-crawlers
Open

feat(blog-app): prerender the blog so crawlers see the content (BAPP-13)#50
nakomis wants to merge 1 commit into
mainfrom
bapp-13-prerender-blog-for-crawlers

Conversation

@nakomis

@nakomis nakomis commented Aug 6, 2026

Copy link
Copy Markdown
Owner

The problem

blog.nakomis.com was a client-rendered SPA. Every URL returned the same ~2,333-byte empty shell, and the content only appeared after React booted. Crawlers, link unfurlers and the AdSense reviewer — which is already running against these pages (ca-pub-1712135199843745 is live in index.html) — all saw a page with nothing in it.

Two independent halves, either of which stands on its own.

1. Surfacing the markdown

deploy.sh has always synced content/blog/*.md to /posts/, and CloudFront has always served it:

GET /posts/2026-05-15-giving-claude-a-persistent-memory.md
→ 200  text/markdown  16,728 bytes

Nothing linked to it, so the cleanest representation of every post was effectively undiscoverable. buildContent.ts now emits llms.txt from the same loop that writes the sitemap, so the two cannot drift.

The sitemap stays HTML-only — sitemaps are for canonical pages, and listing both representations would muddle canonicalisation. Each prerendered page also carries <link rel="alternate" type="text/markdown">.

2. Prerendering

A second Vite pass builds src/entry-server.tsx; scripts/prerender.ts renders every route to its own HTML file with per-page <title>, description, og:*, Twitter card and rel=canonical.

before after
/ 2,333 b 19.8 KB
/2026-07-30-the-1092-bytes… 2,333 b 38.3 KB
/2026-03-30-rag-search-static-blog 2,333 b 33.1 KB

25 routes plus a real 404 page.

Making the content load synchronous

getBlogPosts() was async over a compile-time constant and was awaited inside a useEffect. Effects don't run under renderToString, so prerendering as-was would have baked Loading post... into all 24 pages.

It does no I/O, so the effect, the loading state and the error state all go. Readers stop seeing a loading frame on content that was already in the bundle.

ThemeToggle

useState<Choice>(readChoice) reads localStorage during render, and therefore during SSR. It's caught (it was written for private-browsing mode), but the server would emit the system icon whilst the client's first render emitted dark — a hydration mismatch for anyone who has picked a theme. It now starts at system and resolves in an effect.

Nothing flashes: the inline script in index.html still applies the real theme before first paint. Only the button's own icon settles a moment later.

3. Infrastructure

The rewrite is merged into the existing viewer-request function, not added alongside it — CloudFront permits only one function per event type per behaviour, and legacyRedirectFunction was already there.

/<slug>/ is rewritten rather than 301'd. A redirect drops the query string (CloudFront doesn't carry it into location), which would silently eat utm_* and gclid on any ad click landing on the slashed form. The rel=canonical on every page is the correct tool for the duplicate.

The soft 404 is fixed. The error responses mapped 404 and 403 to /index.html with status 200 — necessary while the SPA had to boot and route client-side, but it meant every typo URL looked to a crawler like a valid page carrying thin content. Now every route is a real file, so a miss is a genuine miss. (403 is listed as well as 404 because the bucket is private behind OAC with only s3:GetObject — with no s3:ListBucket, S3 answers a missing key with AccessDenied.)

⚠️ Deploy ordering matters

There is no CI for infra in this repo — cdk deploy is manual, per infra/README.md. The web deploy must go first.

  • Web first, then infra — safe. Between the two, /slug still resolves via the old fallback; React reports a hydration mismatch and client-renders, so the page works, just less efficiently.
  • Infra first — breaks every post URL. The function rewrites /slug/slug.html, S3 misses, and /404.html isn't in the bucket yet.

So: let scheduled-publish build and deploy the site, confirm the .html files are live, then cdk deploy BlogStack --profile nakom.is-admin.

Verification

Tested against a local server reproducing the CloudFront behaviour (rewrite, trailing slash, 404 mapping):

  • Status codes: / 200, /slug 200, /slug/ 200 (same bytes), /typo 404, /posts/*.md 200 text/markdown, llms.txt and sitemap.xml 200
  • Exactly one <title> and one description per page; og/canonical/alternate correct; 91 <p> in the sample post
  • No React hydration warnings with a theme stored in localStorage
  • Theme persists across reload with no flash; client-side navigation and document.title updates still work
  • The rewrite logic exercised over JS/CSS/.md/.ico/.txt/.xml assets, extensionless paths, trailing slashes, deep paths and both hostnames

cdk synth is clean and the synthesised function and error responses were checked directly.

Not doing

AdSense placement changes; anything to cv.nakomis.com/nakom.is (stays Disallow: /); the Substack mirror or announce pipeline; a framework migration.

blog.nakomis.com was a client-rendered SPA, so every URL returned the same
~2.3KB empty shell. Crawlers, link unfurlers and the AdSense reviewer — which
is already running against these pages — all saw markup with no content in it.

Two independent halves.

Surfacing the markdown. deploy.sh has always synced content/blog/*.md to
/posts/, and CloudFront has always served it, but nothing linked to it, so the
cleanest representation of every post was undiscoverable. buildContent.ts now
emits llms.txt from the same loop that writes the sitemap, so the two cannot
drift. The sitemap stays HTML-only: sitemaps are for canonical pages, and
listing both representations would muddle canonicalisation.

Prerendering. A second Vite pass builds src/entry-server.tsx, and
scripts/prerender.ts renders every route to its own HTML file with per-page
title, description, og:* and canonical tags — plus a rel=alternate pointing at
the markdown. Pages go from 2.3KB of nothing to 12-38KB of article.

That required making the content load synchronous. getBlogPosts() was async
over a compile-time constant and was awaited in a useEffect, which never runs
under renderToString — prerendering it as-was would have baked in "Loading
post...". It does no I/O, so the effect, the loading state and the error state
all go; readers stop seeing a loading frame on content that was already in the
bundle.

ThemeToggle seeded useState from localStorage, which runs during render and so
during SSR. It is caught (it was written for private mode) but the server would
emit the 'system' icon whilst the client's first render emitted 'dark', which
is a hydration mismatch for anyone who has picked a theme. It now starts at
'system' and resolves in an effect. Nothing flashes — the inline script in
index.html still sets the real theme before first paint.

Infrastructure. The URI rewrite has to be merged into the existing viewer-
request function rather than added alongside it, because CloudFront permits
only one function per event type per behaviour. /<slug>/ is rewritten rather
than 301'd: a redirect drops the query string, which would silently eat utm_*
and gclid on any ad click landing on the slashed form. The rel=canonical on
every page is the right tool for the duplicate.

The error responses mapped 404 and 403 to /index.html with status 200. That was
necessary while the SPA had to boot and route client-side, but it is a soft 404
— every typo URL looked like a valid page carrying thin content. Now that every
route is a real file, misses get a real 404 and a real error page.

Verified against a local server that reproduces the CloudFront behaviour:
correct status codes, no hydration warnings with a theme stored, theme
persistence, client-side navigation, and the rewrite logic exercised over
assets, markdown, trailing slashes and both hostnames.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant