IndexNow — re-indexing when you add pages (and when it fails)
- template/scripts/indexnow.mjs
- per-site key files live on all 5 domains
- 22/22 unit tests
What you're doing and why
When your site is new — or when you add a page later — you can tell Bing (and the other engines that listen to IndexNow) "hey, this URL changed, come crawl it" instead of waiting weeks to be discovered. You prove you own the site by hosting a secret key file on your domain, then you ping the IndexNow API with your URLs. The important lesson: this is a re-runnable tool, not a one-time setup step. Every time you add a page, you run it again. And if it fails, your publish must NOT fail — indexing is a nice-to-have, the site going live is the must-have.
The exact steps
- Each site needs its own key. It's already in your site's
site.config.jsonunderindexnow.key(a random UUID per site — never reuse one key across sites). - When you publish or add a page, the build engine drops a
<key>.txtfile at the site root (sohttps://yourdomain.com/<key>.txtanswers with the key) and runs:node scripts/indexnow.mjsfrom the template directory. - To re-index after adding pages later, just run the same command again — it reads the sitemap and submits everything that's new or changed.
- If Bing is having a bad day, the script retries with backoff and then gives up quietly — your publish still succeeds. Re-run it tomorrow; nothing is lost.
What it looked like for us
- All 5 live sites now serve their own key file (verified over HTTPS: each domain answers its
UUID at
/<uuid>.txt). - The script was hardened with retry/backoff and covered by 22 unit tests (all passing) before we trusted it anywhere near a publish.
- We have NOT fired real submissions yet on purpose: our 5 sites still have placeholder business details pending the truth audit, and indexing a site with fake details is how you teach Google to distrust you. The tool is ready; the trigger waits for real data.
Gotchas
- One key per site. Early version had a single shared key in the template — that leaks your whole portfolio footprint (one engine sees all your sites share a key). Per-site UUIDs fixed it.
- Don't let indexing block shipping. The first version could fail a publish if the API hiccuped. Now it fails safe and logs a warning instead.
- A 200 from the API ≠ indexed. IndexNow is a hint, not a guarantee. You still confirm
indexing by searching
site:yourdomain.coma week later. - Adding pages = re-running. It's not set-and-forget; bake the re-run into your "add a page" flow so new pages get discovered in days, not months.