AGENT69MARKETS

SEO & AI Discoverability Runbook

This runbook explains how metadata, sitemaps, robots controls, and AI discovery are managed.

Metadata

  • Default metadata lives in `frontend/app/layout.tsx`.
  • Page-specific metadata uses `buildMetadata` from `frontend/app/seo.ts`.
  • Update canonical URLs by changing `NEXT_PUBLIC_SITE_URL`.
  • Private pages set `indexable: false` in their route layouts.

Sitemaps

  • `/sitemap_index.xml` lists static and markets sitemaps.
  • `/sitemaps/static.xml` is a curated list of public pages.
  • `/sitemaps/markets.xml` is generated from `/api/markets`.

Robots & AI bots

  • Robots policy is generated by `frontend/app/robots.ts`.
  • Use env toggles to allow or deny AI crawlers.

LLMs guidance

  • `/llms.txt` provides curated links and summaries.
  • `/llms-full.txt` provides a longer reference.
  • Markdown mirrors are available at `/ai.md` and `/docs/seo-ai-discoverability.md`.

Environment variables

  • `NEXT_PUBLIC_SITE_URL` and `NEXT_PUBLIC_SITE_ENV`
  • `NEXT_PUBLIC_OG_IMAGE_URL`
  • `NEXT_PUBLIC_GOOGLE_SITE_VERIFICATION`, `NEXT_PUBLIC_BING_SITE_VERIFICATION`
  • `NEXT_PUBLIC_GA4_ID` (optional analytics)
  • `NEXT_PUBLIC_TWITTER_URL`, `NEXT_PUBLIC_GITHUB_URL`, `NEXT_PUBLIC_SUPPORT_EMAIL`
  • `SEO_ALLOW_OPENAI_SEARCH`, `SEO_ALLOW_OPENAI_TRAINING`, `SEO_ALLOW_PERPLEXITY_SEARCH`

Verification

  • Set `NEXT_PUBLIC_GOOGLE_SITE_VERIFICATION` for Google Search Console.
  • Set `NEXT_PUBLIC_BING_SITE_VERIFICATION` for Bing Webmaster Tools.

Bot policy toggles

  • `SEO_ALLOW_OPENAI_SEARCH`
  • `SEO_ALLOW_OPENAI_TRAINING`
  • `SEO_ALLOW_PERPLEXITY_SEARCH`

Indexing pings

  • Run `npm run seo:submit` from `frontend/` to ping sitemaps.
  • Optional IndexNow: set `INDEXNOW_KEY` and `INDEXNOW_KEY_LOCATION`.
  • Submit URLs via `SEO_SUBMIT_URLS` or CLI args (space-separated URLs).

CI checks

  • `npm test` runs SEO metadata + sitemap/robots coverage tests.
  • `npm run lhci:ci` runs Lighthouse CI with SEO thresholds.

Validation

  • Check `/robots.txt`, `/sitemap_index.xml`, `/llms.txt` in production.
  • Validate structured data via Google Rich Results Test.
  • Monitor Core Web Vitals in Search Console.