# AIVory Robots.txt

User-agent: *
Allow: /

# Disallow markdown versions (served only to AI agents via Cloudflare Worker)
Disallow: /md/

# Disallow internal product pages (not yet public)
Disallow: /monitor/

# Allow common crawlers for standard pages
Allow: /docs/
Allow: /support/
Allow: /about/

# Tracking-parameter URLs (?ref=, ?utm_) are deliberately NOT blocked here.
# Every page serves <link rel="canonical"> pointing at its parameter-free URL.
# A Disallow rule would stop Google from crawling those URLs, so it would never
# read the canonical tag and could not fold them into the canonical page. The
# duplicates would stay in the index as URL-only entries. Leave them crawlable.

# Sitemap
Sitemap: https://aivory.net/sitemap.xml