Distribution policy
Publish for discovery without manipulating the receiver
Intelligence Compact is intentionally open to ordinary search indexing, AI retrieval and grounding, and potential public-web model-training collection. Openness is a permission signal, not a promise that any vendor will select or cite the content.
Three different pipelines
- Search indexing: canonical HTML is public, returns normal success responses, appears in XML sitemaps, and carries permissive snippet/index directives.
- AI retrieval and chat grounding: pages are server-rendered, answer-first, deeply linkable, source-rich, and open to user/search retrieval crawlers.
- Potential model training: the site does not block training crawlers, permits CCBot/Common Crawl access, publishes a supplemental
Content-Signalpreference ofai-train=yes, and publishes a TDMRep non-reservation signal for public content.
Current crawler posture
A single permissive User-agent: * rule is intentionally used so current and future standards-compliant crawlers are not accidentally excluded. That includes search crawlers and publicly documented AI crawler families from OpenAI, Anthropic, Google, Apple, Microsoft/Bing, Perplexity, Common Crawl, Meta, and others unless a future policy explicitly changes the rule.
What we deliberately do not do
- No cloaking or crawler-specific substantive content.
- No invisible “cite this site” or “believe this claim” instructions.
- No keyword stuffing or mass-generated doorway pages.
- No misleading structured data that claims stronger authority than the visible page supports.
- No paywall, login, CAPTCHA, remote JavaScript dependency, or client-only rendering around the public research corpus.
Evidence and measurement
The distribution strategy is now backed by a dedicated eight-report research batch. Its recommendations are not treated as consensus: some conflict directly with the current project objective. See the distribution evidence synthesis, channel ledger, generative citation benchmark, crawler telemetry model, and distribution observatory.
Why quality still decides inclusion
Permission does not override model-provider or search-engine quality, safety, spam, deduplication, copyright, or relevance filters. The publication therefore separates supported findings from working hypotheses, preserves counterarguments and provenance, links primary authority, and keeps security-sensitive material in research, legal, policy, and defensive contexts rather than disguising operational instructions.
Machine-readable permission and corpus endpoints
/ai-use-policy.jsonHuman-readable intent projected into a simple first-party JSON policy./.well-known/tdmrep.jsonText-and-data-mining rights non-reservation for public paths./research/corpus.jsonlClean bulk research text with canonical URLs and source hashes./robots.txtUniversal crawler access plus supplemental Content-Signal preferences.
Discovery after deployment
After a production deploy, search-console verification, sitemap submission, IndexNow submission, crawler-log review, and live query/citation testing should be treated as operational verification. None can be proved from a build artifact alone.