Skip to content

feat: agent discovery (Link headers, markdown negotiation, Content-Signal) #263

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
sriramveeraghanta merged 8 commits into from
Apr 21, 2026
Merged

feat: agent discovery (Link headers, markdown negotiation, Content-Signal) #263

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
ff151bf
docs(spec): add agent discovery design
sriramveeraghanta Apr 20, 2026
2bf43d2
docs(plan): add agent discovery implementation plan
sriramveeraghanta Apr 20, 2026
9060762
feat(robots): declare Content-Signal AI usage preferences
sriramveeraghanta Apr 20, 2026
9df7666
feat(vercel): add Link response headers for agent discovery
sriramveeraghanta Apr 20, 2026
ee5d04e
feat(build): copy source .md files to dist for agent markdown negotia…
sriramveeraghanta Apr 20, 2026
9666309
feat(vercel): rewrite to .md when Accept: text/markdown
sriramveeraghanta Apr 20, 2026
c4158a3
style: format with prettier
sriramveeraghanta Apr 20, 2026
2c5fbf7
chore: remove ephemeral agent-discovery spec and plan
sriramveeraghanta Apr 20, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
27 changes: 25 additions & 2 deletions docs/.vitepress/config.mts
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
import { defineConfig, type HeadConfig } from "vitepress";
import { tabsMarkdownPlugin } from "vitepress-plugin-tabs";
import { withMermaid } from "vitepress-plugin-mermaid";
import { readFileSync } from "node:fs";
import { resolve } from "node:path";
import { readFileSync, readdirSync, statSync, mkdirSync, copyFileSync } from "node:fs";
import { resolve, join, relative, dirname } from "node:path";

function loadEnvVar(key: string): string | undefined {
// process.env takes precedence (CI/hosting platforms set vars here)
Expand Down Expand Up @@ -75,6 +75,29 @@ export default withMermaid(
],
},
},
buildEnd(siteConfig) {
// Copy source .md files into dist/ for Accept: text/markdown negotiation.
const srcDir = siteConfig.srcDir;
const outDir = siteConfig.outDir;

function walk(dir: string): void {
for (const entry of readdirSync(dir)) {
if (entry === ".vitepress" || entry === "public" || entry === "node_modules") continue;
const abs = join(dir, entry);
const stat = statSync(abs);
if (stat.isDirectory()) {
walk(abs);
} else if (stat.isFile() && abs.endsWith(".md")) {
const rel = relative(srcDir, abs);
const dest = join(outDir, rel);
mkdirSync(dirname(dest), { recursive: true });
copyFileSync(abs, dest);
}
}
}

walk(srcDir);
},
title: "Plane developer documentation",
description:
"Self-host Plane, integrate with our API, configure webhooks, and extend your project management platform. Complete guides for developers building on Plane.",
Expand Down
4 changes: 4 additions & 0 deletions docs/public/robots.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,10 @@ Allow: /
# Disallow crawling of search results (if any)
Disallow: /search

# Content Signals — AI content usage preferences
# https://contentsignals.org/
Content-Signal: search=yes, ai-train=yes, ai-input=yes

# Disallow crawling of any internal/private paths (add as needed)
# Disallow: /private/
# Disallow: /admin/
Expand Down
18 changes: 18 additions & 0 deletions vercel.json
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,16 @@
{
"cleanUrls": true,
"headers": [
{
"source": "/(.*)",
"headers": [
{
"key": "Link",
"value": "</llms.txt>; rel=\"describedby\"; type=\"text/plain\", </api-reference/introduction>; rel=\"service-doc\"; type=\"text/html\", </sitemap.xml>; rel=\"sitemap\"; type=\"application/xml\""
}
]
}
],
Comment on lines +3 to +13
Copy link
Copy Markdown

@coderabbitai coderabbitai Bot Apr 20, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
set -euo pipefail

: "${PREVIEW_URL:?Set PREVIEW_URL to the Vercel preview origin, e.g. https://example.vercel.app}"

curl -fsSI "$PREVIEW_URL/api-reference/introduction" | tr -d '\r' | grep -i '^vary:.*accept'
curl -fsSI -H 'Accept: text/markdown' "$PREVIEW_URL/api-reference/introduction" | tr -d '\r' | grep -i '^vary:.*accept'

Repository: makeplane/developer-docs

Length of output: 177

🏁 Script executed:

git ls-files | grep -i vercel

Repository: makeplane/developer-docs

Length of output: 79

🏁 Script executed:

cat -n vercel.json | head -120

Repository: makeplane/developer-docs

Length of output: 4281

Add Vary: Accept response header for content negotiated Markdown responses.

The rewrite rule at lines 108–113 serves different content (HTML vs. Markdown) based on the Accept header. Without Vary: Accept, caches may serve the wrong representation to clients requesting a different format.

Proposed fix 
       "headers": [
         {
           "key": "Link",
           "value": "</llms.txt>; rel=\"describedby\"; type=\"text/plain\", </api-reference/introduction>; rel=\"service-doc\"; type=\"text/html\", </sitemap.xml>; rel=\"sitemap\"; type=\"application/xml\""
+        },
+        {
+          "key": "Vary",
+          "value": "Accept"
         }
       ]
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
"headers": [
{
"source": "/(.*)",
"headers": [
{
"key": "Link",
"value": "</llms.txt>; rel=\"describedby\"; type=\"text/plain\", </api-reference/introduction>; rel=\"service-doc\"; type=\"text/html\", </sitemap.xml>; rel=\"sitemap\"; type=\"application/xml\""
}
]
}
],
"headers": [
{
"source": "/(.*)",
"headers": [
{
"key": "Link",
"value": "</llms.txt>; rel=\"describedby\"; type=\"text/plain\", </api-reference/introduction>; rel=\"service-doc\"; type=\"text/html\", </sitemap.xml>; rel=\"sitemap\"; type=\"application/xml\""
},
{
"key": "Vary",
"value": "Accept"
}
]
}
],
🤖 Prompt for AI Agents 
Verify each finding against the current code and only fix it if needed.

In `@vercel.json` around lines 3 - 13, The Link response header rule for the
catch‑all source "/(.*)" needs to also emit a Vary: Accept header so caches
distinguish HTML vs Markdown responses; update the headers array entry for
source "/(.*)" (the object containing the Link header) to include another header
with key "Vary" and value "Accept" alongside the existing Link header so caches
will vary by the Accept request header.

"redirects": [
{
"source": "/api-reference",
Expand Down Expand Up @@ -93,5 +104,12 @@
"source": "/self-hosting",
"destination": "/self-hosting/overview"
}
],
"rewrites": [
{
"source": "/:path*",
"has": [{ "type": "header", "key": "accept", "value": ".*text/markdown.*" }],
"destination": "/:path*.md"
}
Comment on lines +108 to +113
Copy link
Copy Markdown

@coderabbitai coderabbitai Bot Apr 20, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
set -euo pipefail

: "${PREVIEW_URL:?Set PREVIEW_URL to the Vercel preview origin, e.g. https://example.vercel.app}"

curl -fsSI -H 'Accept: text/markdown' "$PREVIEW_URL/" | tr -d '\r' | grep -Ei '^HTTP/|^content-type:'
curl -fsS -H 'Accept: text/markdown' "$PREVIEW_URL/" | head -n 20

Repository: makeplane/developer-docs

Length of output: 177

🏁 Script executed:

cat -n vercel.json | sed -n '100,120p'

Repository: makeplane/developer-docs

Length of output: 566

🏁 Script executed:

# Search for all rewrite rules in vercel.json to understand the full context
rg -A2 '"source"' vercel.json | head -60

Repository: makeplane/developer-docs

Length of output: 1888

🏁 Script executed:

# Check if there's any rewrite rule for the root path "/" already
rg '"source".*"/"' vercel.json

Repository: makeplane/developer-docs

Length of output: 50

Add explicit root rewrite for Markdown content negotiation.

The catch-all rule /:path* → /:path*.md fails for the root path / because the path variable captures an empty string, resulting in /.md instead of /index.md. Add an explicit root rewrite before the catch-all to handle requests to / with Accept: text/markdown header:

Proposed fix 
   "rewrites": [
+    {
+      "source": "/",
+      "has": [{ "type": "header", "key": "accept", "value": ".*text/markdown.*" }],
+      "destination": "/index.md"
+    },
     {
       "source": "/:path*",
       "has": [{ "type": "header", "key": "accept", "value": ".*text/markdown.*" }],
       "destination": "/:path*.md"
     }
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
"rewrites": [
{
"source": "/:path*",
"has": [{ "type": "header", "key": "accept", "value": ".*text/markdown.*" }],
"destination": "/:path*.md"
}
"rewrites": [
{
"source": "/",
"has": [{ "type": "header", "key": "accept", "value": ".*text/markdown.*" }],
"destination": "/index.md"
},
{
"source": "/:path*",
"has": [{ "type": "header", "key": "accept", "value": ".*text/markdown.*" }],
"destination": "/:path*.md"
}
🤖 Prompt for AI Agents 
Verify each finding against the current code and only fix it if needed.

In `@vercel.json` around lines 108 - 113, Add an explicit root rewrite entry
before the existing catch-all rule in the "rewrites" array so requests to "/"
with Accept: text/markdown are rewritten to "/index.md"; specifically, add a
rule with "source": "/" and the same "has" header condition (key: "accept",
value matching ".*text/markdown.*") and "destination": "/index.md" placed above
the existing rule that maps "source": "/:path*" to "/:path*.md".

]
}
Loading