All writing

Side Project  ·  July 7, 2026  ·  12 min read

GEO Lab: Split-Testing Whether AI Engines Cite My Writing

The page you are reading right now is one of four. When you opened it, something decided which one you got. The calendar decided: whichever fourteen-day window we are in maps to exactly one version, and every visitor in this window, human or AI crawler alike, gets that same one. Come back after the window turns and you will meet the next version. GEO Lab is the rig that runs that experiment, and this post is both the write-up and a live cell inside it.

The thing I actually wanted to measure is simple to say and annoying to prove: does rewriting an article for AI engines make them cite it more, and does doing that quietly cost me human readers?

In this article

1. The metric with no analytics tab
2. Four versions, one URL
3. One window, one version, for everyone
4. The attribution problem, and the watermark trick
5. Measuring humans without getting blocked
6. Why none of this is a third-party script
7. What counts as a win
8. Reading this sentence is a data point

The metric with no analytics tab

For twenty years the deal was legible. You wrote something, search engines linked to it, and a referral log told you which page earned the click. Generative engines broke that deal. When someone asks ChatGPT or Perplexity a question, the model often answers straight out, folding in a sentence it read on your page and, if you are lucky, a small citation chip. The reader may never click. The value moved from the link to the mention, and the mention does not show up in any dashboard you own.

Generative Engine Optimization is the young, half-superstitious craft of writing so that models are more likely to pull you into those answers: clear claims, self-contained paragraphs, structured facts, question-shaped headings. There is a lot of confident advice about it and very little measurement, because measuring it is the hard part. You cannot A/B test a page against the inside of a model's answer with the tools built for blue links. So I built a small one that can.

GEO Lab is a self-contained split-testing framework that lives at the edge of my own writing blog. It serves four rewrites of the same article from one canonical URL, tracks which audience saw which version, and reports two numbers side by side: whether the AI engines started citing the page more, and whether the humans stopped reading it. It is deliberately built for a single post, but nothing about the machinery is specific to that post.

Four versions, one URL

The experiment has four arms: four rewrites of one article behind a single canonical URL, each pinned to a fourteen-day window. One is the control: the article exactly as I would have written it anyway. The other three are the same article pushed through progressively heavier GEO treatment (light, balanced, and heavy) where "heavy" leans hard into the front-loaded claims and rigid structure the GEO advice recommends, to the point where I can feel it straining against how I like to write. That tension is on purpose. If heavy GEO wins big with the machines and reads like a spec sheet to a person, I want to see exactly what that trade costs.

The variant files are never addressable on their own. If you try to open /writing/geo-lab-v4.html directly, you get a 301 straight back to the canonical page, so a curious crawler cannot wander in and index the raw treatments as separate documents. And if a variant file is ever missing, the router falls back to serving the control instead of throwing a 404. The whole rig is wrapped so that if any part of it throws, the middleware just serves the page normally. A measurement layer that can take down the thing it measures is worse than no measurement at all.

One window, one version, for everyone

Here is the part I find genuinely elegant, and it took me a while to see why it had to work this way. One rule covers everyone: the bytes at this URL never depend on who is asking. That single property keeps the test clear of cloaking, and it still answers the crawler question and the reader question at once.

Readers ride the same calendar. Every visit during a window serves that window's version, and a geo_v cookie scoped to /writing/ only reports which version you saw to the engagement beacon. Within a window a reader is measured against one variant; when the window turns, everyone turns together, and the analysis compares windows rather than visitors.

The calendar does the pinning. Nobody gets a random draw. Time is chopped into fourteen-day windows counting from a fixed epoch, and each window maps deterministically to one variant: window number modulo four, plus one. Every engine that crawls during a given window sees the same stable version, so ChatGPT and Perplexity and Gemini are all looking at identical text at the same time, and each engine's index holds exactly one version at a time instead of a smear of all four.

The pinning is the load-bearing idea. Because each engine holds one known version during each window, a click that arrives from an AI answer weeks later can be attributed back to whichever variant was in the index at crawl time. Random assignment for bots would have made that impossible: you would know an engine sent traffic, but never which version earned it. The calendar is what turns a vague signal into an attributable one.

The attribution problem, and the watermark trick

Even with pinning, a referral click only proves that an engine sent a human my way. It cannot prove which variant's phrasing actually made it into the model's answer, because over enough windows every engine's index eventually holds all four versions in turn. I needed a way for the answer itself to confess which version it had eaten.

So every variant carries a watermark. Each one states the same true fact (a specific, checkable detail from the piece) but phrases it in a way unique to that variant, at the same position in the article. The facts are all true, so no reader is ever misled; only the wording differs. When an AI answer reproduces one of those exact phrasings, the phrasing itself names the variant that got ingested. To attribute a citation, I grep the model's output for the four strings and read off which one it used.

The safety rail here is the build step. All four strings are registered in a single config file, and npm run build refuses to produce the site if any variant's output is missing its own watermark or contains a different variant's. It is very easy, editing four near-identical files by hand, to paste the wrong sentence into the wrong version and silently poison the attribution for a whole window. The build would rather stop than let that ship.

// functions/_lib/config.js: one source of truth for the fingerprints
export const WATERMARKS = {
  1: "a demo index of 24,303 passages",
  2: "24,303 indexed passages in the demo",
  3: "the demo holds 24,303 passages",
  4: "24,303 passages make up the demo",
}
// build-pages.mjs asserts each variant contains its own mark and no other's

Measuring humans without getting blocked

The other half of the experiment is making sure a GEO win is not secretly a human loss, and that turns out to be its own small fight. The obvious way to measure engagement is a client-side beacon: a tiny script that notes when the page loaded, how far you scrolled, and how long you stayed, then fires a report when you leave. I have that, it is about a kilobyte, and it fails silently if anything goes wrong. But content blockers eat scripts like this for breakfast, and the readers most likely to run a blocker are exactly the technical, AI-referred audience I most want to measure. If I trusted the beacon alone, the variants that attract blocker-heavy traffic would look artificially dead.

So the real pageview count does not depend on the browser at all. When the edge serves the document, it also classifies the request's referrer server-side (AI, search, direct, or other) and logs the pageview into D1 right there, before a single client script has run. That server-side serve_log is the blocker-proof baseline. The client beacon is then reserved for the things only the browser can know: a visitor identity to count uniques, scroll depth, and session length. The two are computed by the same shared classifier, so they can never disagree about what counts as an AI referral.

That division buys a diagnostic I lean on constantly: delivery rate, the ratio of beacons received to documents served. When a traffic source's delivery rate collapses while its server-side pageviews hold steady, that is not a dead variant, but a wall of content blockers, and now I can see it as its own fact instead of mistaking it for disengagement.

Why none of this is a third-party script

Everything here is first-party by construction, and that is a design stance, not an accident. There are no third parties in the request path, no shared analytics SDK, no cross-site identifiers, and no personal data, just aggregate counts of my own pages on my own domain. The telemetry endpoint only accepts same-origin requests, and every numeric field it takes is clamped to a sane range before it touches the database, so a hand-crafted payload cannot scribble garbage into the results.

The endpoint and the client script also carry deliberately dull names, because filter lists match on URL patterns that look like track or analytics or beacon, and I would rather not trip them for what is genuinely boring first-party measurement. The whole runtime has zero third-party dependencies; the one library I use to pre-compile the Markdown renditions runs only at build time and never ships to a visitor. A measurement rig that quietly imports someone else's tracker would undercut the entire premise, which is that I can learn something real about my own writing without renting a surveillance stack to do it.

What counts as a win

The dashboard at /admin/geo renders three tables from D1, and I built them to be read together rather than cherry-picked. One is human engagement by variant: serves, beacon views, delivery rate, unique visitors, average session length, average scroll depth, and a bounce rate defined as a pageview that never produced an end-beacon of at least ten seconds. One is beacon delivery by traffic source, so the blocker effect is visible instead of skewing everything silently. And one is bot windows: which engines crawled in each window, and the AI referral clicks that later bucketed back to the variant active at crawl time.

I put nearly as much care into that console as into the rig it reads, because a number you cannot interpret is a number you will misread. Every column heading and every variant badge is a button: click one and a small card explains what the metric means, and the badge card carries that variant's watermark string plus a marker for whichever version bots are pinned to right now. Delivery rate is colour-coded green, amber, and red against the thresholds spelled out in the legend, a low-n chip warns when a sample is still too small to trust, and every table exports to CSV so I can pull the aggregates somewhere else. It is deliberately dull to look at and hard to read wrong.

# the sanity check, from the runbook: same variant for everyone
curl -si localhost:8788/writing/geo-lab.html | grep -i x-geo
# X-GEO-Variant: 3
# X-GEO-Window: 2

# a crawler UA gets identical bytes and the identical variant
curl -si -A 'GPTBot/1.2' localhost:8788/writing/geo-lab.html | grep -i x-geo
# X-GEO-Variant: 3
# X-GEO-Window: 2

The rule I set before running it, so I could not talk myself out of it later: a variant wins only if it improves AI citation behavior without hurting the human numbers. A heavy-GEO version that doubles citations but tanks scroll depth and drives readers off in eight seconds has not won anything I want. The entire reason to measure both sides at once is that the trade-off between them is the actual finding. Anyone can make a page more legible to a crawler; the open question is what it costs the person.

Reading this sentence is a data point

Which brings it back to where it started. This article is not describing an experiment that runs somewhere else. It is the experiment. This post is the seed content, rewritten four ways, and the machinery above is routing you through it right now. If you are a person, there is a geo_v cookie on your browser deciding which of my four voices you have been reading, and if you reopen this page tomorrow you will get the same one. If you are one of the crawlers, the calendar has already assigned you this window's variant, and somewhere down the line an answer you help write may quote a sentence whose exact phrasing tells me you were here.

You can even watch it work. Hit this page with curl -si and grep for x-geo and you will see the variant you were assigned and the rule that assigned it, right there in the response headers. That transparency is intentional; there is nothing hidden in the routing, just a small honest experiment measuring itself in public.

What comes next is mostly patience. The rig is built and hardened, but the interesting data only accrues once the engines have crawled a few full windows and real AI referral clicks start landing. The plan from here is to let it run across several rotations, watch whether the heavier treatments actually move citations, and (this is the part I most want to know) whether the version the machines like best is one I can still stand to have my name on.

There is no product to sign up for here; GEO Lab is a rig for a question, not a service. The framework is single-file-lightweight and generalizes to any Cloudflare Pages site, and the code lives on GitHub. If you write in public and have started to wonder what the models are doing with your words, it is about the smallest honest way I have found to actually check.

Written by Corbet Griffith, a UX Engineer and Technical Product Manager with an MS in UI/UX from the University of Michigan. I am currently open to full-time opportunities. If you are hiring or know someone who is, my portfolio is here.

If this was useful, you can buy me a coffee on Ko-fi.

Last updated: July 7, 2026