My privacy policy was accurate for nine hours. A commit that evening changed one line inside that page and left the paragraph below it saying the opposite. Eleven assertions covered the page, and all eleven kept passing for the next 78 days.
The page explained how this site handles analytics. Google Consent Mode v2 takes a set of defaults before any measurement happens, and the policy described them in prose: analytics defaulted to denied, nothing was stored until you granted consent, and a consent message decided the rest.
That was a fair description of 59e236c, committed at 09:11 on 21 May 2026.
The commit that falsified it edited the same file
By that evening the analytics dashboard was showing "No data received". Consent Mode had defaulted analytics_storage to denied and nothing ever granted it, so GA4 was sending cookieless pings and recording nothing. The fix was one word.
b51310b 2026-05-21 18:16:52 -0300
- analytics_storage: 'denied'
+ analytics_storage: 'granted'
That commit touched privacy/index.html. It changed exactly one line of it, +1 -1, because the consent script is inlined into every page's <head> and the privacy page is a page like any other. Two hundred lines further down the same file, the prose still told readers that analytics defaulted to denied.
Nine hours and five minutes separated the sentence from the commit that made it false. The commit rebuilt the page carrying the sentence and did not touch a word of it.
Eleven green assertions over a false page
The page was not untested. tests/legal-pages.test.mjs had covered it since the policy went in, and it is not a lazy test by the usual standards. It renders the component, walks the footer, and checks eleven things.
page contains "Privacy" passing
page contains "Google AdSense" passing
page contains "cookies" passing
links to the contact page passing
links to Google Ads Settings passing
links to aboutads.info/choices passing
...
assertions referencing a consent value 0
Every assertion was about presence. Does the page mention cookies? It did. Does it link to the opt-out controls? It did. Nothing in the file compared a single word of the policy against the configuration the policy was describing, because the configuration lives in getConsentBootstrapScript() in scripts/generate-static-pages.mjs and the test never reads it.
So the suite was answering a question I had not thought to distinguish from the one I cared about. I wanted to know the page was correct. I had written a test that knew the page was on topic.
I have now made this mistake twice
The second time is what convinced me it is a shape rather than an oversight.
Earlier this month a body diagram on this site shipped a 114 KB PNG while a 31 KB WebP sat unused on disk beside it. The build had generated four WebP variants and the published markup referenced none of them. That defect is written up in full, but the part that matters here is the test that failed to catch it: broken-images.test.mjs walks every src and srcset in the generated HTML and confirms the file exists.
It does exactly what it says. The file existed. The page served the wrong one.
Different subsystems, different file types, same structure. An assertion that something is present passes trivially on a page where the present thing is the wrong thing. Prose that mentions cookies is not prose that describes your cookies. An image file that exists is not the image file being served.
The guard reads the config
The fix for the policy was to rewrite it against what the code actually does: analytics storage granted, ad_storage, ad_user_data and ad_personalization all denied, no consent message shown to anyone.
Rewriting it is not a guard, though. The same rewrite could go stale on the next config change, in nine hours or nine weeks, and I would find out the same way.
So tests/privacy-consent-claims.test.mjs parses the Consent Mode defaults out of getConsentBootstrapScript() and holds the prose to them. It does not hardcode which state is correct. It reads what shipped and branches:
analytics_storage granted -> page must say analytics storage is enabled by default
-> page must not say "defaults to a denied state"
ad_* all denied -> page must say "non-personalized"
-> page must not promise a consent message
Flip the config back to denied and the test demands the opposite prose. The correspondence is the assertion.
I checked that it fails. I put the 21 May wording back into the page, ran the file, and watched it go red before restoring the correction and watching it pass. A guard written after the fact and never seen failing is a guard you are trusting on faith. The suite went from 147 tests to 149.
What I am not claiming
Two defects in one small repository is a pattern worth watching, not a law. I do not know how far this generalizes, and I have not gone looking through the rest of the suite for a third instance.
What I would defend is narrower. When an assertion checks that something is present, write down what it would still pass on. If the answer is "a page that says the opposite of the truth" or "a file nobody serves", then the test is measuring topic coverage and you are reading it as correctness. The gap between those two is invisible right up until someone reads the page.