Publishing Ahead of the Guidelines, Not the Evidence

Any organisation publishing into a regulated or reputationally sensitive space eventually builds an evidence threshold: a rule stating what has to be true before a claim can go out under its name.

Ours required convergence — independent studies, from independent groups, pointing the same way. A claim that cleared it could be published. A claim that did not was refused, and the pipeline would not produce the piece.

The rule worked. It also produced a growing pile of topics nobody could touch, and when we finally measured that pile it turned out to be the more interesting artefact.

What the refusals actually were

Across 310 topics assessed, 66 held real primary research that sat below the bar. Not speculation, not press releases — actual studies, refused because the evidence had not converged.

Twenty-four of those sat at exactly two studies from two independent groups. One group short.

That number is worth sitting with, because it describes a specific and uncomfortable position. There is genuine research. It points somewhere. It is not enough to assert the finding. Under a binary rule, a topic in that state is indistinguishable from a topic with nothing behind it at all — both are simply refused — and the organisation ends up silent on exactly the questions its audience is actively asking, while less careful publishers answer them badly.

The instinct at that point is to lower the threshold. Two groups instead of three, with a hedge in the title.

That instinct is wrong, and understanding why is the whole point.

A lower bar on the same claim, versus a different claim

“This works” supported by two studies is an overclaim. Adding “may” or “could” to the front of it does not repair the evidence; it makes the sentence harder to falsify while leaving the reader with the same impression.

But consider a genuinely different assertion:

Two small trials suggest X, nobody has replicated it, and here is what would settle the question.

That is not a weaker version of “X works”. It is a separate claim about the state of the evidence, and it is fully supported by exactly the evidence available. Two studies is not marginal support for that sentence. It is complete support, because the sentence’s content is that there are two studies and no replication.

Presented honestly, it may be the most trustworthy thing a publisher in this position can produce. It tells the reader precisely where the question stands, which is more useful than either silence or a hedged assertion.

So we built a second route with that shape, and the design decisions in it are where the value is.

★ Insight ───────────────────────────────────── The reframe generalises well beyond research claims. Any time a threshold is blocking something you believe is publishable, there are two possible fixes and only one is safe. Lowering the bar keeps the claim and weakens the evidence required to support it. Changing the claim keeps the evidence standard and states something the evidence actually establishes. The first is a governance failure wearing a hedge; the second is usually the more honest and more useful piece. ─────────────────────────────────────────────────

The counting relaxed. The language got stricter.

This is the part most people get backwards. A route for less settled material is intuitively a looser route. Ours is looser in exactly one dimension and considerably tighter in several others.

What relaxed: two studies from two independent groups, and disagreement between them is permitted — because on an unsettled question, disagreement is the story rather than a defect in the evidence.

What tightened:

• Efficacy language is refused outright. Improves, reduces, works, effective, helps you. An efficacy verb directly contradicts the standard the claim is being admitted under, so it is banned rather than discouraged.
• Guideline language stays refused, as on the stricter route. Nothing here may imply official endorsement.
• The spoken text must state that the question is unresolved. Not the title — the body. A title cannot carry this, because in our format the narrator does not read the title and the reader does not remember it. Putting the qualification somewhere the audience does not encounter it is the oldest trick in hedged publishing, and the rule exists specifically to close it.
• The claim must name what would settle it. Without that, the piece is a shrug. With it, the reader learns how the question resolves, which is the only defensible reason to publish an unresolved one.

That last requirement did real work immediately. The check for it originally matched any mention of replication — which meant a sentence like “nobody has replicated either” satisfied it. But that sentence describes the current state, which was already required elsewhere; it says nothing about what would resolve the question. The check was narrowed to forward-looking forms only. A shrug no longer counts as a plan.

Declared in advance, never fallen into

The single most important constraint: a piece enters this route only if the author declares it in advance, and the declaration is checked before the stricter route runs.

A claim that merely fails the convergence test is never demoted here.

If failure auto-demoted, every blocked claim would reappear under a hedged framing. The route would stop being an honest way to cover open questions and become a laundering mechanism — which, in a regulated context, is materially worse than the original overclaim, because it pairs an unsupported impression with a disclaimer that makes it look considered.

The test that protects this is the one we care about most: take a claim that passes in the declared route, remove the declaration, and confirm it is judged under the strict rule and fails. The route must be impossible to arrive at by accident.

The failure that taught us where a rule belongs

The first real piece through the new route was blocked entirely — seven claims out of seven — and the gate was correct by the letter of the rule and wrong in substance.

One of the rejected claims was, in effect: what would settle this is a properly designed trial with a pre-specified primary outcome, and one such protocol is registered.

That claim is the thing the rule was asking for. It was refused for not containing it.

The error was in where the requirement sat. “Says it is unresolved” and “says what would settle it” had been written as per-claim requirements. They are properties of the piece as a whole. Enforced per claim, every single sentence has to restate “but this is unsettled, and a trial would settle it” — which produces a chant, not a review, and lands in precisely the hedged-boilerplate register the route exists to avoid.

So the rules were split by the level they actually belong to:

Level	Rule	Why
Per claim	No efficacy verbs, no guideline language, direction asserted per source, two studies, two independent groups	Properties of an individual assertion. These are what prevent overclaiming, and a violation anywhere is a violation.
Whole piece	Says the question is unresolved; says what would settle it	Properties of the narration. Satisfied somewhere, not everywhere.

Moving them cost no safety, and it is worth being precise about why: a piece-level pass cannot launder a confident body, because every sentence in that body is still individually checked for efficacy and guideline language. The per-claim bans that stop overclaiming were untouched.

What this route costs

It is more expensive to produce, not less, and anyone adopting the idea should budget for that.

A confident piece can be written from a conclusion. A piece about an open question has to describe the shape of the evidence — how many studies, by whom, agreeing or not, and what specifically remains unresolved. That is more research, and it cannot be done by someone who has not read the sources, because the honest description of a body of evidence is not recoverable from its abstracts.

It is also harder to write well. “Two trials suggest X” is a less satisfying sentence than “X works”, and holding a reader through the second kind of piece takes better writing than the first. The naming-what-would-settle-it requirement helps here more than it costs: it gives the piece somewhere to go, and it ends on something concrete rather than a shrug.

The payoff is not volume. It is that the pieces which do go out under the confident route are more believable, because the publisher has demonstrated it distinguishes between the two — and a reader who has seen you say “this is unresolved” has a reason to trust you when you say it is not.

The question worth taking away

If your organisation has a publication threshold, it is worth asking what is accumulating behind it.

Some of it is correctly refused and always will be. But some proportion is material where the honest description of the evidence is itself a publishable claim — and a binary threshold cannot tell those two apart, because it only ever asks whether the evidence supports the assertion in front of it. It never asks whether a different assertion is available that the same evidence fully supports.

Three things that make this concrete:

1. Count the pile. Not “we refuse some things” — how many, and how close were they? Ours was 66, with 24 of them one group short. Until it was counted it was a feeling.
2. Check where your qualifications live. If the hedge is in the title, the subtitle or a footer, ask whether your audience actually encounters it. If not, it is decoration.
3. Make the honest route harder to write, not easier. If the less-settled lane has the looser language rules, you have built the laundering mechanism. The looseness belongs in the evidence count; the strictness belongs in the words.

---

Editorial governance, evidence thresholds and the automated gates that enforce them are part of the AI strategy work we do through Ganda Tech Services, with content operations through Cosmos Web Tech.