Skip to content

Research Methodology

This document codifies the rules the research-theory skill follows. The goal: produce reports that are epistemically honest, calibrated, and useful — not essays that either credulously repeat claims or reflexively dismiss them.

Source tiers

Stronger claims require higher-tier sources. A Tier 5 source can suggest a lead but cannot establish a fact.

Tier Description Examples
1 Primary documents Declassified records, court filings, FOIA releases, contemporaneous government reports, original photos/audio/video with verifiable provenance
2 Peer-reviewed academic Articles in reputable peer-reviewed journals; ideally not pay-to-play venues
3 Mainstream investigative journalism NYT, WaPo, Reuters, AP, BBC, ProPublica, Guardian investigative pieces. Investigative > opinion > commentary.
4 Advocacy / partisan media Outlets with a known editorial slant. Useful to understand what advocates argue; not sufficient to establish fact.
5 Social media / anonymous Tweets, anonymous posts, unverified leaks. Treat as leads only. Almost never sufficient on their own.

When citing a source, the report should note its tier (e.g. [3] NYT, 2017 (Tier 3)).

Framework: Steelman → Evidence → Verdict

  1. Steelman the theory. Write the strongest possible version using advocates' actual framing — not a strawman. Pull from advocates' own writings, interviews, speeches where possible. This forces engagement with the theory's strongest form before any evaluation.

  2. Decompose into atomic empirical claims. Distinguish factual claims ("X met Y on date Z") from interpretive ones ("This shows coordination"). Each claim is researched separately.

  3. Evaluate each claim against the source tiers. Seek primary sources first. Note where evidence converges and where it conflicts.

  4. Render a verdict per claim using calibrated vocabulary (below).

  5. Render an overall verdict by synthesizing across claims. The theory may be partially correct: some claims supported, others contradicted, others open.

Calibrated verdict vocabulary

Use these terms — and only these — for per-claim and overall verdicts:

  • Supported — strong evidence from high-tier sources confirms the claim
  • Partially true — kernel of truth, but the theory's framing overstates or distorts what's actually established
  • Contradicted — strong evidence from high-tier sources contradicts the claim
  • Unfalsifiable — claim is not testable as stated (e.g. "the conspirators covered their tracks so well no evidence exists")
  • Insufficient evidence — genuinely open question; evidence too thin to call

Calibrated confidence levels

Use these terms — and only these — for confidence:

  • High — multiple independent Tier 1–2 sources converge
  • Moderate — convergence among Tier 2–3 sources, or single strong Tier 1 source
  • Low — sources are mostly Tier 3–4, or there's significant disagreement among credible sources
  • Insufficient — too little to assign confidence

Hard rules

  • No false certainty. Avoid "definitely", "obviously", "everyone knows", "debunked" without citing the debunking.
  • No false balance. When one side has overwhelming evidence, say so. Don't manufacture symmetry where none exists.
  • Distinguish "no evidence found" from "evidence of absence." These are different epistemic states.
  • Steelman must precede evaluation. Always.
  • "Where I could be wrong" section is required. Surface the strongest counterarguments to your own conclusion, sources you couldn't access, and assumptions you're relying on.
  • Every secondary web source (Tier 3–5) needs a Wayback snapshot URL and an access date. Sources rot. Tier 1 primary sources hosted in permanent institutional archives (*.gov, court PACER, declassified-document repositories) may record Wayback: n/a if a snapshot isn't applicable or possible.
  • Use the source tier annotation on every citation.