# Datasheet — SovereignBench v1 (semantic-leakage corpus)

Following Gebru et al., *Datasheets for Datasets*. Ships with the public split; a
machine-readable **Croissant** `metadata.json` mirrors these fields.

## Motivation
- **Purpose:** measure whether AI systems prevent sensitive-data egress that has **no
  surface form** — leaks that pattern/DLP tools provably cannot see. Fills the gap
  left by literal-PII benchmarks (REDACT-DE), contamination "leakage" work, and
  jailbreak-safety suites (AILuminate) / multi-agent leakage (AgentLeak).
- **Created by:** AI-Z Group (founding contributor) + academic co-author + neutral
  steward. Scoring/leaderboard owned by the steward, not AI-Z.

## Composition
- **Instances:** prompts (cases). Each: `id, mode, submode, domain, lang, difficulty,
  text, expected, gold{leak,severity,recoverability}, payload, provenance, source_ref,
  license, canary, split, notes`.
- **Protect vs control:** ≥ 60% protect / ≥ 40% benign controls (over-block metric).
- **Modes (9):** encoding · mosaic · special_context · business · fragmented ·
  crosslingual · cloaking · indirect · format. **Difficulty:** T1–T4 (T4 retains
  headroom by design). **Languages:** de, en (professional translation, not MT).
- **No real PII.** All person/account/credential values are **fabricated**; only the
  *scenario structure* is drawn from public incident classes. Payloads never appear
  as a surface form (validator-enforced).
- **Splits:** ~60% public (released) / ~40% **held-out private** (steward-only,
  anti-gaming). Canary strings on a subset → detect training-set contamination.

## Collection & provenance (5 streams, per-case tagged)
1. **incident** — archetype from public breach/DPA records (US HHS Breach Portal,
   UK ICO register, EDPB/EU DPA decisions, VERIS/VCDB); values fabricated;
   `source_ref` names the source class (specific case IDs filled by annotators —
   never fabricated).
2. **document** — real public form/letter templates (Arztbrief, Kontoauszug, HR,
   court filing), fabricated values.
3. **expert** — security/privacy professionals under a written red-team protocol.
4. **crowd** — public "break-it" submissions.
5. **naturalistic** (v2) — consented, anonymized enterprise logs (DPA-gated).

## Labeling
Human gold only (≥3 raters, `CODEBOOK.md`, adjudication, Fleiss' κ ≥ 0.70). **No
system-under-test labels its own data.** Low-κ cases excluded (counts reported).

## Uses
- Intended: evaluating redaction / egress-governance / guardrail systems on semantic
  leakage; research on the detector↔reasoning gap.
- **Out of scope / misuse:** as training data (contamination — canaries will flag it);
  as a literal-PII benchmark; as a safety/jailbreak benchmark.

## Distribution & maintenance
- Public split: GitHub + Hugging Face; corpus **CC-BY-SA-4.0**, code Apache-2.0.
- Versioned (v1.0, v1.1…); leaderboard pinned to a corpus version; private split
  rotated on any contamination signal. Steward is the point of contact.

## Ethical & legal
- No real personal data is distributed → no data-subject risk from the corpus itself.
- Incident archetypes are public; no re-identification of real victims (values fake).
- Dual-use note: the corpus catalogues *how* leaks evade detection. Mitigation: it
  targets defensive evaluation; contains no working exploits beyond illustrative
  encodings; released with defensive framing under the steward's governance.