Skip to content

NLAttack v2.0.0 — formal release (adds the Red/Blue CTF family) #23

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
SolshineCode merged 4 commits into from
Jun 27, 2026
Merged

NLAttack v2.0.0 — formal release (adds the Red/Blue CTF family) #23

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
3d1a0bd
v0.2: add Family N — CTF Red/Blue deception capture-the-flag
claude Jun 22, 2026
61cc893
Formalize the v2.0.0 benchmark release
claude Jun 26, 2026
555ddfa
docs: add v2.0.0 release-review packet (DRAFT)
claude Jun 26, 2026
3e57882
ctf: address Gemini review (JSON validity, hit consistency, threshold)
claude Jun 27, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
70 changes: 70 additions & 0 deletions CHANGELOG.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,70 @@
# Changelog

All notable changes to NLAttack. The format follows [Keep a Changelog](https://keepachangelog.com/),
and the project follows the generation/freeze policy in
[docs/VERSIONING.md](docs/VERSIONING.md): once a generation is tagged, its benchmark
content and published results are frozen.

## v2.0.0 — 2026-06

The second benchmark generation. **Additive:** every v1 result is carried forward
unchanged; v2 adds a new evaluation paradigm and matures the release process.

### Added
- **Family N — CTF Red/Blue** (`plans/N_ctf_red_blue.md`, P119–P128): an adversarial
capture-the-flag packaging of the NLA-as-monitor threat model. A Red model pursues
a hidden misaligned objective (the flag — exfiltration, sandbagging, a concealed
override, collusion, a backdoor) while a Blue NLA-monitor reads the AV
verbalization and tries to catch it. Reported as a scoreboard: clean-capture rate
(Red), catch rate (Blue's recall on genuine captures), stealth gap (the probe−AV
exploitable surface), and evasion cost.
- **`nla_eval/ctf.py`** — the runnable harness: `Flag`/`CTF_FLAGS`, `BlueMonitor`/
`nla_blue_monitor`, `survival_oracle`/`probe_capture_oracle`, `naive_red_policy`/
`adaptive_red_policy`, `play_ctf`, `CTFScore`. Reuses `redteam.adaptive_evade` and
`deception.DECEPTION_ANCHORS` (it packages existing axes, not a parallel suite).
- **`docs/CTF_RED_BLUE.md`** — the design, the four-outcome model, and the
control-vs-primary integrity tiers.
- **`experiments/ctf_red_blue_demo.py`** + `results/ctf/ctf_red_blue_mock.json` — an
offline, reproducible, CPU-only control-tier demo against `MockNLA`.
- **`CHANGELOG.md`** and **`results/README.md`** (generation provenance map).
- Domain-aware contrastive Gemma-4-E2B verbalizer-axis result
(`results/local_gemma_e2b/domainaware_nla_verbalizer_axes.json`), reported as an
honest negative on the generation axes.

### Changed
- Catalog grows from 118 plans / 13 families to **128 plans / 14 families** (the
v1 catalog A–M is unchanged; Family N is new).
- Version bumped to `2.0.0` (`nla_eval/__init__.py`, `CITATION.cff`).
- README simplified into a single-screen front page; `docs/VERSIONING.md` rewritten
around plain `v1`/`v2` generation labels.

### Integrity
- Capture is judged on the activation (the bottleneck probe), the flag on the AV
text — never the same signal (the independent-review P0 #1 separation, already
enforced in `redteam.py`).
- A CTF run is **primary-tier** only with a real bottleneck-probe oracle over the
model's own behavior; the model-agnostic survival-proxy oracle over constructed
flags is **control-tier** (mechanics, not a monitor certification). Every
`CTFScore` carries its `tier` (Family N / P128, mirroring Family M / P118).

### Not yet in this release (v2.x roadmap)
- The multi-domain corpus expansion and the H1–H12 harness-correctness fixes
(`docs/ROADMAP_v2.md`). The tagged v2.0.0 does not claim domain-coverage or
fix-dependent results it has not produced.

## v1.0.0 — 2026-06 (frozen)

The first public benchmark generation (released as `0.1.0`).

### Highlights
- 118-plan catalog across 13 families (A–M): concept survival, content adjacency and
laundering, deception and knowledge asymmetry, ATT&CK misuse detection, bottleneck
probes, matcher/verbalizer confound, adaptive red-team, faithfulness, distribution
shift, calibration, emergence, literature-informed evals, and
deception/misalignment monitoring.
- Published result: EmergenceIndex 0.601 ("established") on `gemma-3-27b-it /
kitft-l41`, plus the cross-NLA leaderboard (Llama-3.3-70B, Gemma-3-27B, and the
local Gemma-4-E2B NLA). See `docs/RESULTS.md`.
- Floor-first design and null controls on every axis; API vs full-access tiers.

**Status: frozen.** Its content and numbers do not change; v2 carries them forward.
7 changes: 4 additions & 3 deletions CITATION.cff
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,15 +6,16 @@ abstract: >-
interpretability methods that verbalize a model's internal activations into
human-readable text and reconstruct them. It scores both the bottleneck
(activation) and the verbalizer sides with explicit null controls, works on
weak or early-training NLAs, and includes a 118-plan catalog across 13
families plus deception/misalignment monitoring.
weak or early-training NLAs, and includes (as of v2) a 128-plan catalog across
14 families spanning concept survival, misuse detection, deception/misalignment
monitoring, and an adversarial Red/Blue capture-the-flag family.
type: software
authors:
- family-names: DeLeeuw
given-names: Caleb
alias: SolshineCode
email: caleb.deleeuw@gmail.com
version: "0.1.0"
version: "2.0.0"
license: Apache-2.0
url: "https://github.com/SolshineCode/NLAttack"
repository-code: "https://github.com/SolshineCode/NLAttack"
Expand Down
162 changes: 62 additions & 100 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,154 +7,116 @@
**An evaluation suite for Natural Language Autoencoders (NLAs).**

![license](https://img.shields.io/badge/license-Apache--2.0-blue.svg)
![version](https://img.shields.io/badge/release-v2.0.0-brightgreen.svg)
![status](https://img.shields.io/badge/status-research%20software-orange.svg)

NLAttack measures how well a Natural Language Autoencoder turns a model's internal
activations into human-readable text, and whether that text is good enough to use as
a safety monitor for real-world LLM misuse. It is built to work on weak, small, or
early-training NLAs, scores both the bottleneck and the verbalizer with explicit
null controls, and ships a catalog of 118 evaluations, ready-made adapters for
hosted and local NLAs, and committed result artifacts.

A **Natural Language Autoencoder** explains a model's internal state in plain
language. An activation verbalizer (AV) reads a hidden activation and writes a
description of it. An activation reconstructor (AR) reads that description and
rebuilds the activation:
language: an activation verbalizer (AV) turns a hidden activation into text, and an
activation reconstructor (AR) rebuilds the activation from that text.

```
activation --[ AV ]--> natural-language text --[ AR ]--> activation'
```

The human-readable bottleneck is the AV's text. NLAttack asks two questions about
it: does the NLA work at all (capability), and can you catch misuse by reading it
(safety)? See [docs/METHODOLOGY.md](docs/METHODOLOGY.md) for the full background.

## Results

![NLAttack benchmark overview across the evaluated NLAs](results/cross_nla/benchmark_overview.png)

Headline numbers, attributed by NLA name (full tables and caveats in
[docs/RESULTS.md](docs/RESULTS.md)):

| NLA | Access | Concept retention | Doc retrieval (semantic) | Bottleneck probe (in / OOD) |
|---|---|---|---|---|
| `Llama-3.3-70B-NLA-av@L53` | hosted | 0.95 | 0.503 | not available over API |
| `nla-gemma3-27b-av@L41` | hosted | 0.90 | not yet run | not available over API |
| `Gemma-4-E2B-NLA@L23` | local | 0.00 (out-of-domain) | 0.135 (in-domain) | 0.988 / 0.695 |

The hosted NLAs are verbalizer-strong (a reader of their AV text recovers most
concepts). The local Gemma-4-E2B v0.1 is the mirror image: its bottleneck probes
near-perfectly in-distribution, but its verbalizer is weak and domain-specific, so
it collapses on the out-of-domain dataset. Separating those failure modes is the
point of the suite.

## Installation

```bash
git clone https://github.com/SolshineCode/NLAttack.git
cd NLAttack
python -m venv .venv && source .venv/bin/activate # Windows: .venv\Scripts\activate
pip install -r requirements.txt
```

The core harness has **no required dependencies** (the lexical matcher and the mock
NLA run on the standard library). The extras unlock stronger matching
(`sentence-transformers`, `nltk`), the probe/emergence/deception axes
(`scikit-learn`, `numpy`, `scipy`), and running a local NLA (`torch`,
`transformers`, `safetensors`).
NLAttack asks two questions about that human-readable bottleneck — **does the NLA
work at all** (capability) and **can you catch misuse by reading it** (safety) — and
answers them with explicit null controls, on NLAs as weak as a 4 GB-GPU checkpoint.

## Quickstart

Run the offline smoke test (a deliberately lossy mock NLA, no network or GPU):

```bash
python run_example.py
git clone https://github.com/SolshineCode/NLAttack.git && cd NLAttack
pip install -r requirements.txt # core harness needs no dependencies
python run_example.py # offline smoke test (mock NLA, no GPU)
```

Plug in any NLA by implementing one method, then evaluate it:
Score any hosted NLA in three lines:

```python
from nla_eval import NeuronpediaNLA, EnsembleMatcher, Example, run

# A dataset is a list of Example(id, text, concepts): the controlled concepts
# present in `text` that we check survived the bottleneck.
my_dataset = [
Example(id="ex1", text="The committee published its annual budget report",
concepts=["budget", "report"]),
Example(id="ex2", text="A storm warning was issued for the coastal region",
concepts=["storm", "warning"]),
]

nla = NeuronpediaNLA(model_id="llama3.3-70b-it", nla_source_id="kitft-l53")
result = run(nla, my_dataset, matcher=EnsembleMatcher())
result = run(nla, [Example(id="ex1", text="A storm warning was issued for the coast",
concepts=["storm", "warning"])], matcher=EnsembleMatcher())
```

Discover hosted NLAs with `GET https://www.neuronpedia.org/api/nla/sources`. To
evaluate your own local NLA, implement the one-method `NLA` adapter (see
[docs/EVALUATIONS.md](docs/EVALUATIONS.md)).
To evaluate your own local NLA, implement the one-method `NLA` adapter
([docs/EVALUATIONS.md](docs/EVALUATIONS.md)).

## Results (v1, frozen)

The v1 benchmark results are published and frozen; v2 carries them forward
unchanged. Full tables and caveats in [docs/RESULTS.md](docs/RESULTS.md).

![NLAttack benchmark overview across the evaluated NLAs](results/cross_nla/benchmark_overview.png)

| NLA | Access | Concept retention | Doc retrieval (semantic) | Bottleneck probe (in / OOD) |
|---|---|---|---|---|
| `Llama-3.3-70B-NLA-av@L53` | hosted | 0.95 | 0.503 | not available over API |
| `nla-gemma3-27b-av@L41` | hosted | 0.90 | not yet run | not available over API |
| `Gemma-4-E2B-NLA@L23` | local | 0.00 (out-of-domain) | 0.135 (in-domain) | 0.988 / 0.695 |

The hosted NLAs are verbalizer-strong (a reader of their AV text recovers most
concepts). The local Gemma-4-E2B NLA is the mirror image — its bottleneck probes
near-perfectly in-distribution but its verbalizer is weak and domain-specific.
Separating those two failure modes is the point of the suite.

## What's inside

- **118 evaluations across 13 families (A-M).** Each evaluation is a designed test
with a hypothesis, a method, a metric, and a null-control baseline (the catalog
entries are called "plans"). A subset is implemented as runnable code in the
harness today, and the rest are documented designs, some awaiting GPU or data. The
families group them by theme: concept survival, content adjacency and laundering,
deception, ATT&CK misuse detection, bottleneck probes, faithfulness, distribution
shift, calibration, and emergence. Index: [plans/INDEX.md](plans/INDEX.md).
- **Two access tiers.** The API tier scores any hosted, text-only NLA (the
universal leaderboard). The full-access tier adds white-box axes (probes,
emergence) that need raw activations. Query it in code via `nla_eval.access`.
- **A 128-plan catalog across 14 families (A–N).** Every plan is a falsifiable test
with a hypothesis, a method, a metric, and a null-control baseline. Index:
[plans/INDEX.md](plans/INDEX.md).
- **Two access tiers.** An API tier scores any hosted, text-only NLA (the universal
leaderboard); a full-access tier adds white-box probe/emergence axes that need raw
activations. Query it in code via `nla_eval.access`.
- **Deception / misalignment monitoring** (Family M): can the NLA's text be read to
catch a model's own deceptive behavior?
- **Two principles throughout:** floor-first (one reliable per-concept primitive,
so weak NLAs still yield signal) and null controls on everything (a result counts
only when it clears a permutation floor).
- **CTF Red/Blue** (Family N, **new in v2**): a Red model captures a hidden
misaligned-objective flag while a Blue NLA-monitor reads the AV text to catch it,
reported as an adversarial scoreboard (clean-capture rate vs. catch rate). Offline
demo: `python experiments/ctf_red_blue_demo.py` · design:
[docs/CTF_RED_BLUE.md](docs/CTF_RED_BLUE.md).
- **Two principles throughout:** floor-first (one reliable per-concept primitive, so
weak NLAs still yield signal) and null controls on everything (a result counts only
when it clears a permutation floor).

## Documentation

| Document | Contents |
|---|---|
| [docs/METHODOLOGY.md](docs/METHODOLOGY.md) | What NLAs are, the two purposes, the ATT&CK Navigator background, and the validity limits |
| [docs/EVALUATIONS.md](docs/EVALUATIONS.md) | The 118-plan catalog, the implemented harness modules, the access tiers, and the deception family |
| [docs/RESULTS.md](docs/RESULTS.md) | Reproducible findings, the leaderboard, and the result-attribution convention |
| [DESIGN_REVIEW.md](DESIGN_REVIEW.md) | Validity threats and the rationale for the controls |
| [docs/LITERATURE.md](docs/LITERATURE.md) | The reading list with arXiv ids |
| [docs/METHODOLOGY.md](docs/METHODOLOGY.md) | What NLAs are, the two purposes, and the validity limits |
| [docs/EVALUATIONS.md](docs/EVALUATIONS.md) | The 128-plan catalog, the harness modules, and the access tiers |
| [docs/RESULTS.md](docs/RESULTS.md) | Reproducible findings, the leaderboard, and the attribution convention |
| [docs/CTF_RED_BLUE.md](docs/CTF_RED_BLUE.md) | The v2 Red/Blue capture-the-flag family (Family N) |
| [CHANGELOG.md](CHANGELOG.md) · [docs/VERSIONING.md](docs/VERSIONING.md) | Release history and the freeze-on-release policy |
| [DESIGN_REVIEW.md](DESIGN_REVIEW.md) · [docs/LITERATURE.md](docs/LITERATURE.md) | Validity threats; the reading list with arXiv ids |

## How to cite

A `CITATION.cff` is included, so GitHub shows a "Cite this repository" button.
A score is meaningful only with the NLA it was measured on — cite the canonical NLA
id, the suite version, the dataset, and the date (see [docs/RESULTS.md](docs/RESULTS.md)).

```bibtex
@software{deleeuw_nlattack_2026,
author = {DeLeeuw, Caleb},
title = {{NLAttack}: An Evaluation Suite for Natural Language Autoencoders},
year = {2026},
url = {https://github.com/SolshineCode/NLAttack},
note = {Version 0.1.0}
author = {DeLeeuw, Caleb},
title = {{NLAttack}: An Evaluation Suite for Natural Language Autoencoders},
year = {2026},
url = {https://github.com/SolshineCode/NLAttack},
note = {Version 2.0.0}
}
```

A score is meaningful only with the NLA it was measured on. When citing a result,
report the canonical NLA id, the suite commit, the dataset, and the date (see
[docs/RESULTS.md](docs/RESULTS.md)).

## Acknowledgements

NLAs are introduced in Anthropic's Natural Language Autoencoders work
([overview](https://www.anthropic.com/research/natural-language-autoencoders),
[writeup](https://transformer-circuits.pub/2026/nla/)); interactive NLAs are hosted
on [Neuronpedia](https://www.neuronpedia.org/nla). The misuse family is grounded in
Anthropic's [LLM ATT&CK Navigator](https://red.anthropic.com/2026/attack-navigator/)
and the [MITRE ATT&CK](https://attack.mitre.org/) framework.
and [MITRE ATT&CK](https://attack.mitre.org/).

## License

Apache-2.0. See [LICENSE](LICENSE) and [NOTICE](NOTICE). You may use, modify, and
redistribute it, including commercially, provided you retain the copyright,
attribution, and license notices and state your changes (Section 4). The license
includes an explicit patent grant.

## Contact

Caleb DeLeeuw (`SolshineCode`), caleb.deleeuw@gmail.com.
Apache-2.0. See [LICENSE](LICENSE) and [NOTICE](NOTICE). Contact: Caleb DeLeeuw
(`SolshineCode`), caleb.deleeuw@gmail.com.
Loading