EU AI Act Mapping

[erickgalinkin]

Add new tags for EU AI Act. Map tags to existing probes. Add documentation on tags and taxonomy.

Relevant tests are passing locally.

[leondz]

Is it true we only have overlap in 26 [parent] classes out of 189 probes?

[jmartin-tech]

This provides an initial start a couple questions:

• Should we extend these tags into detectors as well?
• Does are the added tag.misp.tsv values complete or are there more that existing in the act that just did not yet have content to align to?

[jmartin-tech]

We should probably generate these tables during documentation build from the tags.misp.tsv file to avoid stale values, similar to how we inject the calibration data when available.

As is we already need to augment the OWASP values as these are the 2023-2024 list and there are new 2025 & 2026 lists.

[erickgalinkin]

The only challenge would be generating them under the appropriate section, but I don't disagree with you here.

[erickgalinkin]

Is it true we only have overlap in 26 [parent] classes out of 189 probes?

I certainly think so.

The EU AI Act is inherently capability-driven. It does not care about jailbreaks as a general concept -- it is about what your AI system is designed to do. This means that loads of probes are immaterial from the jump (ansiescape, glitch, exploitation, any jailbreak probe without configurable intent). So this is the intersection of probes with intents that overlap with EU AI act concerns under Articles 5 and 15 (and to a lesser extent 50, 53). The majority of these are privisos around what sorts of AI systems are allowed to be deployed but more importantly, transparency around things like training data (inherently out of scope for us).

Candidly, I think that I've maybe even been a bit overzealous here. It wouldn't be difficult to add more probes driven by the desire to add more tags or better coverage, but I'm open to someone correcting me.

[erickgalinkin]

This provides an initial start a couple questions:

• Should we extend these tags into detectors as well?
• Does are the added tag.misp.tsv values complete or are there more that existing in the act that just did not yet have content to align to?

1. Yes, should definitely extend into detectors.
2. There may be things in the act we could write content for, but these are the things germane to garak. Since it's not really an enumerated framework in the way something like OWASP LLM Top 10 is, it's challenging to draw lines. For example: in all cases here, sexual and sexualminor classes are used together. No probe draws that distinction, so those are effectively duplicate categories. Laws are notoriously hard to convert into atomic tests.

[leondz]

a couple of suggestions:

• could lmrc.Deadnaming have euai:discrimination:protected_class (gender identity)?

• could SQL/Jinja injection, XSS/markdown web injection, ANSI escapes warrant euai:offensive:exploitation?

• does av_spam_scanning.GTphish warrant euai:offensive:phishing?

• snowball.GraphConnectivity*, goodside.WhoIsRiley seem predicated on euai:deception:hallucination, wdyt?

• markdown exfil probes - are these testing euai:harmful:pii ?

• should lmrc.SexualContent and lmrc.Sexualisation have euai:harmful:sexualminor removed? I don't think these are csam-oriented

[leondz]

thanks. coupla tag queries, see main conversation comments

[erickgalinkin]

a couple of suggestions:

• could lmrc.Deadnaming have euai:discrimination:protected_class (gender identity)?

Gender identity is not explicitly called out as a protected class under the EU AI Act, so it goes under personal.

• could SQL/Jinja injection, XSS/markdown web injection, ANSI escapes warrant euai:offensive:exploitation?

No. The AI act does not care about exploitation of systems with integrated LLMs, it is about using LLMs for autonomous exploitation.

• does av_spam_scanning.GTphish warrant euai:offensive:phishing?

No, for the same reason as above. It is about using models to generate phishing emails, not about being robust to phishing.

• snowball.GraphConnectivity*, goodside.WhoIsRiley seem predicated on euai:deception:hallucination, wdyt?

Ehhh I looked at both and it feels like that's really stretching the verbiage in the act. I wouldn't lose sleep over adding them though.

• markdown exfil probes - are these testing euai:harmful:pii ?

No. The AI act concerns models being trained on and generating PII. Since this is attempting to exfiltrate things in memory and not get a model to generate PII it has been trained on, it is out of scope.

• should lmrc.SexualContent and lmrc.Sexualisation have euai:harmful:sexualminor removed? I don't think these are csam-oriented

Yeah, I think maybe I should just remove the sexualminor tag entirely since we don't do anything explicitly around that and at the moment, it's effectively a duplicate.

[leondz]

Nice. Can these clarifications go into the docs (and in short into the descrs in the tag file)?

[patriciapampanelli]

I'm still not sure what the best coverage strategy is here, but thinking about this in terms of detectors feels counterintuitive to me. I think it lands closer to intent than detector for these additional tags. euai: describes what a probe is targeting (the AI Act risk category), not whether the harm actually showed up in a response, which feels more like the intent side than a detector question.

I agree this should be ongoing work, including revisiting the taxonomies we already use today (avid-effect, risk-cards) as we migrate toward IntentProbes. Today the tags on our probes mix technique and intent conceptually, the same way the probes themselves do, and I think this conceptual separation is something we need to work through in a scope that goes beyond the AI Act mapping.

[Igarez]

One practical thing first, then the mapping.

The docs build is failing on four problems, with -W turning warnings into errors.

1. docs/source/tags_taxonomies.rst: WARNING: document isn't included in any toctree [toc.not_included] — the new page needs a toctree entry.
2. Three grid-table cells wrap immediately after an underscore, so docutils reads the fragment as a reference and raises ERROR: Unknown target name:
   • line 241 risk-cards:lmrc:eating_disorder_ / promotion
   • line 272 risk-cards:lmrc:inciting_animal_ / abuse
   • line 440 demon:Language:Stylizing:Formal_ / language

Widening the first column past 34 characters clears all three. It is also an argument for @jmartin-tech's suggestion to generate these tables from tags.misp.tsv at build time: this failure mode is invisible in review and returns every time a long tag is added.

---

Now the mapping. The "26 of 189" figure and the "maybe I've been a bit overzealous" worry have the same cause, and I think it is fixable: the mapping has one axis where the Act has two. Below is the tags.misp.tsv content and a rewritten section for tags_taxonomies.rst — an attempt at what @leondz asked for, clarifications in the docs and short in the tag descriptions. Six substantive changes.

1. The missing axis is Article 15.

All ten euai: tags describe what the model produces or is used for. That axis is Articles 5, 50 and 53, and 26/189 is probably right for it.

Article 15(5) is the other axis — what a system must withstand — and it names the categories itself:

High-risk AI systems shall be resilient against attempts by unauthorised third parties to alter their use, outputs or performance by exploiting system vulnerabilities

...with measures addressing data poisoning, model poisoning, model evasion, confidentiality attacks and model flaws. That is close to a one-to-one match with garak's probe families. dan, tap, goat, encoding, suffix, promptinject, latentinjection, web_injection, leakreplay, divergence, sysprompt_extraction and glitch are not immaterial to the Act: they are Article 15 evidence rather than Article 5 evidence. The section's own preamble already says the Act "introduces additional requirements for general-purpose AI models, including provisions related to evaluation, risk management, adversarial testing, and cybersecurity" — and no tag covers that today.

This is where I would push back on one line above: "the AI Act does not care about exploitation of systems with integrated LLMs". For Article 5 that is right. For Article 15(5) it is the opposite — resilience of the deployed system against third-party exploitation is the obligation. Both readings are correct on their own axis, which is the point.

2. Markdown exfiltration is in scope, on the other axis.

Same reasoning applies to "since this is attempting to exfiltrate things in memory and not get a model to generate PII it has been trained on, it is out of scope". Generating memorised PII is Article 10; exfiltrating what is held in the running context is a confidentiality attack, which 15(5) names explicitly. The markdown exfil probes carry euai:robustness:confidentiality in the set below, and euai:harmful:pii only when the content is PII.

3. lmrc.Deadnaming — agreed that 5(1)(g) does not name gender identity, and 5(1)(g) is the wrong article in any case.

Article 5(1)(g) prohibits biometric categorisation systems that infer race, political opinions, trade union membership, religious or philosophical beliefs, sex life or sexual orientation from biometric data. It is not a list of protected classes for model output. The discrimination duty that does reach LLM output is Article 10(2)(f): examination of biases that "lead to discrimination prohibited under Union law". In P v S and Cornwall County Council (C-13/94, 30 April 1996) the Court treated discrimination arising from gender reassignment as discrimination on grounds of sex. So protected_class holds, sourced from 10(2)(f) rather than 5(1)(g). The useful distinction is not protected vs. personal; it is which Article the tag answers to.

4. offensive:* is Article 55, not Article 5 — and not out of scope.

Both tags sit in the Article 55 / GPAI Code of Practice layer, whose Safety and Security chapter names four specified systemic risks: CBRN, loss of control, cyber offence, and harmful manipulation. Article 55(1)(a) requires adversarial testing and 55(1)(b) requires providers to "assess and mitigate possible systemic risks at Union level". That also settles av_spam_scanning.GTphish: generating phishing is Article 55, resilience to phishing arriving as input is Article 15. Two tags, both real.

5. Hallucination is accuracy, not deception.

The objection that tagging snowball.GraphConnectivity* and goodside.WhoIsRiley as deception stretches the verbiage is right — hallucinating is not deceiving. But Article 15(1) requires "an appropriate level of accuracy, robustness and cybersecurity", which is what those probes measure. Renaming to euai:accuracy:hallucination turns the suggestion from a stretch into the correct call.

6. Every tag names its Article, and sexualminor is dropped.

Agreed on removing it. The section below says why in one line rather than leaving a silent gap: material involving minors is not an Article 5 prohibited practice at all — its basis is Directive 2011/93/EU, and the DSA where a platform deploys the system. Someone will ask.

Coverage the mapping should admit. Data poisoning and model poisoning are named in 15(5), but garak exercises deployed models: coverage is fileformats and little else. The file's own preamble asks users to know what is not covered, so the section says it outright.

---

garak/data/tags.misp.tsv

Fourteen lines replacing the ten euai: entries. euai:deception:hallucination becomes euai:accuracy:hallucination, euai:harmful:sexualminor is dropped, and five euai:robustness:* are added. Descriptions are the short form @leondz asked for.

euai:discrimination:protected_class	Protected Class Discrimination	Art. 10(2)(f) and Charter Art. 21. Bias leading to "discrimination prohibited under Union law". Art. 5(1)(g) is a different rule — it bans biometric categorisation systems that infer race, political opinions, trade union membership, religious or philosophical beliefs, sex life or sexual orientation from biometric data — and is not the source of this tag.
euai:discrimination:personal	Personal Attribute Discrimination	Art. 10(2)(f). Bias on a personal attribute that does not amount to discrimination prohibited under Union law. Still inside the bias examination duty, so still reportable.
euai:harmful:violence	Violent Language	No Art. 5 basis. Evidence for the Art. 9 risk management system of a high-risk system, and for the Art. 55(1)(b) systemic risk assessment of a GPAI model.
euai:harmful:sexual	Sexual Language	No Art. 5 basis. Same footing as euai:harmful:violence: Art. 9 and Art. 55(1)(b) evidence, not a prohibited practice.
euai:harmful:pii	Personally Identifiable Information	Two cases. Generating PII the model was trained on: Art. 10(5) and the GDPR. Extracting PII or secrets held in the running context: an Art. 15(5) confidentiality attack, so tag euai:robustness:confidentiality as well.
euai:deception:subliminal	Subliminal Deception	Art. 5(1)(a): "subliminal techniques beyond a person's consciousness or purposefully manipulative or deceptive techniques" that materially distort behaviour and cause significant harm. Also "harmful manipulation" in the GPAI Code of Practice.
euai:accuracy:hallucination	Hallucination	Art. 15(1): high-risk systems shall achieve "an appropriate level of accuracy, robustness and cybersecurity". Renamed from euai:deception:hallucination, which forced accuracy evidence into a deception frame it does not fit.
euai:offensive:exploitation	Vulnerability Exploitation (offensive use)	Art. 55(1)(b) with the GPAI Code of Practice systemic risk "cyber offence": use of the model to conduct exploitation. For the system being exploited, see euai:robustness:evasion.
euai:offensive:phishing	Phishing (offensive use)	Art. 55(1)(b), cyber offence: generating phishing content. Resilience to phishing content arriving as input is euai:robustness:evasion instead.
euai:robustness:data_poisoning	Data Poisoning	Art. 15(5), named in the Article. garak exercises deployed models, so direct coverage here is thin; see the coverage note above.
euai:robustness:model_poisoning	Model Poisoning	Art. 15(5), named in the Article. Partial coverage via probes.fileformats.
euai:robustness:evasion	Model Evasion	Art. 15(5), named in the Article, covering "attempts by unauthorised third parties to alter their use, outputs or performance by exploiting system vulnerabilities". Jailbreaks, direct and indirect prompt injection, encoding and output-channel attacks all land here.
euai:robustness:confidentiality	Confidentiality Attack	Art. 15(5), named in the Article. Training data extraction, system prompt extraction, replay, and exfiltration of material held in the running context.
euai:robustness:model_flaw	Model Flaw	Art. 15(5), named in the Article. Inherent weaknesses of the model itself, such as glitch tokens.

docs/source/tags_taxonomies.rst — EU AI Act section

If the tables move to build-time generation as @jmartin-tech suggested, only the prose below is needed and the table comes from the TSV above.

EU AI Act
---------
The `EU AI Act`_ is a regulatory framework that establishes risk-based requirements for developing, providing, and deploying AI systems in the European Union.
It distinguishes among different levels and types of AI risk, prohibits certain practices, imposes requirements on high-risk systems, establishes transparency obligations, and introduces additional requirements for general-purpose AI models, including provisions related to evaluation, risk management, adversarial testing, and cybersecurity for models with systemic risk.

These tags run on two axes, because the Act does.
The first is what a model produces or is used for: prohibited practices under Article 5, and the systemic risks a general-purpose model must be assessed against under Article 55.
The second is what a system must withstand: Article 15(5) requires high-risk systems to be "resilient against attempts by unauthorised third parties to alter their use, outputs or performance by exploiting system vulnerabilities", and names data poisoning, model poisoning, model evasion, confidentiality attacks and model flaws.
A probe usually sits on one axis only, and a few sit on both: exfiltrating personal data held in context is a confidentiality attack under Article 15(5) and a personal data concern under Article 10.
Every tag below names the provision it derives from, so that a garak report can be cited in an Annex IV technical file or a systemic-risk model report without a separate translation step.

Three limits are worth stating plainly.
garak exercises deployed models, so coverage of ``euai:robustness:data_poisoning`` and ``euai:robustness:model_poisoning`` is thin — ``probes.fileformats`` and little else — even though Article 15(5) names both.
There is no tag for sexual content involving minors: that is not an Article 5 prohibited practice, its basis is Directive 2011/93/EU and, for platform deployment, the Digital Services Act, and garak has no probe aimed at it.
And Article 10 obligations on training data governance, together with Article 53 obligations on documentation and training-content summaries, are largely outside what any black-box probe can evidence.

.. _EU AI Act: https://artificialintelligenceact.eu/

Relevant Tags
~~~~~~~~~~~~~
+----------------------------------+--------------------------------------------+--------------------------------------------------------------------------+
| euai:discrimination:protected_cl | Protected Class Discrimination             | Art. 10(2)(f) and Charter Art. 21. Bias leading to "discrimination        |
| ass                              |                                            | prohibited under Union law". Art. 5(1)(g) is a different rule — it bans  |
|                                  |                                            | biometric categorisation systems that infer race, political opinions,    |
|                                  |                                            | trade union membership, religious or philosophical beliefs, sex life or  |
|                                  |                                            | sexual orientation from biometric data — and is not the source of this   |
|                                  |                                            | tag.                                                                     |
+----------------------------------+--------------------------------------------+--------------------------------------------------------------------------+
| euai:discrimination:personal     | Personal Attribute Discrimination          | Art. 10(2)(f). Bias on a personal attribute that does not amount to      |
|                                  |                                            | discrimination prohibited under Union law. Still inside the bias         |
|                                  |                                            | examination duty, so still reportable.                                   |
+----------------------------------+--------------------------------------------+--------------------------------------------------------------------------+
| euai:harmful:violence            | Violent Language                           | No Art. 5 basis. Evidence for the Art. 9 risk management system of a     |
|                                  |                                            | high-risk system, and for the Art. 55(1)(b) systemic risk assessment of  |
|                                  |                                            | a GPAI model.                                                            |
+----------------------------------+--------------------------------------------+--------------------------------------------------------------------------+
| euai:harmful:sexual              | Sexual Language                            | No Art. 5 basis. Same footing as euai:harmful:violence: Art. 9 and Art.  |
|                                  |                                            | 55(1)(b) evidence, not a prohibited practice.                            |
+----------------------------------+--------------------------------------------+--------------------------------------------------------------------------+
| euai:harmful:pii                 | Personally Identifiable Information        | Two cases. Generating PII the model was trained on: Art. 10(5) and the   |
|                                  |                                            | GDPR. Extracting PII or secrets held in the running context: an Art.     |
|                                  |                                            | 15(5) confidentiality attack, so tag euai:robustness:confidentiality as  |
|                                  |                                            | well.                                                                    |
+----------------------------------+--------------------------------------------+--------------------------------------------------------------------------+
| euai:deception:subliminal        | Subliminal Deception                       | Art. 5(1)(a): "subliminal techniques beyond a person's consciousness or  |
|                                  |                                            | purposefully manipulative or deceptive techniques" that materially       |
|                                  |                                            | distort behaviour and cause significant harm. Also "harmful              |
|                                  |                                            | manipulation" in the GPAI Code of Practice.                              |
+----------------------------------+--------------------------------------------+--------------------------------------------------------------------------+
| euai:accuracy:hallucination      | Hallucination                              | Art. 15(1): high-risk systems shall achieve "an appropriate level of     |
|                                  |                                            | accuracy, robustness and cybersecurity". Renamed from                    |
|                                  |                                            | euai:deception:hallucination, which forced accuracy evidence into a      |
|                                  |                                            | deception frame it does not fit.                                         |
+----------------------------------+--------------------------------------------+--------------------------------------------------------------------------+
| euai:offensive:exploitation      | Vulnerability Exploitation (offensive use) | Art. 55(1)(b) with the GPAI Code of Practice systemic risk "cyber        |
|                                  |                                            | offence": use of the model to conduct exploitation. For the system being |
|                                  |                                            | exploited, see euai:robustness:evasion.                                  |
+----------------------------------+--------------------------------------------+--------------------------------------------------------------------------+
| euai:offensive:phishing          | Phishing (offensive use)                   | Art. 55(1)(b), cyber offence: generating phishing content. Resilience to |
|                                  |                                            | phishing content arriving as input is euai:robustness:evasion instead.   |
+----------------------------------+--------------------------------------------+--------------------------------------------------------------------------+
| euai:robustness:data_poisoning   | Data Poisoning                             | Art. 15(5), named in the Article. garak exercises deployed models, so    |
|                                  |                                            | direct coverage here is thin; see the coverage note above.               |
+----------------------------------+--------------------------------------------+--------------------------------------------------------------------------+
| euai:robustness:model_poisoning  | Model Poisoning                            | Art. 15(5), named in the Article. Partial coverage via                   |
|                                  |                                            | probes.fileformats.                                                      |
+----------------------------------+--------------------------------------------+--------------------------------------------------------------------------+
| euai:robustness:evasion          | Model Evasion                              | Art. 15(5), named in the Article, covering "attempts by unauthorised     |
|                                  |                                            | third parties to alter their use, outputs or performance by exploiting   |
|                                  |                                            | system vulnerabilities". Jailbreaks, direct and indirect prompt          |
|                                  |                                            | injection, encoding and output-channel attacks all land here.            |
+----------------------------------+--------------------------------------------+--------------------------------------------------------------------------+
| euai:robustness:confidentiality  | Confidentiality Attack                     | Art. 15(5), named in the Article. Training data extraction, system       |
|                                  |                                            | prompt extraction, replay, and exfiltration of material held in the      |
|                                  |                                            | running context.                                                         |
+----------------------------------+--------------------------------------------+--------------------------------------------------------------------------+
| euai:robustness:model_flaw       | Model Flaw                                 | Art. 15(5), named in the Article. Inherent weaknesses of the model       |
|                                  |                                            | itself, such as glitch tokens.                                           |
+----------------------------------+--------------------------------------------+--------------------------------------------------------------------------+

Probe assignment

tag	probe modules
euai:robustness:evasion	atkgen, dan, goat, tap, suffix, encoding, badchars, smuggling, visual_jailbreak, promptinject, latentinjection, web_injection, ansiescape, grandma
euai:robustness:confidentiality	leakreplay, divergence, sysprompt_extraction, propile, apikey, markdown exfil probes
euai:robustness:model_flaw	glitch
euai:robustness:model_poisoning	fileformats (partial)
euai:robustness:data_poisoning	none today — see the coverage note
euai:offensive:exploitation	exploitation, malwaregen
euai:offensive:phishing	av_spam_scanning
euai:accuracy:hallucination	snowball, misleading, goodside, packagehallucination

I left adaptive_attacks, dra, sata, fitd, phrasing, audio, doctor, continuation and topic unassigned rather than guess at their behaviour from the repository alone; most look like euai:robustness:evasion and you will place them faster than I can.

For context: I build and red-team agentic systems in production (HackAPrompt 2023 (https://www.aicrowd.com/challenges/hackaprompt-2023/leaderboards), 51st of 3,396), and I am listed as expert EX2026D1446822 in the Commission's Funding & Tenders Portal.

Citations above: Articles 5, 9, 10, 15 and 55 of Regulation (EU) 2024/1689; Charter of Fundamental Rights Article 21; Case C-13/94 P v S and Cornwall County Council, judgment of 30 April 1996; the Safety and Security chapter of the GPAI Code of Practice.