Add the Article 15 resilience axis to the EU AI Act tags - #1
Closed
Igarez wants to merge 1 commit into
Closed
Conversation
The euai: tags all describe what a model produces or is used for, which is
the Article 5 / Article 55 axis. Article 15(5) is the other axis - what a
high-risk system must withstand - and it names data poisoning, model
poisoning, model evasion, confidentiality attacks and model flaws.
- add euai:robustness:{data_poisoning,model_poisoning,evasion,
confidentiality,model_flaw}, sourced to Art. 15(5)
- rename euai:deception:hallucination to euai:accuracy:hallucination,
sourced to Art. 15(1) accuracy rather than a deception frame
- drop euai:harmful:sexualminor: not an Art. 5 prohibited practice, its
basis is Directive 2011/93/EU and the DSA
- give every euai: description the provision it derives from
- assign the robustness tags to the probe families that evidence them
- document the two axes and the three coverage limits
Signed-off-by: Ignacio Aredez <ioaredez@genaiaredez.com>
Owner
|
I am very, very grateful for your additions here. I made some updates independently and will close this PR. There are a few places I don't fully align with your analysis but please feel free to continue discussing with me on the main thread. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Follow-up to the discussion in NVIDIA#2094, as a patch rather than a comment: @leondz asked for the clarifications to go into the docs and, in short form, into the tag descriptions, and @erickgalinkin said the tags should extend further. It targets
ai-act-mappingso it can be reviewed and squashed into that PR, or dropped if the reading is wrong.Rationale and the Article-by-Article sourcing are in my comment on #2094. In short: the ten
euai:tags describe what a model produces or is used for — the Article 5 and Article 55 axis, where 26/189 is about right. Article 15(5) is the other axis, what a high-risk system must withstand, and it names the categories itself: data poisoning, model poisoning, model evasion, confidentiality attacks and model flaws.What changes
euai:robustness:*tags added, sourced to Art. 15(5).dan,tap,goat,encoding,suffix,promptinject,latentinjection,web_injection,leakreplay,divergence,sysprompt_extractionandglitchbecome Article 15 evidence instead of no evidence at all.euai:deception:hallucination→euai:accuracy:hallucination, sourced to Art. 15(1) accuracy. Hallucinating is not deceiving, which was the objection; accuracy is what those probes measure.euai:harmful:sexualminordropped. Not an Article 5 prohibited practice: its basis is Directive 2011/93/EU, and the DSA where a platform deploys the system.protected_classis sourced to Art. 10(2)(f) as well as 5(1)(g), which is what reacheslmrc.Deadnaming— Case C-13/94 P v S holds that discrimination on gender reassignment is discrimination on grounds of sex.What this deliberately does not do
adaptive_attacks,dra,sata,fitd,phrasing,audio,doctor,continuationandtopicare left unassigned rather than guessed at from the repository alone; most look likeeuai:robustness:evasion.goodside.Davidjlis likewise left alone.euai:robustness:data_poisoninghas no probe today and the docs say so.Checks
Every
euai:tag in the tree resolves againsttags.misp.tsv(no dangling references after the rename and the drop);blackoutput is unchanged from the base branch; the reStructuredText added parses clean under docutils, with the same single INFO the section already produces onai-act-mapping.