Add indirect prompt injection probe, tool messages - #2058
Conversation
…` to check the schema of context JSON files.
…` to check the schema of context JSON files.
|
We are exploring the same problem from a slightly different angle over on our fork at RedHat, so I wanted to flag it early to avoid duplicated effort and hopefully converge. At a high level, our approach is replay-based rather than in-tree construction. We are producing harmful conversations + tools as part of a different process (asago artifact generator) so a payload can be injected anywhere in the dialogue, including as a tool-call result. Garak then replays each conversation unchanged against the target and judges the response with an LLM-as-a-judge. Concretely, we've been building out:
Leaving aside the actual attack vector/probe I think it would be nice to get a solid |
Draft PR for discussion. Adds
"tool"role to messages, necessary tool calling params. Will need to adapt a bit more.Have not verified yet.