Conversation
mbaluda
left a comment
There was a problem hiding this comment.
Please add test cases for the Anthropic models
| @@ -20,7 +20,7 @@ async def get_input_openai(): | |||
|
|
|||
| response2 = client.responses.create( | |||
| instructions="Talks like a " + persona, # $ Alert[py/prompt-injection] | |||
| input=[ | |||
| input=[ # $ Alert[py/prompt-injection] | |||
There was a problem hiding this comment.
Originally the idea was to avoid duplicate alerts like this (already reported for content),
that is why we have that logic in getContentNode()
Can you add a test if that is not sufficient?
yoff
left a comment
There was a problem hiding this comment.
LGTM so far. I assume you will take it out of draft when you want a final review.
0208d67 to
25a8aa9
Compare
|
Apologies for letting this PR linger. |
There was a problem hiding this comment.
Pull request overview
This PR extends Python prompt-injection modeling and tests to cover additional LLM SDK call patterns (OpenAI responses + chat.completions, and Anthropic messages APIs), ensuring the query flags user-controlled data flowing into these prompt construction sinks.
Changes:
- Added new OpenAI prompt-injection sinks for
chat.completions.create(messages[].content)andresponses.create(input/instructions). - Introduced Anthropic prompt-injection sink modeling (system prompts + message content) plus corresponding type modeling.
- Expanded the CWE-1427 PromptInjection query test suite and updated expected results accordingly.
Show a summary per file
| File | Description |
|---|---|
| python/ql/test/experimental/query-tests/Security/CWE-1427-PromptInjection/openai_test.py | Adds an additional alert annotation to validate responses.create(input=[...]) modeling. |
| python/ql/test/experimental/query-tests/Security/CWE-1427-PromptInjection/anthropic_test.py | New test coverage for Anthropic SDK prompt sinks (system, messages[].content) across sync/async/beta APIs. |
| python/ql/test/experimental/query-tests/Security/CWE-1427-PromptInjection/PromptInjection.expected | Updates expected results to include new Anthropic/OpenAI sink findings and paths. |
| python/ql/lib/semmle/python/frameworks/openai.model.yml | Adds OpenAI sink models for chat completions message content and responses API inputs/instructions. |
| python/ql/lib/semmle/python/frameworks/anthropic.model.yml | New Anthropic sink + type models to support prompt-injection detection. |
Copilot's findings
- Files reviewed: 5/5 changed files
- Comments generated: 0
I have a few repos where I'd like the prompt injection to trigger, and I've verified that it at least finds new sources for these:
For more info on these repos, see:
https://github.com/dsp-testing/xpi-000