generate() only type-checks _call_model output on 2 of its 4 paths

[karthikchundi-commits]

What

Generator.generate() calls _verify_target_result() to enforce that _call_model returns a list of Message/None items. But it only calls it on two of the four code paths that populate outputs:

path	condition	verified?
1	generations_this_call == 1	no
2	supports_multiple_generations == True	no
3	parallel (pool.imap_unordered)	yes
4	serial multi-generation loop	yes

Path 1 is the common case for a normal scan. A generator whose _call_model returns malformed output (a raw str instead of a Message, wrong item type) sails straight through, then fails later with an opaque AttributeError: 'str' object has no attribute 'text' inside _prune_skip_sequences or a detector — instead of the clear contract assertion _verify_target_result exists to raise.

Reproduction

import garak._config; garak._config.load_base_config()
from garak import _plugins
from garak.attempt import Conversation, Turn, Message

g = _plugins.load_plugin("generators.test.Blank")
g.supports_multiple_generations = False
g._call_model = lambda prompt, n=1: ["raw str, not a Message"] * n
conv = Conversation([Turn("user", [Message("hi")])])

print(g.generate(conv, 1))   # -> ['raw str, not a Message']  (no error; should assert)

The serial-multi path (g.generate(conv, 3) with supports_multiple_generations=False) does raise here — the inconsistency is the bug.

This is not hypothetical

CohereGenerator._call_model currently returns raw str on every non-empty-prompt success path (v1: return [gen.text for ...]; v2: responses.append(content_item.text)). Every other generator (openai, anthropic, mistral, …) wraps in Message. The incomplete guardrail is why this hasn't surfaced as a hard failure. (Open PR #1673 fixes Cohere as part of a broader modernization.)

Fix

Have generate() verify outputs on paths 1 and 2 as well, and make _verify_target_result check every item + take an explicit expected length. I have a fix + tests ready (tests/generators/test_generators_base.py, currently empty) and will open a PR.

Tested on: Python 3.12, Windows; pytest tests/generators/.