Fix quadratic-time PEM post-boundary check

[BrianWillows]

Fix quadratic-time PEM post-boundary check

Summary

lib/Crypto/IO/PEM.py:133 verifies the post-encapsulation boundary with

r = re.compile(r"-----END (.*)-----\s*$")
m = r.search(pem_data)

search gives every occurrence of -----END in the input its own start
position, and (.*) is greedy, so at each one it consumes the whole remainder
and backtracks looking for ----- followed only by whitespace to end of
string. The cost is quadratic in len(pem_data).

PEM.decode is reached from RSA.import_key, ECC.import_key,
DSA.import_key and Crypto.IO.PKCS8, so an application that imports a key or
certificate supplied by a user reaches this with attacker-controlled length.
There is no size limit anywhere on that path.

Because setup.py generates pycryptodomex from this same lib/Crypto tree,
one change fixes both distributions.

Reproducing

from Crypto.PublicKey import RSA
pem = "-----BEGIN X-----\n" + "-----END " * 14000   # about 125 KB of text
RSA.import_key(pem)                                  # raises, after ~10 s

The pre-boundary regex on line 126 is applied with match, not search, and
is not involved: measured flat at 0.00 ms across the whole range below.

Measured on a clean install of 3.23.0, warmed up, best of three:

input	PEM.decode	RSA.import_key	valid PEM, same size
8 KB	41.6 ms	36.3 ms	0.0 ms
16 KB	184.8 ms	189.5 ms	0.1 ms
32 KB	717.7 ms	720.2 ms	0.1 ms
64 KB	2,539.8 ms	2,627.4 ms	0.2 ms
128 KB	10,118.4 ms	10,179.1 ms	0.4 ms

Roughly x3.8 per doubling. A valid PEM of the same size stays under half a
millisecond, so the cost is the boundary check and not the decoding.

The change

The marker is already known: it was captured by the pre-boundary match three
lines above. So the search is not needed at all, and neither is the
m.group(1) != marker comparison, since the marker is built into the string
being matched.

if not pem_data.rstrip().endswith("-----END %s-----" % marker):
    raise ValueError("Not a valid PEM post boundary")

input	before	after
8 KB	36.3 ms	0.0 ms
32 KB	720.2 ms	0.0 ms
128 KB	10,179.1 ms	0.2 ms

After the patch the adversarial input costs the same as a valid PEM of the same
size. The growth exponent changes, not just the constant.

Equivalence

Differential test over 12 inputs, shipped logic against patched logic, with
no disagreements (equivalence.py in the linked report):

plain; missing trailing newline; CRLF line endings; trailing spaces and blank
lines; marker mismatch; absent end boundary; marker containing a hyphen
(A-B); RSA PRIVATE KEY; ENCRYPTED PRIVATE KEY; EC PARAMETERS; trailing
junk after the boundary; a second -----END mid-document.

The hyphen case is why this uses endswith rather than the smaller change of
([^-\n]*) for (.*). That alternative is also linear, but it would reject
markers containing a hyphen, which PEM.encode accepts from the caller.

Test suite

test_PKCS8, test_PBES, test_import_RSA, test_import_ECC,
test_import_DSA, test_import_Curve25519, test_import_Curve448:

SHIPPED   run=91 fail=0 err=0
PATCHED   run=91 fail=0 err=0

To confirm those tests actually exercise the changed line rather than passing
vacuously, the patched comparison was then deliberately corrupted
(-----END to -----ZZZ) and the suite re-run: it fails, including
testImportKey8, testImportKey9, test_x509v1 and test_x509v3. Restoring
the patch returns it to 91 passing.

Note

This patch and its measurements were prepared with AI assistance (Claude).
Every figure above was measured on a clean install rather than estimated, the
equivalence table is the output of a differential test rather than a claim
about the change, and the test result carries the sabotage control above so
that "91 passed" means something.