Policy: add memorize cache and verify request efficiency

[tonistiigi]

In some cases, certain policy conditions run multiple times, eg, after more data is pulled in or when build steps are slightly different(eg, metadata resolve and evaluate for the same image source). Rego helpers have built-in memoization, but at least in some cases, it needed to be disabled as it was incorrectly remembering results for “unknown” fields.

The same command doing multiple requests also shouldn’t need to verify the same operation twice.

Additionally, need to verify HTTP requests made by buildkit during verification are not duplicated. Cache was added for attestation pull in v0.27, but the whole code path still needs to be verified.

[slyt3]

Hey, could I work on this?

[tonistiigi]

@slyt3 If you have a concrete case then yes. Otherwise this is not ideal first issue as I haven't really fully investigated the extent of what actually needs to be fixed myself. As mentioned in the issue, this isn't just setting Memoize: true in the rego library as that would actually do too much caching and can lead to incorrect results.