feat(NODE-7537): add QE String GA - rename TextPreview to String, sync spec tests#4972
feat(NODE-7537): add QE String GA - rename TextPreview to String, sync spec tests#4972tadjik1 wants to merge 14 commits into
TextPreview to String, sync spec tests#4972Conversation
TextPreview to String, sync spec tests
…c spec tests - Add 'String' algorithm and 'prefix'/'suffix' query types (GA on MongoDB 9.0+) - Rename TextQueryOptions → StringQueryOptions; keep TextQueryOptions as deprecated alias - Add stringOptions to ClientEncryptionEncryptOptions; textOptions kept as deprecated alias - Rewrite prose test 27 for the GA API with all 11 cases (DRIVERS-3470 regression cases included) - Vendor QE-Text-prefix/suffix unified tests and new encryptedFields data files from specifications
Revert DRIVERS-2295 YAML reformatting changes and remove data files from DRIVERS-3547 and DRIVERS-2297 that were over-synced into this PR. This PR only implements NODE-7537, NODE-7221, and NODE-7434.
…se 27 Pass extraOptions with the crypt_shared library path when creating autoEncryptedClient, matching the pattern used elsewhere. Without this, mongocrypt_init() fails on environments where crypt_shared is already loaded by another client in the same process.
82ba80c to
9b68494
Compare
| { requires: { ...defaultMetadata.requires, mongodb: '>=8.1.0' } } | ||
| ); | ||
|
|
||
| // Case 8 expects one of two error messages depending on mongocryptd/crypt_shared and libmongocrypt versions: |
There was a problem hiding this comment.
Case 8 accepts two error messages because the server changed the error text in 8.2 via MONGOCRYPT-793. See spec prose test 25 Case 8.
The second message (csfleEncryptionSchemas) was not observable before libmongocrypt 1.17.0.
| { requires: { ...defaultMetadata.requires, mongodb: '>=7.0.0 <8.1.0' } } | ||
| ); | ||
|
|
||
| // Case 10 requires server 8.2+, mongocryptd/crypt_shared 8.2+, and libmongocrypt 1.17.0+. |
There was a problem hiding this comment.
Case 10 is new in NODE-7221: verifies that a QE collection can $lookup into a collection with a non-encryption JSON schema on server 8.2+. See spec prose test 25 Case 10.
Cases 8-11 (DRIVERS-3470 regression tests for case/diacritic-insensitive string encryption) belong to NODE-7578 which is a separate backlog ticket. Remove them along with their encryptedFields data files and the autoEncryptedClient that was only used by those cases.
… 25/27 Add spec reference comments matching the pattern used in other prose tests (e.g. prose.12). Fix prose 25 describe title from '$lookup support' to '25. Test $lookup' to match the spec section heading.
…names
Remove (GA)/(preview) from it() titles — they aren't part of the spec.
Use context('Case N: ...') with it('GA') / it('preview') inside instead.
…oping Spec prose comments are exact copies of spec wording and must be present. Restore them for prose 27 with updated terminology (String/stringOptions/prefix/suffix). Also revert the encryptedPrefixSuffix/encryptedSubstring rename — use block-scoped encryptedText instead so the variable name matches the spec's field name and the diff stays minimal.
The GA qualifier is context-specific to this PR. Once the preview path is gone, the distinction disappears — so the primary metadata object is just `metadata`.
There was a problem hiding this comment.
Pull request overview
This PR promotes Queryable Encryption (QE) text search from Technical Preview to GA for MongoDB 9.0 by renaming the “TextPreview” algorithm/options to “String”, updating query type names (e.g., prefixPreview/suffixPreview → prefix/suffix), syncing the upstream CSFLE/QE spec fixtures, and adjusting prose integration tests accordingly. It also updates the mongodb-client-encryption dependency range to pick up newer libmongocrypt support.
Changes:
- Introduce
StringQueryOptionsin the public API and route bothstringOptionsand deprecatedtextOptionsthrough explicit encryption context building. - Sync CSFLE/QE unified spec test fixtures for prefix/suffix GA vs preview behavior, and add new GA fixture files for server 9.0+.
- Update CSFLE prose integration tests for “String Explicit Encryption” and
$lookupbehavior/error message variations, plus widenmongodb-client-encryptiondependency range.
Reviewed changes
Copilot reviewed 21 out of 22 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
| test/spec/client-side-encryption/tests/unified/QE-Text-suffixPreview.yml | Bumps libmongocrypt minimum and corrects suffix preview test description. |
| test/spec/client-side-encryption/tests/unified/QE-Text-suffixPreview.json | JSON equivalent of suffix preview fixture updates. |
| test/spec/client-side-encryption/tests/unified/QE-Text-suffix.yml | Adds GA suffix unified fixture for server 9.0+. |
| test/spec/client-side-encryption/tests/unified/QE-Text-suffix.json | JSON equivalent of GA suffix unified fixture. |
| test/spec/client-side-encryption/tests/unified/QE-Text-prefixPreview.yml | Bumps libmongocrypt minimum for prefix preview fixture. |
| test/spec/client-side-encryption/tests/unified/QE-Text-prefixPreview.json | JSON equivalent of prefix preview fixture updates. |
| test/spec/client-side-encryption/tests/unified/QE-Text-prefix.yml | Adds GA prefix unified fixture for server 9.0+. |
| test/spec/client-side-encryption/tests/unified/QE-Text-prefix.json | JSON equivalent of GA prefix unified fixture. |
| test/spec/client-side-encryption/tests/unified/QE-Text-compactStructuredEncryptionData.yml | Moves structured-encryption-data test to GA server/libmongocrypt requirements and queryType rename. |
| test/spec/client-side-encryption/tests/unified/QE-Text-compactStructuredEncryptionData.json | JSON equivalent of compact structured encryption data fixture updates. |
| test/spec/client-side-encryption/tests/unified/QE-Text-cleanupStructuredEncryptionData.yml | Moves cleanup structured-encryption-data test to GA server/libmongocrypt requirements and queryType rename. |
| test/spec/client-side-encryption/tests/unified/QE-Text-cleanupStructuredEncryptionData.json | JSON equivalent of cleanup structured encryption data fixture updates. |
| test/spec/client-side-encryption/etc/data/lookup/schema-non-csfle.json | Adds non-encryption JSON schema validator fixture for $lookup prose tests. |
| test/spec/client-side-encryption/etc/data/encryptedFields-substring.json | Updates substring encryptedFields fixture formatting and adds contention. |
| test/spec/client-side-encryption/etc/data/encryptedFields-prefix-suffix.json | Renames query types to GA (prefix/suffix) and adds contention. |
| test/spec/client-side-encryption/etc/data/encryptedFields-prefix-suffix-preview.json | Adds preview-only encryptedFields fixture for pre-9.0 servers. |
| test/integration/client-side-encryption/client_side_encryption.prose.27.text_queries.test.ts | Updates prose test #27 to “String Explicit Encryption” and splits GA vs preview coverage. |
| test/integration/client-side-encryption/client_side_encryption.prose.25.lookup.test.ts | Updates prose test #25 naming, fixtures, and error-message matching; adds a new case using non-CSFLE schema. |
| src/index.ts | Exports StringQueryOptions from the public entrypoint. |
| src/client-side-encryption/client_encryption.ts | Adds String algorithm/query type support, introduces StringQueryOptions, and keeps TextQueryOptions as deprecated alias. |
| package.json | Widens mongodb-client-encryption range to ^7.0.0. |
| package-lock.json | Locks mongodb-client-encryption to 7.1.0. |
| const resolvedStringOptions = stringOptions ?? textOptions; | ||
| if (typeof resolvedStringOptions === 'object') { | ||
| contextOptions.textOptions = serialize(resolvedStringOptions); | ||
| } |
There was a problem hiding this comment.
Disagree here, we already preferred new stringOptions, and accepting both we just help potential clients to migrate. Left a comment above with self-review on this specific line.
There was a problem hiding this comment.
If we want to accept both (I'm not sure we do), then we should flip the order of stringOptions ?? textOptions and prioritize the new setting over the old setting.
There was a problem hiding this comment.
We can't remove it, that's why textOptions is deprecated. And stringOptions has precedence as new standard way to pass these options.
Technically speaking this is our "internal" field, we don't have to rename it, but I just thought it would be more consistent. Godriver does the same here, for example.
I'm ok to add this suggested condition and disallow passing both options at the same time, is that what you suggest?
There was a problem hiding this comment.
Gotcha. When both settings are specified, it seems appropriate here to coalesce between the old and the new.
I was thinking of throwing, but it looks like in the driver we typically only do that for conflicting options, not for renamed ones. Is there a plan to drop the old property? Should we add a deprecation notice, or invoke emitWarningOnce?
| "gcp-metadata": "^7.0.1", | ||
| "kerberos": "^7.0.0", | ||
| "mongodb-client-encryption": ">=7.0.0 <7.1.0", | ||
| "mongodb-client-encryption": "^7.0.0", |
There was a problem hiding this comment.
Should this be ^7.1.0 so the latest libomngocrypt always gets picked up?
There was a problem hiding this comment.
Yes! It should, great catch
PavelSafronov
left a comment
There was a problem hiding this comment.
Changes look good. But the PR description is missing release highlights.
Sending this to Team Review as-is, will approve after release notes are added.
|
Thanks @PavelSafronov! You are absolutely right about release notes, I have approached the team asking if we are going to have some general release notes template for this feature, as it's common among all drivers. I will update this PR once I get feedback on that. |
Description
Summary of Changes
Promotes QE text search from Technical Preview to GA for MongoDB 9.0 (NODE-7537).
Three tickets are bundled — they are closely related and too small to split into separate PRs:
TextPreviewtoString,prefixPreview/suffixPreviewtoprefix/suffixin source and tests,@deprecatedold names$lookupprose test 25: fix Case 8 error message for server 8.2+ (MONGOCRYPT-793), add Case 10 (db.qejoinsdb.non_csfle_schema)Includes the NODE-7569 dependency bump (libmongocrypt 1.19.2).
Release Highlight
Queryable Encryption string queries are now generally available
Queryable Encryption string queries are now generally available for MongoDB 9.0. Building on the technical preview introduced in earlier releases, this feature lets you run exact and range-style string matching against encrypted fields. As part of the promotion to GA, the API has been renamed:
TextOptsAPI is replaced withStringOpts.TextPreviewalgorithm is replaced withString.prefix,suffix, andsubstringquery types are now generally available.prefixPreview,suffixPreview, andsubstringPreviewquery types remain experimental and may be removed in a future release.Double check the following
npm run check:lint)type(NODE-xxxx)[!]: description