[MongoDB] Handle out-of-range BSON datetimes to prevent sync failures

[Jan-Kazlouski-elastic]

Summary

Documents containing dates outside the Python datetime range (years 1–9999) cause pymongo to raise InvalidBSON while decoding, which aborts the entire MongoDB sync:

year 643385 is out of range (Consider Using CodecOptions(datetime_conversion=DATETIME_AUTO) ...)

This PR configures the MongoDB client with DatetimeConversion.DATETIME_CLAMP, so out-of-range values are clamped to datetime.min / datetime.max and remain valid dates. They then serialize to normal ISO date strings, exactly like in-range dates.

Why DATETIME_CLAMP (and not DATETIME_AUTO)

The earlier (closed) PR #3476 used DATETIME_AUTO, which keeps in-range values as dates but converts out-of-range values to long (epoch millis). That makes a single field hold mixed types (ISO string vs. long). Because Elasticsearch infers a field's type from the first document it sees, a later document of the other type is then rejected — turning the hard crash into an order-dependent ingestion failure.

DATETIME_CLAMP avoids this entirely: every date serializes as a date string, so there is one consistent mapping, no crash, and no breaking change for existing users. The trade-off is that extreme dates lose their exact value (clamped to the datetime bounds) — an acceptable outcome for what are almost always sentinel/garbage dates.

Context

This revives and supersedes the work from the earlier (closed) PR #3476, which was never merged and predates the connector package refactor (it targeted the old single-file connectors/sources/mongo.py). This PR ports the fix to the current connectors/sources/mongo/ package layout, switches the strategy to clamping, and adds unit test coverage.

• Original PR: Add support for DatetimeMS in MongoDB #3476
• Original issue: MongoDB connector - add support for DatetimeMS objects #3444
• Support case: elastic/sdh-search#1924

Changes

• get_client(): set datetime_conversion=DatetimeConversion.DATETIME_CLAMP.
• serialize(): no special date handling needed — clamped values are ordinary datetime objects and go through the existing datetime -> isoformat() branch.
• Added unit tests: in-range and clamped-bound datetimes serialize to ISO strings, a client-config test asserting DATETIME_CLAMP, and an end-to-end test that encodes an out-of-range BSON date, decodes it with the client codec, and confirms it becomes a clamped datetime serialized as an ISO string (never a DatetimeMS or a number).

Test plan

• make clean install autoformat lint test PYTHON=python3.11 (all MongoDB tests pass; unrelated pre-existing flaky failures only)
• Verify against a MongoDB collection containing out-of-range dates

Release Note

MongoDB connector: set default datetime_conversion to DatetimeConversion.DATETIME_CLAMP so out-of-range datetime values from MongoDB are clamped to valid dates instead of aborting the sync. See https://www.mongodb.com/docs/languages/python/pymongo-driver/current/data-formats/dates-and-times/#handling-out-of-range-datetimes for additional information.

[artem-shelkovnikov]

Just checking - how is it related to the support case?

[Jan-Kazlouski-elastic]

Just checking - how is it related to the support case?

Not directly. Looking into the support case just triggered agent to look into related issues and this was found. I decided it would be a good easy drive by fix.