fix(connectors): Harden Elasticsearch sink test readiness#3729
fix(connectors): Harden Elasticsearch sink test readiness#3729ryerraguntla wants to merge 4 commits into
Conversation
Add a timeout to the Elasticsearch sink transport, make connectors-runtime readiness check `/health`, and strengthen the Elasticsearch test fixture startup path with retry, cluster-health polling, and stale-index cleanup. Also refresh the sink index before counting documents so test assertions see newly indexed data sooner.
Use short-timeout probes without retries for cluster health and stale index sweep, and cap docker rm so a wedged daemon cannot stall setup.
|
Thanks for the PR. It is labeled Slash commands (own line, regular comment) move it around the queue:
See CONTRIBUTING.md for details. |
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## master #3729 +/- ##
=============================================
- Coverage 74.34% 37.82% -36.53%
Complexity 950 950
=============================================
Files 1303 1301 -2
Lines 148712 129057 -19655
Branches 124225 104570 -19655
=============================================
- Hits 110562 48811 -61751
- Misses 34673 77640 +42967
+ Partials 3477 2606 -871
🚀 New features to boost your workflow:
|
|
/request-review @user-or-team |
There was a problem hiding this comment.
start() force-removes the shared container after any try_start() error, not only after confirming it is wedged. Transient health, port-inspection, or Docker errors can therefore remove an Elasticsearch container still used by another process. The fixed name limits which container is removed but does not provide ownership or concurrency safety. Could this recovery path verify the container is unhealthy and coordinate removal with a cross-process lock before running docker rm -f?
There was a problem hiding this comment.
Let me add remove when required.
Only docker rm the shared reuse container after inspect shows a removable state, and serialize recovery with a cross-process lock so transient start flakes cannot yank a healthy instance from a peer nextest worker.
Summary
iggy-connectorsnever becomes healthy because sink
open()can hang on Elasticsearch HTTPwith no client timeout, while the connectors HTTP API only binds after
plugin open.
127.0.0.1, alwaysre-check
/_cluster/healthon reuse attach, self-heal a wedgediggy-test-elasticsearchcontainer only, and sweep stale harness indicesolder than 30 minutes.
/healthand fail fast if thechild process dies during retries; refresh the index before document-count
polls in the sink fixture.
Closes #3728
Test plan
cargo fmt --allcargo sort --no-format --workspacecargo clippy -p iggy_connector_elasticsearch_sink -p integration --all-features --all-targets -- -D warningscargo test -p integration -- connectors::elasticsearch::elasticsearch_sink::elasticsearch_sink_stores_json_messagesdocker rm -frecovery targets onlyiggy-test-elasticsearch(no other Elasticsearch containers)
iggy-test-elasticsearchrunning and re-runthe sink test to verify reuse + readiness path
Notes
iggy-test-elasticsearchonly.tests sharing the reused container are not disrupted.