CASSANALYTICS-177: Add instanceId query parameter to fix HTTP 421 err… - #222
CASSANALYTICS-177: Add instanceId query parameter to fix HTTP 421 err…#222bianca-stanciu29 wants to merge 4 commits into
Conversation
…ors when Sidecar is behind a load balancer
skoppu22
left a comment
There was a problem hiding this comment.
This PR stamps the same instanceId on every request — so if the job writes to more than one instance, all but one get mislabeled.
Example:
Setup (a normal 3-node bulk write)
Cassandra ring: instance 1 | instance 2 | instance 3
Sidecar (one config knows all 3, ids 1/2/3), fronted by a load balancer at sidecar-lb:9043
Operator sets: spark.cassandra_analytics.sidecar.instance.id = 2
The writer builds ONE client over all three (CassandraContext.java:90):
AnalyticsSidecarClient.from(new SimpleSidecarInstancesProvider(clusterConfig /* 3 instances */), conf)
Copy
What happens per request (with this PR)
request meant for instance 1 ──▶ ...?instanceId=2 ──▶ server: instanceFromId(2) ──▶ instance 2 ❌ WRONG
request meant for instance 2 ──▶ ...?instanceId=2 ──▶ server: instanceFromId(2) ──▶ instance 2 ✅ right
request meant for instance 3 ──▶ ...?instanceId=2 ──▶ server: instanceFromId(2) ──▶ instance 2 ❌ WRONG
The ?instanceId=2 overrides the Host header on the server, so the server confidently routes the instance-1 and instance-3 work to instance 2. Because config.instanceId() is a single job-global value, vertxRequest(...) applies it uniformly even though it already knows the real target (sidecarInstance) and could distinguish them.
Net: a fixed id is only correct when the job touches exactly one instance. For any real multi-node ring it silently misroutes → the data-correctness risk behind the WARNING.
… misrouting on multi-node bulk write jobs
…ors when Sidecar is behind a load balancer