Skip to content

CASSANALYTICS-177: Add instanceId query parameter to fix HTTP 421 err… - #222

Open
bianca-stanciu29 wants to merge 4 commits into
apache:trunkfrom
bianca-stanciu29:CASSANALYTICS-177
Open

CASSANALYTICS-177: Add instanceId query parameter to fix HTTP 421 err…#222
bianca-stanciu29 wants to merge 4 commits into
apache:trunkfrom
bianca-stanciu29:CASSANALYTICS-177

Conversation

@bianca-stanciu29

Copy link
Copy Markdown

…ors when Sidecar is behind a load balancer

@skoppu22 skoppu22 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This PR stamps the same instanceId on every request — so if the job writes to more than one instance, all but one get mislabeled.

Example:
Setup (a normal 3-node bulk write)
Cassandra ring: instance 1 | instance 2 | instance 3
Sidecar (one config knows all 3, ids 1/2/3), fronted by a load balancer at sidecar-lb:9043
Operator sets: spark.cassandra_analytics.sidecar.instance.id = 2
The writer builds ONE client over all three (CassandraContext.java:90):

AnalyticsSidecarClient.from(new SimpleSidecarInstancesProvider(clusterConfig /* 3 instances */), conf)
Copy
What happens per request (with this PR)
request meant for instance 1 ──▶ ...?instanceId=2 ──▶ server: instanceFromId(2) ──▶ instance 2 ❌ WRONG
request meant for instance 2 ──▶ ...?instanceId=2 ──▶ server: instanceFromId(2) ──▶ instance 2 ✅ right
request meant for instance 3 ──▶ ...?instanceId=2 ──▶ server: instanceFromId(2) ──▶ instance 2 ❌ WRONG
The ?instanceId=2 overrides the Host header on the server, so the server confidently routes the instance-1 and instance-3 work to instance 2. Because config.instanceId() is a single job-global value, vertxRequest(...) applies it uniformly even though it already knows the real target (sidecarInstance) and could distinguish them.

Net: a fixed id is only correct when the job touches exactly one instance. For any real multi-node ring it silently misroutes → the data-correctness risk behind the WARNING.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants