[CASSANALYTICS-181] - Support Sidecar behind a load balancer for time… - #223
[CASSANALYTICS-181] - Support Sidecar behind a load balancer for time…#223bianca-stanciu29 wants to merge 1 commit into
Conversation
…-skew validation in coordinated writes
| if (conf.sidecarBehindLoadBalancer) | ||
| { | ||
| LOGGER.info("Using CoordinatedCassandraClusterInfo for load-balanced Sidecar. clusterId={}", clusterId); | ||
| return new CoordinatedCassandraClusterInfo(conf, clusterId); |
There was a problem hiding this comment.
This name CoordinatedCassandraClusterInfo is misleading. For the reader, this naming sounds like Coordinated work is happening only when sidecarBehindLoadBalancer is true, and else case below doing non-coordinated work. Need to pick appropriate name instead of CoordinatedCassandraClusterInfo
| private static CassandraClusterInfo createClusterInfo(BroadcastableClusterInfo bci) | ||
| { | ||
| return bci.getConf().sidecarBehindLoadBalancer | ||
| ? new CoordinatedCassandraClusterInfo(bci) |
| @Override | ||
| protected CompletableFuture<TimeSkewResponse> fetchTimeSkew(Range<BigInteger> range) | ||
| { | ||
| return getCassandraContext().getSidecarClient().timeSkew(); |
There was a problem hiding this comment.
The same problem exists for Single cluster behind a load balancer. Hence instead of using 'Coordinated' word, good to develop this as a generic framework for handling cluster/clusters behind a load balancer, then we can invoke it from anywhere needed
There was a problem hiding this comment.
Also, range param is unused, hence mention a comment something like 'range is irrelevant; contact points are queried directly'
| this.coordinatedWriteConf = buildCoordinatedWriteConf(dataTransportInfo.getTransport(), logger); | ||
| if (this.sidecarBehindLoadBalancer && this.coordinatedWriteConf == null && logger != null) | ||
| { | ||
| logger.warn("{} is set but {} is not configured; the flag is only honored for coordinated writes and will be ignored on the single-cluster path.", |
There was a problem hiding this comment.
What about single cluster behind a load balancer? Can you create a Jira for supporting this case and add Jira here as a TODO ?
There was a problem hiding this comment.
I would recommend making LB handling as generic framework and invoke it from both coordinated and single cluster case, then we will have uniformity and support for both cases.
| * {@link org.apache.cassandra.spark.bulkwriter.WriterOptions#SIDECAR_BEHIND_LOAD_BALANCER} | ||
| * is set. | ||
| */ | ||
| public class CoordinatedCassandraClusterInfo extends CassandraClusterInfo |
There was a problem hiding this comment.
As I mentioned above, this naming is incorrect, this class is for Cluster behind a LB, nothing specific to Corodinated
2 participants
…-skew validation in coordinated writes
Problem
Time-skew validation before a coordinated bulk write fans out HTTP calls directly to per-replica FQDNs resolved from the Cassandra token map. In deployments where Sidecar is fronted by a load balancer, those FQDNs are not routable from Spark executors, only the load balancer address is. This causes time-skew validation to fail at the start of every coordinated write job, blocking bulk writes entirely in load-balanced Sidecar deployments.
Solution
Add a SIDECAR_BEHIND_LOAD_BALANCER writer option (default: false). When true, time-skew validation skips the per-replica fan-out and routes through the preconfigured load balancer contact points instead