Skip to content

CASSJAVA-145 Reduce CPU cost of DefaultLoadBalancingPolicy query plans - #2105

Open
shawjef3 wants to merge 1 commit into
apache:4.xfrom
shawjef3:default-lbp-query-plan-cpu
Open

shawjef3 wants to merge 1 commit into
apache:4.xfrom
shawjef3:default-lbp-query-plan-cpu

Conversation

@shawjef3

@shawjef3 shawjef3 commented Nov 3, 2026

Copy link
Copy Markdown

patch by Jeff Shaw

I removed ArrayUtils.rotate() since it is unused after this PR. It's a public interface change so it breaks binary compatibility. I'll put it back if it's better to keep it.

patch by Jeff Shaw

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@shawjef3
shawjef3 force-pushed the default-lbp-query-plan-cpu branch from 8937cb1 to 0393789 Compare November 4, 2026 14:42
@shawjef3

shawjef3 commented Nov 4, 2026

Copy link
Copy Markdown
Author

I tested this in a production-like environment and it increased throughput by 15 to 20%.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

No active review issues remain for the query-plan construction optimizations.

No open issues 🎉

What changed in this PR

This PR reduces query-plan construction costs in the Java driver’s basic and default load-balancing policies while preserving node ordering.

Changes:

  • Introduces lazy round-robin rotation and stops replica searches once all replicas are found.
  • Uses array copies for node movement, removes the unused ArrayUtils.rotate() method, and avoids unnecessary in-flight request checks.
  • Adds regression tests for ordering and optimized operations, plus a changelog entry.
File Description
core/​src/​test/​java/​com/​datastax/​oss/​driver/​internal/​core/​util/​collection/​RotatedQueryPlanTest.java Tests rotation, polling, and iteration against previous behavior.
core/​src/​test/​java/​com/​datastax/​oss/​driver/​internal/​core/​util/​ArrayUtilsTest.java Verifies optimized node movement matches the old implementation.
core/​src/​test/​java/​com/​datastax/​oss/​driver/​internal/​core/​loadbalancing/​DefaultLoadBalancingPolicyQueryPlanTest.java Tests in-flight check counts under different response rates.
core/​src/​test/​java/​com/​datastax/​oss/​driver/​internal/​core/​loadbalancing/​BasicLoadBalancingPolicyQueryPlanTest.java Verifies replica searches stop early.
core/​src/​main/​java/​com/​datastax/​oss/​driver/​internal/​core/​util/​collection/​RotatedQueryPlan.java Adds query plans that map rotated positions without moving nodes.
core/​src/​main/​java/​com/​datastax/​oss/​driver/​internal/​core/​util/​ArrayUtils.java Optimizes node movement and removes eager rotation.
core/​src/​main/​java/​com/​datastax/​oss/​driver/​internal/​core/​loadbalancing/​DefaultLoadBalancingPolicy.java Uses lazy rotation, bounded replica searches, and reordered health checks.
core/​src/​main/​java/​com/​datastax/​oss/​driver/​internal/​core/​loadbalancing/​BasicLoadBalancingPolicy.java Uses lazy rotation and bounded replica searches.
changelog/​README.md Records the query-plan performance improvement.

Review effort: Balanced

QueryPlan plan =
currentNodes.length == 0
? QueryPlan.EMPTY
: new RotatedQueryPlan(

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for the contribution! My first thought is - can you achieve the same by changing the implementation of ArrayUtils.rotate() instead of a new RotateQueryPlan class?

@shawjef3 shawjef3 Nov 9, 2026 •

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You're welcome. I'm happy to help! RotateQueryPlan has the benefit of not doing any work for elements that aren't needed. Merely changing ArrayUtils.rotate() would not give that benefit.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I took a look and I agree that a new class might be the best approach here. The best performance is when we never actually rotate the array but using the mapped virtual address. I tried out whether I can achieve the same without a new class, but that means I have to basically implement an alternative constructor of QueryPlanBase and SimpleQueryPlan, because both classes are used elsewhere without rotation.... so it became even more complex.

@SiyaoIsHiding SiyaoIsHiding left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I took a deeper look in this. I think this is a great improvement! Thank you!

I see there are four independent optimization:

  1. Move replicas to the beginning, stopping once all of them have been found
  2. The array rotation
  3. Swap isBusy(node, session) && isResponseRateInsufficient(node, now)
  4. bubbleUp and bubbleDown implementation.

Would you please run performance testing for only each one of them separately? Especially for 1️⃣ and 3️⃣ , I'm not fully convinced that they improve performance for most users in common production cases without bringing regression.

I believe 2️⃣ and 4️⃣ will help a lot tho :) Thank you again for the fix!

if (sourceIndex > targetIndex) {
ElementT element = elements[sourceIndex];
System.arraycopy(elements, targetIndex, elements, targetIndex + 1, sourceIndex - targetIndex);
elements[targetIndex] = element;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Did my research and learnt that this is indeed faster and generally preferred, cuz it's optimized at the JVM level. https://www.examclouds.com/java/java-core/system-arraycopy-example

Comment thread changelog/README.md

<!-- Note: contrary to 3.x, insert new entries *first* in their section -->

### 4.19.5 (in progress)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We usually only add to changelog when we release, so this can be skipped in this PR

QueryPlan plan =
currentNodes.length == 0
? QueryPlan.EMPTY
: new RotatedQueryPlan(

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I took a look and I agree that a new class might be the best approach here. The best performance is when we never actually rotate the array but using the mapped virtual address. I tried out whether I can achieve the same without a new class, but that means I have to basically implement an alternative constructor of QueryPlanBase and SimpleQueryPlan, because both classes are used elsewhere without rotation.... so it became even more complex.

// Move replicas to the beginning
for (int i = 0; i < currentNodes.length; i++) {
// Move replicas to the beginning, stopping once all of them have been found
int allReplicasCount = allReplicas.size();

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this allReplicas.size() for all replicas in the whole cluster or all replicas in the local DC?

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

⌂ Proxy