Skip to content

Remove pending acquirePromise once chained responseFuture fails #1115

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
edwardxia wants to merge 1 commit into
base: main
Choose a base branch
from
Open

Remove pending acquirePromise once chained responseFuture fails #1115

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions pushy/src/main/java/com/eatthepath/pushy/apns/ApnsChannelPool.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -178,6 +178,14 @@ private void acquireWithinEventExecutor(final Promise<Channel> acquirePromise) {
this.acquireWithinEventExecutor(acquirePromise);
}
} else {
// If the deferred acquirePromise is cancelled, remove it from pendingAcquisitionPromises
// to allow garbage collection.
acquirePromise.addListener(future -> {
if (future.isCancelled()) {
pendingAcquisitionPromises.remove(acquirePromise);
}
});

// We don't have any connections ready to go, and don't have any more capacity to create new
// channels. Add this acquisition to the queue waiting for channels to become available.
pendingAcquisitionPromises.add(acquirePromise);
Expand Down
9 changes: 8 additions & 1 deletion pushy/src/main/java/com/eatthepath/pushy/apns/ApnsClient.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -184,7 +184,8 @@ public <T extends ApnsPushNotification> PushNotificationFuture<T, PushNotificati
if (!this.isClosed.get()) {
final long start = System.nanoTime();

this.channelPool.acquire().addListener((GenericFutureListener<Future<Channel>>) acquireFuture -> {
Future<Channel> acquirePromise = this.channelPool.acquire();
acquirePromise.addListener((GenericFutureListener<Future<Channel>>) acquireFuture -> {
if (acquireFuture.isSuccess()) {
final Channel channel = acquireFuture.getNow();

Expand All @@ -207,6 +208,12 @@ public <T extends ApnsPushNotification> PushNotificationFuture<T, PushNotificati
ApnsClient.this.metricsListener.handleNotificationAcknowledged(response, end - start);
} else {
ApnsClient.this.metricsListener.handleWriteFailure(notification.getTopic());

// Try cancel acquirePromise if responseFuture fails (e.g. TimeoutException)
// before acquirePromise gets resolved.
if (acquirePromise.isCancellable()) {
acquirePromise.cancel(true);
}
Comment on lines +212 to +216

@jchambers jchambers May 24, 2026

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This doesn't make sense to me; if we're in the acquirePromise's completion listener, doesn't that by definition mean that the acquirePromise has completed by some means and is therefore not cancellable?

@edwardxia edwardxia May 26, 2026

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

acquirePromise would hang forever when apple server is not reachable. because these promises do not have any timeout on them. responseFuture timeout would not complete the promise it listens on.

@jchambers jchambers May 26, 2026

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

these promises do not have any timeout on them.

They do if you configure a timeout, right?

pushy/pushy/src/main/java/com/eatthepath/pushy/apns/ApnsChannelFactory.java

Lines 88 to 89 in 7bd7c80

clientConfiguration.getConnectionTimeout().ifPresent(timeout ->
this.bootstrapTemplate.option(ChannelOption.CONNECT_TIMEOUT_MILLIS, (int) timeout.toMillis()));

@edwardxia edwardxia May 26, 2026

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Netty has CONNECT_TIMEOUT_MILLIS default 30s when it's not configured, but we saw these promises stuck in memory forever until JVM crashes. This option is likely not working.

I don't have time to investigate but this might be related: netty/netty#9399

@jchambers jchambers May 27, 2026 •
edited
Loading

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This option is likely not working… I don't have time to investigate but this might be related: netty/netty#9399

The issue you linked stems from somebody blocking their event loop, and I don't think it's relevant here. I also don't think it's reasonable to conclude that connection timeouts aren't working based on the evidence we've seen here. I want to be clear that I believe you had a problem and I'm interested in figuring out what went wrong, but this doesn't seem like the explanation.

Taking a step back, I think we've gotten a little sidetracked. Again, emphasizing that it would be good to understand and fix whatever went wrong for you, I still don't understand this proposed fix. The key problem for me is that it appears that we're trying to cancel the acquisition promise after it has already completed. Do you agree with that assessment? If not, can you please explain why?

}
});
} else {
Expand Down