When running in a managed runtime, on shutdown it appears the SDK registers a shutdown hook, which stops the thread pool before the JGroups channel is stopped which then attempts to recreate the thread pool during shutdown so that it can call remove on the blob.
Azure SDK operation attempted during JVM shutdown (like your AZURE_PING.remove() during WildFly stop) triggers the cascade: addShutdownHookSafely throws → SharedExecutorService.createSharedExecutor fails →
SelectorManager closes → IOException → RequestRetryPolicy retries 4x with backoff sleeps → ~1 minute stall.
16:54:08,394 WARN [com.azure.core.http.jdk.httpclient.JdkHttpClient] (ServerService Thread Pool -- 118) {"az.sdk.message":"The header is restricted by 'java.net.http.HttpClient' and will be ignored. To allow this header to be set on the request, configure 'jdk.httpclient.allowRestrictedHeaders' with the header added in the comma-separated list.","headerName":"content-length"}
16:54:08,395 ERROR [com.azure.core.http.jdk.httpclient.JdkHttpClient] (ServerService Thread Pool -- 118) java.io.IOException: selector manager closed
16:54:08,396 ERROR [com.azure.storage.common.policy.RequestRetryPolicy] (ServerService Thread Pool -- 118) java.io.IOException: selector manager closed
16:54:08,397 ERROR [com.azure.storage.common.implementation.StorageImplUtils] (ServerService Thread Pool -- 118) java.io.IOException: selector manager closed
16:54:08,397 ERROR [com.azure.storage.blob.specialized.BlobClientBase] (ServerService Thread Pool -- 118) java.io.IOException: selector manager closed
16:54:08,397 ERROR [org.jgroups.protocols.azure.AZURE_PING] (ServerService Thread Pool -- 118) Error deleting ping data file 'ejb-0d6b6919-8468-44b7-91b6-61eeda6fb99e.node1.list'.: java.lang.RuntimeException: java.io.IOException: selector manager closed
at com.azure.storage.blob//com.azure.storage.common.implementation.StorageImplUtils.sendRequest(StorageImplUtils.java:539)
at com.azure.storage.blob//com.azure.storage.blob.specialized.BlobClientBase.deleteWithResponse(BlobClientBase.java:1629)
at com.azure.storage.blob//com.azure.storage.blob.specialized.BlobClientBase.deleteIfExistsWithResponse(BlobClientBase.java:1691)
at com.azure.storage.blob//com.azure.storage.blob.specialized.BlobClientBase.deleteIfExists(BlobClientBase.java:1652)
at org.jgroups.azure@3.0.0.Final//org.jgroups.protocols.azure.AZURE_PING.remove(AZURE_PING.java:252)
at org.jgroups@5.5.4.Final//org.jgroups.protocols.FILE_PING.stop(FILE_PING.java:139)
at java.base/java.util.ArrayList.forEach(ArrayList.java:1511)
at org.jgroups@5.5.4.Final//org.jgroups.stack.ProtocolStack.stopStack(ProtocolStack.java:930)
at org.jgroups@5.5.4.Final//org.jgroups.JChannel.stopStack(JChannel.java:912)
at org.jgroups@5.5.4.Final//org.jgroups.JChannel.disconnect(JChannel.java:413)
at org.jboss.as.clustering.jgroups@40.0.0.Beta1-SNAPSHOT//org.jboss.as.clustering.jgroups.subsystem.AbstractChannelResourceDefinitionRegistrar$4$1.stop(AbstractChannelResourceDefinitionRegistrar.java:291)
at java.base/java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1804)
at org.jboss.threads@3.9.2//org.jboss.threads.ContextHandler$1.runWith(ContextHandler.java:18)
at org.jboss.threads@3.9.2//org.jboss.threads.EnhancedQueueExecutor$Task.doRunWith(EnhancedQueueExecutor.java:2651)
at org.jboss.threads@3.9.2//org.jboss.threads.EnhancedQueueExecutor$Task.run(EnhancedQueueExecutor.java:2630)
at org.jboss.threads@3.9.2//org.jboss.threads.EnhancedQueueExecutor.runThreadBody(EnhancedQueueExecutor.java:1622)
at org.jboss.threads@3.9.2//org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1589)
at java.base/java.lang.Thread.run(Thread.java:840)
at org.jboss.threads@3.9.2//org.jboss.threads.JBossThread.run(JBossThread.java:501)
Caused by: java.io.IOException: selector manager closed
at java.net.http/jdk.internal.net.http.HttpClientImpl.send(HttpClientImpl.java:917)
at java.net.http/jdk.internal.net.http.HttpClientFacade.send(HttpClientFacade.java:133)
at com.azure.storage.blob//com.azure.core.http.jdk.httpclient.JdkHttpClient.sendSync(JdkHttpClient.java:132)
at com.azure.storage.blob//com.azure.core.http.HttpPipelineNextSyncPolicy.processSync(HttpPipelineNextSyncPolicy.java:51)
at com.azure.storage.blob//com.azure.storage.common.policy.ScrubEtagPolicy.processSync(ScrubEtagPolicy.java:35)
at com.azure.storage.blob//com.azure.core.http.HttpPipelineNextSyncPolicy.processSync(HttpPipelineNextSyncPolicy.java:53)
at com.azure.storage.blob//com.azure.core.http.policy.HttpLoggingPolicy.processSync(HttpLoggingPolicy.java:175)
at com.azure.storage.blob//com.azure.core.http.HttpPipelineNextSyncPolicy.processSync(HttpPipelineNextSyncPolicy.java:53)
at com.azure.storage.blob//com.azure.storage.common.policy.ResponseValidationPolicyBuilder$ResponseValidationPolicy.processSync(ResponseValidationPolicyBuilder.java:140)
at com.azure.storage.blob//com.azure.core.http.HttpPipelineNextSyncPolicy.processSync(HttpPipelineNextSyncPolicy.java:53)
at com.azure.storage.blob//com.azure.core.implementation.http.policy.InstrumentationPolicy.processSync(InstrumentationPolicy.java:101)
at com.azure.storage.blob//com.azure.core.http.HttpPipelineNextSyncPolicy.processSync(HttpPipelineNextSyncPolicy.java:53)
at com.azure.storage.blob//com.azure.core.http.policy.BearerTokenAuthenticationPolicy.processSync(BearerTokenAuthenticationPolicy.java:175)
at com.azure.storage.blob//com.azure.core.http.HttpPipelineNextSyncPolicy.processSync(HttpPipelineNextSyncPolicy.java:53)
at com.azure.storage.blob//com.azure.storage.common.policy.MetadataValidationPolicy.processSync(MetadataValidationPolicy.java:42)
at com.azure.storage.blob//com.azure.core.http.HttpPipelineNextSyncPolicy.processSync(HttpPipelineNextSyncPolicy.java:53)
at com.azure.storage.blob//com.azure.core.http.policy.AddHeadersFromContextPolicy.processSync(AddHeadersFromContextPolicy.java:67)
at com.azure.storage.blob//com.azure.core.http.HttpPipelineNextSyncPolicy.processSync(HttpPipelineNextSyncPolicy.java:53)
at com.azure.storage.blob//com.azure.core.http.policy.AddDatePolicy.processSync(AddDatePolicy.java:50)
at com.azure.storage.blob//com.azure.core.http.HttpPipelineNextSyncPolicy.processSync(HttpPipelineNextSyncPolicy.java:53)
at com.azure.storage.blob//com.azure.storage.common.policy.RequestRetryPolicy.lambda$attemptSync$0(RequestRetryPolicy.java:263)
at com.azure.storage.blob//reactor.core.publisher.MonoCallable.block(MonoCallable.java:62)
at com.azure.storage.blob//reactor.core.publisher.MonoCallable.block(MonoCallable.java:55)
at com.azure.storage.blob//com.azure.storage.common.policy.RequestRetryPolicy.attemptSync(RequestRetryPolicy.java:272)
at com.azure.storage.blob//com.azure.storage.common.policy.RequestRetryPolicy.attemptSync(RequestRetryPolicy.java:321)
at com.azure.storage.blob//com.azure.storage.common.policy.RequestRetryPolicy.attemptSync(RequestRetryPolicy.java:321)
at com.azure.storage.blob//com.azure.storage.common.policy.RequestRetryPolicy.attemptSync(RequestRetryPolicy.java:321)
at com.azure.storage.blob//com.azure.storage.common.policy.RequestRetryPolicy.processSync(RequestRetryPolicy.java:73)
at com.azure.storage.blob//com.azure.core.http.HttpPipelineNextSyncPolicy.processSync(HttpPipelineNextSyncPolicy.java:53)
at com.azure.storage.blob//com.azure.core.http.policy.RequestIdPolicy.processSync(RequestIdPolicy.java:77)
at com.azure.storage.blob//com.azure.core.http.HttpPipelineNextSyncPolicy.processSync(HttpPipelineNextSyncPolicy.java:53)
at com.azure.storage.blob//com.azure.core.http.policy.HttpPipelineSyncPolicy.processSync(HttpPipelineSyncPolicy.java:51)
at com.azure.storage.blob//com.azure.core.http.policy.UserAgentPolicy.processSync(UserAgentPolicy.java:174)
at com.azure.storage.blob//com.azure.core.http.HttpPipelineNextSyncPolicy.processSync(HttpPipelineNextSyncPolicy.java:53)
at com.azure.storage.blob//com.azure.core.http.HttpPipeline.sendSync(HttpPipeline.java:138)
at com.azure.storage.blob//com.azure.core.implementation.http.rest.SyncRestProxy.send(SyncRestProxy.java:62)
at com.azure.storage.blob//com.azure.core.implementation.http.rest.SyncRestProxy.invoke(SyncRestProxy.java:83)
at com.azure.storage.blob//com.azure.core.implementation.http.rest.RestProxyBase.invoke(RestProxyBase.java:124)
at com.azure.storage.blob//com.azure.core.http.rest.RestProxy.invoke(RestProxy.java:95)
at com.azure.storage.blob/jdk.proxy11/jdk.proxy11.$Proxy33.deleteNoCustomHeadersSync(Unknown Source)
at com.azure.storage.blob//com.azure.storage.blob.implementation.BlobsImpl.deleteNoCustomHeadersWithResponse(BlobsImpl.java:3167)
at com.azure.storage.blob//com.azure.storage.blob.specialized.BlobClientBase.lambda$deleteWithResponse$0(BlobClientBase.java:1623)
at com.azure.storage.blob//com.azure.storage.common.implementation.StorageImplUtils.sendRequest(StorageImplUtils.java:522)
... 18 more
Caused by: java.io.IOException: selector manager closed
at java.net.http/jdk.internal.net.http.HttpClientImpl$SelectorManager.selectorClosedException(HttpClientImpl.java:1073)
at java.net.http/jdk.internal.net.http.HttpClientImpl.sendAsync(HttpClientImpl.java:954)
at java.net.http/jdk.internal.net.http.HttpClientImpl.send(HttpClientImpl.java:878)
... 60 more
Caused by: java.lang.IllegalStateException: Shutdown in progress
at java.base/java.lang.ApplicationShutdownHooks.add(ApplicationShutdownHooks.java:66)
at java.base/java.lang.Runtime.addShutdownHook(Runtime.java:216)
at com.azure.storage.blob//com.azure.core.implementation.ImplUtils.addShutdownHookSafely(ImplUtils.java:525)
at com.azure.storage.blob//com.azure.core.util.CoreUtils.addShutdownHookSafely(CoreUtils.java:666)
at com.azure.storage.blob//com.azure.core.util.SharedExecutorService.createSharedExecutor(SharedExecutorService.java:388)
at com.azure.storage.blob//com.azure.core.util.SharedExecutorService.lambda$ensureNotShutdown$0(SharedExecutorService.java:364)
at java.base/java.util.concurrent.atomic.AtomicReferenceFieldUpdater.updateAndGet(AtomicReferenceFieldUpdater.java:240)
at com.azure.storage.blob//com.azure.core.util.SharedExecutorService.ensureNotShutdown(SharedExecutorService.java:363)
at com.azure.storage.blob//com.azure.core.util.SharedExecutorService.execute(SharedExecutorService.java:302)
at java.net.http/jdk.internal.net.http.HttpClientImpl$DelegatingExecutor.ensureExecutedAsync(HttpClientImpl.java:180)
at java.net.http/jdk.internal.net.http.HttpClientImpl$DelegatingExecutor.execute(HttpClientImpl.java:172)
at java.net.http/jdk.internal.net.http.common.SequentialScheduler.runOrSchedule(SequentialScheduler.java:305)
at java.net.http/jdk.internal.net.http.common.SequentialScheduler.runOrSchedule(SequentialScheduler.java:274)
at java.net.http/jdk.internal.net.http.common.SSLFlowDelegate$Reader.incoming(SSLFlowDelegate.java:316)
at java.net.http/jdk.internal.net.http.common.SubscriberWrapper.incomingCaller(SubscriberWrapper.java:438)
at java.net.http/jdk.internal.net.http.common.SubscriberWrapper.onNext(SubscriberWrapper.java:379)
at java.net.http/jdk.internal.net.http.common.SubscriberWrapper.onNext(SubscriberWrapper.java:57)
at java.net.http/jdk.internal.net.http.SocketTube$InternalReadPublisher$InternalReadSubscription.read(SocketTube.java:865)
at java.net.http/jdk.internal.net.http.SocketTube$SocketFlowTask.run(SocketTube.java:181)
at java.net.http/jdk.internal.net.http.common.SequentialScheduler$SchedulableTask.run(SequentialScheduler.java:230)
at java.net.http/jdk.internal.net.http.common.SequentialScheduler.runOrSchedule(SequentialScheduler.java:303)
at java.net.http/jdk.internal.net.http.common.SequentialScheduler.runOrSchedule(SequentialScheduler.java:256)
at java.net.http/jdk.internal.net.http.SocketTube$InternalReadPublisher$InternalReadSubscription.signalReadable(SocketTube.java:782)
at java.net.http/jdk.internal.net.http.SocketTube$InternalReadPublisher$ReadEvent.signalEvent(SocketTube.java:965)
at java.net.http/jdk.internal.net.http.SocketTube$SocketFlowEvent.handle(SocketTube.java:253)
at java.net.http/jdk.internal.net.http.HttpClientImpl$SelectorManager.handleEvent(HttpClientImpl.java:1402)
at java.net.http/jdk.internal.net.http.HttpClientImpl$SelectorManager.lambda$run$3(HttpClientImpl.java:1347)
at java.base/java.util.ArrayList.forEach(ArrayList.java:1511)
at java.net.http/jdk.internal.net.http.HttpClientImpl$SelectorManager.run(HttpClientImpl.java:1347)
16:54:08,406 INFO [org.jboss.as.clustering.jgroups] (ServerService Thread Pool -- 118) WFLYCLJG0035: Disconnected 'ee' channel. 'node1' left cluster 'ejb'
16:54:08,412 INFO [org.jboss.as] (MSC service thread 1-3) WFLYSRV0050: WildFly 40.0.0.Beta1-SNAPSHOT (WildFly Core 32.0.0.Beta7) stopped in 44148ms
When running in a managed runtime, on shutdown it appears the SDK registers a shutdown hook, which stops the thread pool before the JGroups channel is stopped which then attempts to recreate the thread pool during shutdown so that it can call remove on the blob.
Azure SDK operation attempted during JVM shutdown (like your AZURE_PING.remove() during WildFly stop) triggers the cascade: addShutdownHookSafely throws → SharedExecutorService.createSharedExecutor fails →
SelectorManager closes → IOException → RequestRetryPolicy retries 4x with backoff sleeps → ~1 minute stall.