From 622b392b7667e24e3999bbd1c1e425cc5026dd47 Mon Sep 17 00:00:00 2001 From: wind57 Date: Wed, 15 Oct 2025 18:33:08 +0300 Subject: [PATCH 01/16] start with documentation Signed-off-by: wind57 --- docs/modules/ROOT/pages/leader-election.adoc | 86 ++++++++++++++++++++ 1 file changed, 86 insertions(+) diff --git a/docs/modules/ROOT/pages/leader-election.adoc b/docs/modules/ROOT/pages/leader-election.adoc index bc0d920a77..42730e6c54 100644 --- a/docs/modules/ROOT/pages/leader-election.adoc +++ b/docs/modules/ROOT/pages/leader-election.adoc @@ -25,3 +25,89 @@ To specify the name of the configmap used for leader election use the following ---- spring.cloud.kubernetes.leader.config-map-name=leader ---- + + +''' + +There is another way you can configure leader election, and it comes with native support in the fabric8 library (k8s native client support is not yet implemented). In the long run, this will be the default way to configure leader election, while the previous one will be dropped. You can treat this one much like the JDKs "preview" features. + +To be able to use it, you need to set the property: + +[source] +---- +spring.cloud.kubernetes.leader.election.enabled=true +---- + +Unlike the old implementation, this one will use either the `Lease` _or_ `ConfigMap` as the lock, depending on your cluster version. You can force using configMap still, even if leases are supported, via : + +[source] +---- +spring.cloud.kubernetes.leader.election.use-config-map-as-lock=true +---- + +The name of that `Lease` or `ConfigMap` can be defined using the property (default value is `spring-k8s-leader-election-lock`): + +[source] +---- +spring.cloud.kubernetes.leader.election.lockName=other-name +---- + +The namespace where the lock is created (`default` being set if no explicit one exists) can be set also: + +[source] +---- +spring.cloud.kubernetes.leader.election.lockNamespace=other-namespace +---- + +Before the leader election process kicks in, you can wait until the pod is ready (via the readiness check). This is enabled by default, but you can disable it if needed: + +[source] +---- +spring.cloud.kubernetes.leader.election.waitForPodReady=false +---- + +Like with the old implementation, we will publish events by default, but this can be disabled: + +[source] +---- +spring.cloud.kubernetes.leader.election.publishEvents=false +---- + +There are a few parameters that control how the leader election process will happen. To explain them, we need to look at the high-level implementation of this process. All the candidate pods try to become the leader, or they try to _acquire_ the lock. If the lock is already taken, they will continue to retry to acquire it every `spring.cloud.kubernetes.leader.election.retryPeriod` (value is specified as `java.time.Duration`, and by default it is 2 seconds). + +If the lock is not taken, current pod becomes the leader. It does so by inserting a so-called "record" into the lock (`Lease` or `ConfigMap`). Among the things that the "record" contains, is the `leaseDuration` (that you can specify via `spring.cloud.kubernetes.leader.election.leaseDuration`; by default it is 15 seconds and is of type `java.time.Duration`). This acts like a TTL on the lock: no other candidate can acquire the lock, unless this period has expired (from the last renewal time). + +Once a certain pod establishes itself as the leader (by acquiring the lock), it will continuously (every `spring.cloud.kubernetes.leader.election.retryPeriod`) try to renew its lease, or in other words: it will try to extend its leadership. When a renewal happens, the "record" that is stored inside the lock, is updated. For example, `renewTime` is updated inside the record, to denote when the last renewal happened. (You can always peek inside these fields by using `kubectl describe lease...` for example). + +Renewal must happen within a certain interval, specified by `spring.cloud.kubernetes.leader.election.renewDeadline`. By default, it is equal to 10 seconds, and it means that the leader pod has a maximum of 10 seconds to renew its leadership. If that does not happen, this pod loses its leadership and leader election starts again. Because other pods try to become leaders every 2 seconds (by default), it could mean that the pod that just lost leadership, will become leader again. If you want other pods to have a higher chance of becoming leaders, you can set the property (specified in seconds, by default it is 0) : + +[source] +---- +spring.cloud.kubernetes.leader.election.wait-after-renewal-failure=3 +---- + +This will mean that the pod (that could not renew its lease) and lost leadership, will wait this many seconds, before trying to become leader again. + +Let's try to explain these settings based on an example: there are two pods that participate in leader election. For simplicity let's call them `podA` and `podB`. They both start at the same time: `12:00:00`, but `podA` establishes itself as the leader. This means that every two seconds (`retryPeriod`), `podB` will try to become the new leader. So at `12:00:02`, then at `12:00:04` and so on, it will basically ask : "Can I become the leader?". In our simplified example, the answer to that question can be answered based on `podA` activity. + +After `podA` has become the leader, at every 2 seconds, it will try to "extend" or _renew_ its leadership. So at `12:00:02`, then at `12:00:04` and so on, `podA` goes to the lock and updates its record to reflect that it is still the leader. Between the last successful renewal and the next one, it has exactly 10 seconds (`renewalDeadline`). If it fails to renew its leadership (there is a connection problem or a big GC pause, etc.) within those 10 seconds, it stops leading and `podB` can acquire the leadership now. When `podA` stops being a leader in a graceful way, the lock record is "cleared", basically meaning that `podB` can acquire leadership immediately. + +A different story happens when `podA` dies with an OutOfMemory for example, without being able to gracefully update lock record and this is when `leaseDuration` argument matters. The easiest way to explain is via an example: + +`podA` has renewed its leadership at `12:00:04`, but at `12:00:05` it has been killed by the OOMKiller. At `12:00:06`, `podB` will try to become the leader. It will check if "now" (`12:00:06`) is _after_ last renewal + lease duration, essentially it will check: + +[source] +---- +12:00:06 > (12:00:04 + 00:00:10) +---- + +The condition is not fulfilled, so it can't become the leader. Same result will be at `12:00:08`, `12:00:10` and so on, until `12:00:16` and this is where the TTL (`leaseDuration`) of the lock will expire and `podB` can acquire it. As such, a lower value of `leaseDuration` will mean a faster acquiring of leadership by other pods. + +You might have to give proper RBAC to be able to use this functionality, for example: + +[source] +---- + - apiGroups: [ "coordination.k8s.io" ] + resources: [ "leases", "configmaps" ] + verbs: [ "get", "update", "create", "patch"] +---- From 26d6b77bf60cecbd9aad07e6601db4182a11a96a Mon Sep 17 00:00:00 2001 From: wind57 Date: Wed, 15 Oct 2025 19:17:01 +0300 Subject: [PATCH 02/16] wip Signed-off-by: wind57 --- .../commons/CachedSingleThreadScheduler.java | 106 ++++++++++++++++++ .../kubernetes/commons/DeleteMeSandbox.java | 34 ++++++ 2 files changed, 140 insertions(+) create mode 100644 spring-cloud-kubernetes-commons/src/main/java/org/springframework/cloud/kubernetes/commons/CachedSingleThreadScheduler.java create mode 100644 spring-cloud-kubernetes-commons/src/main/java/org/springframework/cloud/kubernetes/commons/DeleteMeSandbox.java diff --git a/spring-cloud-kubernetes-commons/src/main/java/org/springframework/cloud/kubernetes/commons/CachedSingleThreadScheduler.java b/spring-cloud-kubernetes-commons/src/main/java/org/springframework/cloud/kubernetes/commons/CachedSingleThreadScheduler.java new file mode 100644 index 0000000000..a5ed4c2f68 --- /dev/null +++ b/spring-cloud-kubernetes-commons/src/main/java/org/springframework/cloud/kubernetes/commons/CachedSingleThreadScheduler.java @@ -0,0 +1,106 @@ +/* + * Copyright 2013-present the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.springframework.cloud.kubernetes.commons; + +import jakarta.annotation.Nonnull; + +import java.util.concurrent.Executors; +import java.util.concurrent.ScheduledFuture; +import java.util.concurrent.ScheduledThreadPoolExecutor; +import java.util.concurrent.ThreadFactory; +import java.util.concurrent.TimeUnit; +import java.util.concurrent.locks.ReentrantLock; + +/** + * This is taken from fabric8 with some minor changes (we need it, so it could be placed + * in the common package). + * + * @author wind57 + */ +public final class CachedSingleThreadScheduler { + + private final ReentrantLock lock = new ReentrantLock(); + + private final long ttlMillis; + + private ScheduledThreadPoolExecutor executor; + + public CachedSingleThreadScheduler(long ttlMillis) { + this.ttlMillis = ttlMillis; + } + + public ScheduledFuture scheduleWithFixedDelay(Runnable command, long initialDelay, long delay, TimeUnit unit) { + try { + lock.lock(); + this.startExecutor(); + return this.executor.scheduleWithFixedDelay(command, initialDelay, delay, unit); + } + finally { + lock.unlock(); + } + } + + public ScheduledFuture schedule(Runnable command, long delay, TimeUnit unit) { + try { + lock.lock(); + this.startExecutor(); + return this.executor.schedule(command, delay, unit); + } + finally { + lock.unlock(); + } + } + + private void startExecutor() { + if (this.executor == null) { + this.executor = new ScheduledThreadPoolExecutor(1, threadFactory()); + this.executor.setRemoveOnCancelPolicy(true); + this.executor.scheduleWithFixedDelay(this::shutdownCheck, this.ttlMillis, this.ttlMillis, + TimeUnit.MILLISECONDS); + } + + } + + private void shutdownCheck() { + try { + lock.lock(); + if (this.executor.getQueue().isEmpty()) { + this.executor.shutdownNow(); + this.executor = null; + } + } + finally { + lock.unlock(); + } + + } + + private ThreadFactory threadFactory() { + return new ThreadFactory() { + final ThreadFactory threadFactory = Executors.defaultThreadFactory(); + + @Override + public Thread newThread(@Nonnull Runnable runnable) { + Thread thread = threadFactory.newThread(runnable); + thread.setName("fabric8-leader-election" + "-" + thread.getName()); + thread.setDaemon(true); + return thread; + } + }; + } + +} diff --git a/spring-cloud-kubernetes-commons/src/main/java/org/springframework/cloud/kubernetes/commons/DeleteMeSandbox.java b/spring-cloud-kubernetes-commons/src/main/java/org/springframework/cloud/kubernetes/commons/DeleteMeSandbox.java new file mode 100644 index 0000000000..e23a9c09be --- /dev/null +++ b/spring-cloud-kubernetes-commons/src/main/java/org/springframework/cloud/kubernetes/commons/DeleteMeSandbox.java @@ -0,0 +1,34 @@ +package org.springframework.cloud.kubernetes.commons; + +import java.util.concurrent.ScheduledFuture; +import java.util.concurrent.ScheduledThreadPoolExecutor; +import java.util.concurrent.TimeUnit; + +public class DeleteMeSandbox { + + public static void main(String[] args) throws InterruptedException { + ScheduledThreadPoolExecutor pool = new ScheduledThreadPoolExecutor(1); + Runnable task = () -> { + System.out.println("running"); + }; + ScheduledFuture future = pool.scheduleAtFixedRate(task, 1, 1, TimeUnit.SECONDS); + + + Runnable outputTask = () -> { + while (true) { + System.out.println("pool size : " + pool.getQueue().size()); + try { + Thread.sleep(500); + } catch (InterruptedException e) { + throw new RuntimeException(e); + } + } + }; + Thread thread = new Thread(outputTask); + thread.start(); + + Thread.sleep(3_000); + future.cancel(true); + } + +} From 7345421dde1fe22545a0ea1b2dbf4c0721b87efe Mon Sep 17 00:00:00 2001 From: wind57 Date: Thu, 16 Oct 2025 22:15:24 +0300 Subject: [PATCH 03/16] started work Signed-off-by: wind57 --- .../kubernetes/commons/DeleteMeSandbox.java | 34 ------- .../commons/leader/LeaderUtils.java | 59 ++++++++++++ .../CachedSingleThreadScheduler.java | 12 +-- .../ConditionalOnLeaderElectionDisabled.java | 52 +++++++++++ .../ConditionalOnLeaderElectionEnabled.java | 43 +++++++++ .../election/LeaderElectionCallbacks.java | 87 ++++++++++++++++++ .../election/LeaderElectionProperties.java | 90 +++++++++++++++++++ .../commons/leader/election/PodReady.java | 70 +++++++++++++++ .../election/events/NewLeaderEvent.java | 34 +++++++ .../election/events/StartLeadingEvent.java | 37 ++++++++ .../election/events/StopLeadingEvent.java | 37 ++++++++ .../commons/leader/LeaderUtilsTests.java | 20 +++++ .../LeaderElectionPropertiesTests.java | 83 +++++++++++++++++ 13 files changed, 619 insertions(+), 39 deletions(-) delete mode 100644 spring-cloud-kubernetes-commons/src/main/java/org/springframework/cloud/kubernetes/commons/DeleteMeSandbox.java rename spring-cloud-kubernetes-commons/src/main/java/org/springframework/cloud/kubernetes/commons/{ => leader/election}/CachedSingleThreadScheduler.java (88%) create mode 100644 spring-cloud-kubernetes-commons/src/main/java/org/springframework/cloud/kubernetes/commons/leader/election/ConditionalOnLeaderElectionDisabled.java create mode 100644 spring-cloud-kubernetes-commons/src/main/java/org/springframework/cloud/kubernetes/commons/leader/election/ConditionalOnLeaderElectionEnabled.java create mode 100644 spring-cloud-kubernetes-commons/src/main/java/org/springframework/cloud/kubernetes/commons/leader/election/LeaderElectionCallbacks.java create mode 100644 spring-cloud-kubernetes-commons/src/main/java/org/springframework/cloud/kubernetes/commons/leader/election/LeaderElectionProperties.java create mode 100644 spring-cloud-kubernetes-commons/src/main/java/org/springframework/cloud/kubernetes/commons/leader/election/PodReady.java create mode 100644 spring-cloud-kubernetes-commons/src/main/java/org/springframework/cloud/kubernetes/commons/leader/election/events/NewLeaderEvent.java create mode 100644 spring-cloud-kubernetes-commons/src/main/java/org/springframework/cloud/kubernetes/commons/leader/election/events/StartLeadingEvent.java create mode 100644 spring-cloud-kubernetes-commons/src/main/java/org/springframework/cloud/kubernetes/commons/leader/election/events/StopLeadingEvent.java create mode 100644 spring-cloud-kubernetes-commons/src/test/java/org/springframework/cloud/kubernetes/commons/leader/election/LeaderElectionPropertiesTests.java diff --git a/spring-cloud-kubernetes-commons/src/main/java/org/springframework/cloud/kubernetes/commons/DeleteMeSandbox.java b/spring-cloud-kubernetes-commons/src/main/java/org/springframework/cloud/kubernetes/commons/DeleteMeSandbox.java deleted file mode 100644 index e23a9c09be..0000000000 --- a/spring-cloud-kubernetes-commons/src/main/java/org/springframework/cloud/kubernetes/commons/DeleteMeSandbox.java +++ /dev/null @@ -1,34 +0,0 @@ -package org.springframework.cloud.kubernetes.commons; - -import java.util.concurrent.ScheduledFuture; -import java.util.concurrent.ScheduledThreadPoolExecutor; -import java.util.concurrent.TimeUnit; - -public class DeleteMeSandbox { - - public static void main(String[] args) throws InterruptedException { - ScheduledThreadPoolExecutor pool = new ScheduledThreadPoolExecutor(1); - Runnable task = () -> { - System.out.println("running"); - }; - ScheduledFuture future = pool.scheduleAtFixedRate(task, 1, 1, TimeUnit.SECONDS); - - - Runnable outputTask = () -> { - while (true) { - System.out.println("pool size : " + pool.getQueue().size()); - try { - Thread.sleep(500); - } catch (InterruptedException e) { - throw new RuntimeException(e); - } - } - }; - Thread thread = new Thread(outputTask); - thread.start(); - - Thread.sleep(3_000); - future.cancel(true); - } - -} diff --git a/spring-cloud-kubernetes-commons/src/main/java/org/springframework/cloud/kubernetes/commons/leader/LeaderUtils.java b/spring-cloud-kubernetes-commons/src/main/java/org/springframework/cloud/kubernetes/commons/leader/LeaderUtils.java index 12e7180857..27dcdcc2d3 100644 --- a/spring-cloud-kubernetes-commons/src/main/java/org/springframework/cloud/kubernetes/commons/leader/LeaderUtils.java +++ b/spring-cloud-kubernetes-commons/src/main/java/org/springframework/cloud/kubernetes/commons/leader/LeaderUtils.java @@ -16,18 +16,56 @@ package org.springframework.cloud.kubernetes.commons.leader; +import java.io.File; +import java.io.IOException; +import java.io.UncheckedIOException; import java.net.InetAddress; import java.net.UnknownHostException; +import java.nio.file.Files; +import java.nio.file.Path; +import java.util.Optional; import java.util.concurrent.locks.ReentrantLock; import org.springframework.cloud.kubernetes.commons.EnvReader; +import org.springframework.core.log.LogAccessor; import org.springframework.util.StringUtils; +import static org.springframework.cloud.kubernetes.commons.KubernetesClientProperties.SERVICE_ACCOUNT_NAMESPACE_PATH; + /** * @author wind57 */ public final class LeaderUtils { + /** + * Coordination group for leader election. + */ + public static final String COORDINATION_GROUP = "coordination.k8s.io"; + + /** + * Coordination version for leader election. + */ + public static final String COORDINATION_VERSION = "v1"; + + /** + * Lease constant. + */ + public static final String LEASE = "Lease"; + + /** + * Prefix for all properties related to leader election. + */ + public static final String LEADER_ELECTION_PROPERTY_PREFIX = "spring.cloud.kubernetes.leader.election"; + + /** + * Property that controls whether leader election is enabled. + */ + public static final String LEADER_ELECTION_ENABLED_PROPERTY = LEADER_ELECTION_PROPERTY_PREFIX + ".enabled"; + + private static final String POD_NAMESPACE = "POD_NAMESPACE"; + + private static final LogAccessor LOG = new LogAccessor(LeaderUtils.class); + // k8s environment variable responsible for host name private static final String HOSTNAME = "HOSTNAME"; @@ -35,6 +73,27 @@ private LeaderUtils() { } + /** + * ideally, should always be present. If not, downward api must enable this one. + */ + public static Optional podNamespace() { + Path serviceAccountPath = new File(SERVICE_ACCOUNT_NAMESPACE_PATH).toPath(); + boolean serviceAccountNamespaceExists = Files.isRegularFile(serviceAccountPath); + if (serviceAccountNamespaceExists) { + try { + String namespace = new String(Files.readAllBytes(serviceAccountPath)).replace(System.lineSeparator(), + ""); + LOG.info(() -> "read namespace : " + namespace + " from service account " + serviceAccountPath); + return Optional.of(namespace); + } + catch (IOException e) { + throw new UncheckedIOException(e); + } + + } + return Optional.ofNullable(EnvReader.getEnv(POD_NAMESPACE)); + } + public static String hostName() throws UnknownHostException { String hostName = EnvReader.getEnv(HOSTNAME); if (StringUtils.hasText(hostName)) { diff --git a/spring-cloud-kubernetes-commons/src/main/java/org/springframework/cloud/kubernetes/commons/CachedSingleThreadScheduler.java b/spring-cloud-kubernetes-commons/src/main/java/org/springframework/cloud/kubernetes/commons/leader/election/CachedSingleThreadScheduler.java similarity index 88% rename from spring-cloud-kubernetes-commons/src/main/java/org/springframework/cloud/kubernetes/commons/CachedSingleThreadScheduler.java rename to spring-cloud-kubernetes-commons/src/main/java/org/springframework/cloud/kubernetes/commons/leader/election/CachedSingleThreadScheduler.java index a5ed4c2f68..1b485ff59f 100644 --- a/spring-cloud-kubernetes-commons/src/main/java/org/springframework/cloud/kubernetes/commons/CachedSingleThreadScheduler.java +++ b/spring-cloud-kubernetes-commons/src/main/java/org/springframework/cloud/kubernetes/commons/leader/election/CachedSingleThreadScheduler.java @@ -14,9 +14,7 @@ * limitations under the License. */ -package org.springframework.cloud.kubernetes.commons; - -import jakarta.annotation.Nonnull; +package org.springframework.cloud.kubernetes.commons.leader.election; import java.util.concurrent.Executors; import java.util.concurrent.ScheduledFuture; @@ -25,9 +23,13 @@ import java.util.concurrent.TimeUnit; import java.util.concurrent.locks.ReentrantLock; +import jakarta.annotation.Nonnull; + /** * This is taken from fabric8 with some minor changes (we need it, so it could be placed - * in the common package). + * in the common package). A single thread scheduler that will shutdown itself when there + * are no more jobs running inside it. When all ScheduledFuture::cancel are called, the + * queue of tasks will be empty and there is an internal runnable that checks that. * * @author wind57 */ @@ -70,7 +72,7 @@ private void startExecutor() { this.executor = new ScheduledThreadPoolExecutor(1, threadFactory()); this.executor.setRemoveOnCancelPolicy(true); this.executor.scheduleWithFixedDelay(this::shutdownCheck, this.ttlMillis, this.ttlMillis, - TimeUnit.MILLISECONDS); + TimeUnit.MILLISECONDS); } } diff --git a/spring-cloud-kubernetes-commons/src/main/java/org/springframework/cloud/kubernetes/commons/leader/election/ConditionalOnLeaderElectionDisabled.java b/spring-cloud-kubernetes-commons/src/main/java/org/springframework/cloud/kubernetes/commons/leader/election/ConditionalOnLeaderElectionDisabled.java new file mode 100644 index 0000000000..6fce5c9a6e --- /dev/null +++ b/spring-cloud-kubernetes-commons/src/main/java/org/springframework/cloud/kubernetes/commons/leader/election/ConditionalOnLeaderElectionDisabled.java @@ -0,0 +1,52 @@ +/* + * Copyright 2013-present the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.springframework.cloud.kubernetes.commons.leader.election; + +import java.lang.annotation.Documented; +import java.lang.annotation.ElementType; +import java.lang.annotation.Inherited; +import java.lang.annotation.Retention; +import java.lang.annotation.RetentionPolicy; +import java.lang.annotation.Target; + +import org.springframework.boot.autoconfigure.condition.NoneNestedConditions; +import org.springframework.context.annotation.Conditional; + +/** + * @author wind57 + */ +@Target({ ElementType.TYPE, ElementType.METHOD }) +@Retention(RetentionPolicy.RUNTIME) +@Documented +@Inherited +@Conditional(ConditionalOnLeaderElectionDisabled.OnLeaderElectionDisabled.class) +public @interface ConditionalOnLeaderElectionDisabled { + + class OnLeaderElectionDisabled extends NoneNestedConditions { + + OnLeaderElectionDisabled() { + super(ConfigurationPhase.REGISTER_BEAN); + } + + @ConditionalOnLeaderElectionEnabled + static class OnLeaderElectionDisabledClass { + + } + + } + +} diff --git a/spring-cloud-kubernetes-commons/src/main/java/org/springframework/cloud/kubernetes/commons/leader/election/ConditionalOnLeaderElectionEnabled.java b/spring-cloud-kubernetes-commons/src/main/java/org/springframework/cloud/kubernetes/commons/leader/election/ConditionalOnLeaderElectionEnabled.java new file mode 100644 index 0000000000..a58a68e553 --- /dev/null +++ b/spring-cloud-kubernetes-commons/src/main/java/org/springframework/cloud/kubernetes/commons/leader/election/ConditionalOnLeaderElectionEnabled.java @@ -0,0 +1,43 @@ +/* + * Copyright 2013-present the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.springframework.cloud.kubernetes.commons.leader.election; + +import java.lang.annotation.Documented; +import java.lang.annotation.ElementType; +import java.lang.annotation.Inherited; +import java.lang.annotation.Retention; +import java.lang.annotation.RetentionPolicy; +import java.lang.annotation.Target; + +import org.springframework.boot.autoconfigure.condition.ConditionalOnProperty; + +import static org.springframework.cloud.kubernetes.commons.leader.LeaderUtils.LEADER_ELECTION_ENABLED_PROPERTY; + +/** + * Provides a more succinct conditional for: + * spring.cloud.kubernetes.leader.election.enabled. + * + * @author wind57 + */ +@Target({ ElementType.TYPE, ElementType.METHOD }) +@Retention(RetentionPolicy.RUNTIME) +@Documented +@Inherited +@ConditionalOnProperty(value = LEADER_ELECTION_ENABLED_PROPERTY, havingValue = "true", matchIfMissing = false) +public @interface ConditionalOnLeaderElectionEnabled { + +} diff --git a/spring-cloud-kubernetes-commons/src/main/java/org/springframework/cloud/kubernetes/commons/leader/election/LeaderElectionCallbacks.java b/spring-cloud-kubernetes-commons/src/main/java/org/springframework/cloud/kubernetes/commons/leader/election/LeaderElectionCallbacks.java new file mode 100644 index 0000000000..0c8e75655a --- /dev/null +++ b/spring-cloud-kubernetes-commons/src/main/java/org/springframework/cloud/kubernetes/commons/leader/election/LeaderElectionCallbacks.java @@ -0,0 +1,87 @@ +/* + * Copyright 2013-present the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.springframework.cloud.kubernetes.commons.leader.election; + +import java.net.UnknownHostException; +import java.util.function.Consumer; + +import org.springframework.cloud.kubernetes.commons.leader.LeaderUtils; +import org.springframework.cloud.kubernetes.commons.leader.election.events.NewLeaderEvent; +import org.springframework.cloud.kubernetes.commons.leader.election.events.StartLeadingEvent; +import org.springframework.cloud.kubernetes.commons.leader.election.events.StopLeadingEvent; +import org.springframework.context.ApplicationEventPublisher; +import org.springframework.context.annotation.Bean; +import org.springframework.core.log.LogAccessor; + +/** + * common leader election callbacks that are supposed to be used in both fabric8 and + * k8s-native clients. + * + * @author wind57 + */ +public class LeaderElectionCallbacks { + + private static final LogAccessor LOG = new LogAccessor(LeaderElectionCallbacks.class); + + @Bean + public final String holderIdentity() throws UnknownHostException { + String podHostName = LeaderUtils.hostName(); + LOG.debug(() -> "using pod hostname : " + podHostName); + return podHostName; + } + + @Bean + public final String podNamespace() { + String podNamespace = LeaderUtils.podNamespace().orElse("default"); + LOG.debug(() -> "using pod namespace : " + podNamespace); + return podNamespace; + } + + @Bean + public final Runnable onStartLeadingCallback(ApplicationEventPublisher applicationEventPublisher, + String holderIdentity, LeaderElectionProperties properties) { + return () -> { + LOG.info(() -> "id : " + holderIdentity + " is now a leader"); + if (properties.publishEvents()) { + applicationEventPublisher.publishEvent(new StartLeadingEvent(holderIdentity)); + } + }; + } + + @Bean + public final Runnable onStopLeadingCallback(ApplicationEventPublisher applicationEventPublisher, + String holderIdentity, LeaderElectionProperties properties) { + return () -> { + LOG.info(() -> "id : " + holderIdentity + " stopped being a leader"); + if (properties.publishEvents()) { + applicationEventPublisher.publishEvent(new StopLeadingEvent(holderIdentity)); + } + }; + } + + @Bean + public final Consumer onNewLeaderCallback(ApplicationEventPublisher applicationEventPublisher, + LeaderElectionProperties properties) { + return holderIdentity -> { + LOG.info(() -> "id : " + holderIdentity + " is the new leader"); + if (properties.publishEvents()) { + applicationEventPublisher.publishEvent(new NewLeaderEvent(holderIdentity)); + } + }; + } + +} diff --git a/spring-cloud-kubernetes-commons/src/main/java/org/springframework/cloud/kubernetes/commons/leader/election/LeaderElectionProperties.java b/spring-cloud-kubernetes-commons/src/main/java/org/springframework/cloud/kubernetes/commons/leader/election/LeaderElectionProperties.java new file mode 100644 index 0000000000..f8b2465e48 --- /dev/null +++ b/spring-cloud-kubernetes-commons/src/main/java/org/springframework/cloud/kubernetes/commons/leader/election/LeaderElectionProperties.java @@ -0,0 +1,90 @@ +/* + * Copyright 2013-present the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.springframework.cloud.kubernetes.commons.leader.election; + +import java.time.Duration; + +import org.springframework.boot.context.properties.ConfigurationProperties; +import org.springframework.boot.context.properties.bind.DefaultValue; + +import static org.springframework.cloud.kubernetes.commons.leader.LeaderUtils.LEADER_ELECTION_PROPERTY_PREFIX; + +/** + * 
+ * waitForPodReady: should we wait for the readiness of the pod,
+ *      before we even trigger the leader election process.
+ * publishEvents: should we publish events (ApplicationEvent)
+ *      when the state of leaders changes.
+ * leaseDuration: TTL of the lease. No other leader candidate
+ *      can acquire the lease unless this one expires.
+ * lockNamespace: where to create the "lock"
+ *      (this is either a lease or a config map)
+ * lockName: the name of the lease or configmap
+ * renewDeadline: once the lock is acquired,
+ *      and we are the current leader, we try to "extend" the lease.
+ *      We must extend it within this timeline.
+ * retryPeriod: how often to retry when trying to get
+ *      the lock to become the leader. In our current code,
+ *      this is what we use in LeaderInitiator::start,
+ *      more exactly in the scheduleAtFixRate
+ *
+ *
+ * First, we try to acquire the lock (lock is either a configmap or a lease)
+ * and by "acquire" I mean write to it (or its annotations for a configmap).
+ * Whoever writes first (all others will get a 409) becomes the leader.
+ * All leader candidates that are not leaders will continue to spin forever
+ * until they get a chance to become one. They retry every 'retryPeriod'.
+ * The current leader, after it establishes itself as one,
+ * will spin forever too, but will try to extend its leadership.
+ * It extends that by updating the entries in the lease,
+ * specifically the one we care about is: renewTime.
+ * This one is updated every 'retryPeriod'. For example,
+ * every 2 seconds (retryPeriod), it will update its 'renewTime' with "now".
+ *
+ * All other, non-leaders are spinning and check a few things in each cycle:
+ * "Am I the leader?" If the answer is no, they go below:
+ * "Can I become the leader?" This is answered by looking at:
+ * now().isAfter(leaderElectionRecord.getRenewTime()
+ *           .plus(leaderElectionConfig.getLeaseDuration()))
+ * So they can only try to acquire the leadership if 'leaseDuration'
+ * (basically a TTL) + renewTime (when was the last renewal) has expired.
+ * This means that no one will be able to even try to acquire the lock
+ * until that leaseDuration expires. When the pod is killed or dies
+ * unexpectedly (OOM, for example), all non-leaders will wait until
+ * leaseDuration expires.
+ *
+ * In case of a graceful shutdown (we call CompletableFuture::cancel on the fabric8 instances),
+ * there is code that fabric8 will trigger to "reset" the lease:
+ * they will set the renewTime to "now" and leaseDuration to 1 second.
+ *
+ * + * @author wind57 + */ +// @formatter:off +@ConfigurationProperties(LEADER_ELECTION_PROPERTY_PREFIX) +public record LeaderElectionProperties( + @DefaultValue("true") boolean waitForPodReady, + @DefaultValue("true") boolean publishEvents, + @DefaultValue("15s") Duration leaseDuration, + @DefaultValue("default") String lockNamespace, + @DefaultValue("spring-k8s-leader-election-lock") String lockName, + @DefaultValue("10s") Duration renewDeadline, + @DefaultValue("2s") Duration retryPeriod, + @DefaultValue("0s") Duration waitAfterRenewalFailure, + @DefaultValue("false") boolean useConfigMapAsLock) { +// @formatter:on +} diff --git a/spring-cloud-kubernetes-commons/src/main/java/org/springframework/cloud/kubernetes/commons/leader/election/PodReady.java b/spring-cloud-kubernetes-commons/src/main/java/org/springframework/cloud/kubernetes/commons/leader/election/PodReady.java new file mode 100644 index 0000000000..da2a61bc14 --- /dev/null +++ b/spring-cloud-kubernetes-commons/src/main/java/org/springframework/cloud/kubernetes/commons/leader/election/PodReady.java @@ -0,0 +1,70 @@ +/* + * Copyright 2013-present the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.springframework.cloud.kubernetes.commons.leader.election; + +import java.util.concurrent.CompletableFuture; +import java.util.concurrent.ScheduledFuture; +import java.util.concurrent.TimeUnit; +import java.util.function.BooleanSupplier; + +import org.springframework.core.log.LogAccessor; + +/** + * @author wind57 + */ +public final class PodReady { + + private static final LogAccessor LOG = new LogAccessor(PodReady.class); + + private final CachedSingleThreadScheduler podReadyScheduler = new CachedSingleThreadScheduler( + TimeUnit.SECONDS.toMillis(10)); + + public CompletableFuture podReady(BooleanSupplier podReadySupplier, String holderIdentity, + String podNamespace) { + + CompletableFuture podReadyFuture = new CompletableFuture<>(); + + ScheduledFuture future = podReadyScheduler.scheduleWithFixedDelay(() -> { + + try { + if (podReadySupplier.getAsBoolean()) { + LOG.info(() -> "Pod : " + holderIdentity + " in namespace : " + podNamespace + " is ready"); + podReadyFuture.complete(null); + } + else { + LOG.debug(() -> "Pod : " + holderIdentity + " in namespace : " + podNamespace + " is not ready, " + + "will retry in one second"); + } + } + catch (Exception e) { + LOG.error(() -> "exception waiting for pod : " + e.getMessage()); + LOG.error(() -> "leader election for " + holderIdentity + " was not successful"); + podReadyFuture.completeExceptionally(e); + } + + }, 1, 1, TimeUnit.SECONDS); + + // cancel the future, thus shutting down the executor + podReadyFuture.whenComplete((ok, nok) -> { + future.cancel(true); + }); + + return podReadyFuture; + + } + +} diff --git a/spring-cloud-kubernetes-commons/src/main/java/org/springframework/cloud/kubernetes/commons/leader/election/events/NewLeaderEvent.java b/spring-cloud-kubernetes-commons/src/main/java/org/springframework/cloud/kubernetes/commons/leader/election/events/NewLeaderEvent.java new file mode 100644 index 0000000000..727dce8ba4 --- /dev/null +++ b/spring-cloud-kubernetes-commons/src/main/java/org/springframework/cloud/kubernetes/commons/leader/election/events/NewLeaderEvent.java @@ -0,0 +1,34 @@ +/* + * Copyright 2013-present the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.springframework.cloud.kubernetes.commons.leader.election.events; + +import org.springframework.context.ApplicationEvent; + +public final class NewLeaderEvent extends ApplicationEvent { + + private final String holderIdentity; + + public NewLeaderEvent(Object source) { + super(source); + holderIdentity = (String) source; + } + + public String holderIdentity() { + return holderIdentity; + } + +} diff --git a/spring-cloud-kubernetes-commons/src/main/java/org/springframework/cloud/kubernetes/commons/leader/election/events/StartLeadingEvent.java b/spring-cloud-kubernetes-commons/src/main/java/org/springframework/cloud/kubernetes/commons/leader/election/events/StartLeadingEvent.java new file mode 100644 index 0000000000..2d580d7013 --- /dev/null +++ b/spring-cloud-kubernetes-commons/src/main/java/org/springframework/cloud/kubernetes/commons/leader/election/events/StartLeadingEvent.java @@ -0,0 +1,37 @@ +/* + * Copyright 2013-present the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.springframework.cloud.kubernetes.commons.leader.election.events; + +import org.springframework.context.ApplicationEvent; + +/** + * @author wind57 + */ +public final class StartLeadingEvent extends ApplicationEvent { + + private final String holderIdentity; + + public StartLeadingEvent(Object source) { + super(source); + holderIdentity = (String) source; + } + + public String holderIdentity() { + return holderIdentity; + } + +} diff --git a/spring-cloud-kubernetes-commons/src/main/java/org/springframework/cloud/kubernetes/commons/leader/election/events/StopLeadingEvent.java b/spring-cloud-kubernetes-commons/src/main/java/org/springframework/cloud/kubernetes/commons/leader/election/events/StopLeadingEvent.java new file mode 100644 index 0000000000..ee0bc1abbe --- /dev/null +++ b/spring-cloud-kubernetes-commons/src/main/java/org/springframework/cloud/kubernetes/commons/leader/election/events/StopLeadingEvent.java @@ -0,0 +1,37 @@ +/* + * Copyright 2013-present the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.springframework.cloud.kubernetes.commons.leader.election.events; + +import org.springframework.context.ApplicationEvent; + +/** + * @author wind57 + */ +public final class StopLeadingEvent extends ApplicationEvent { + + private final String holderIdentity; + + public StopLeadingEvent(Object source) { + super(source); + holderIdentity = (String) source; + } + + public String holderIdentity() { + return holderIdentity; + } + +} diff --git a/spring-cloud-kubernetes-commons/src/test/java/org/springframework/cloud/kubernetes/commons/leader/LeaderUtilsTests.java b/spring-cloud-kubernetes-commons/src/test/java/org/springframework/cloud/kubernetes/commons/leader/LeaderUtilsTests.java index 74fb4997b9..b3bad5be3b 100644 --- a/spring-cloud-kubernetes-commons/src/test/java/org/springframework/cloud/kubernetes/commons/leader/LeaderUtilsTests.java +++ b/spring-cloud-kubernetes-commons/src/test/java/org/springframework/cloud/kubernetes/commons/leader/LeaderUtilsTests.java @@ -18,6 +18,7 @@ import java.net.InetAddress; import java.net.UnknownHostException; +import java.util.Optional; import org.assertj.core.api.Assertions; import org.junit.jupiter.api.Test; @@ -62,4 +63,23 @@ void hostNameReadFromApiCall() throws UnknownHostException { inet4AddressMockedStatic.close(); } + @Test + void podNamespaceMissing() { + MockedStatic envReaderMockedStatic = Mockito.mockStatic(EnvReader.class); + // envReaderMockedStatic.when(() -> EnvReader.getEnv("")).thenReturn(""); + Optional podNamespace = LeaderUtils.podNamespace(); + Assertions.assertThat(podNamespace.isEmpty()).isTrue(); + envReaderMockedStatic.close(); + } + + @Test + void podNamespacePresent() { + MockedStatic envReaderMockedStatic = Mockito.mockStatic(EnvReader.class); + envReaderMockedStatic.when(() -> EnvReader.getEnv("POD_NAMESPACE")).thenReturn("podNamespace"); + Optional podNamespace = LeaderUtils.podNamespace(); + Assertions.assertThat(podNamespace.isPresent()).isTrue(); + Assertions.assertThat(podNamespace.get()).isEqualTo("podNamespace"); + envReaderMockedStatic.close(); + } + } diff --git a/spring-cloud-kubernetes-commons/src/test/java/org/springframework/cloud/kubernetes/commons/leader/election/LeaderElectionPropertiesTests.java b/spring-cloud-kubernetes-commons/src/test/java/org/springframework/cloud/kubernetes/commons/leader/election/LeaderElectionPropertiesTests.java new file mode 100644 index 0000000000..291cef6aa5 --- /dev/null +++ b/spring-cloud-kubernetes-commons/src/test/java/org/springframework/cloud/kubernetes/commons/leader/election/LeaderElectionPropertiesTests.java @@ -0,0 +1,83 @@ +/* + * Copyright 2013-present the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.springframework.cloud.kubernetes.commons.leader.election; + +import java.time.Duration; + +import org.assertj.core.api.Assertions; +import org.junit.jupiter.api.Test; + +import org.springframework.boot.context.properties.EnableConfigurationProperties; +import org.springframework.boot.test.context.runner.ApplicationContextRunner; +import org.springframework.context.annotation.Configuration; + +/** + * @author wind57 + */ +class LeaderElectionPropertiesTests { + + @Test + void testDefaults() { + new ApplicationContextRunner().withUserConfiguration(Config.class).run(context -> { + LeaderElectionProperties properties = context.getBean(LeaderElectionProperties.class); + Assertions.assertThat(properties).isNotNull(); + Assertions.assertThat(properties.publishEvents()).isTrue(); + Assertions.assertThat(properties.waitForPodReady()).isTrue(); + Assertions.assertThat(properties.leaseDuration()).isEqualTo(Duration.ofSeconds(15)); + Assertions.assertThat(properties.lockNamespace()).isEqualTo("default"); + Assertions.assertThat(properties.lockName()).isEqualTo("spring-k8s-leader-election-lock"); + Assertions.assertThat(properties.renewDeadline()).isEqualTo(Duration.ofSeconds(10)); + Assertions.assertThat(properties.retryPeriod()).isEqualTo(Duration.ofSeconds(2)); + Assertions.assertThat(properties.waitAfterRenewalFailure()).isEqualTo(Duration.ofSeconds(0)); + Assertions.assertThat(properties.useConfigMapAsLock()).isFalse(); + }); + } + + @Test + void testNonDefaults() { + new ApplicationContextRunner().withUserConfiguration(Config.class) + .withPropertyValues("spring.cloud.kubernetes.leader.election.wait-for-pod-ready=false", + "spring.cloud.kubernetes.leader.election.publish-events=false", + "spring.cloud.kubernetes.leader.election.lease-duration=10s", + "spring.cloud.kubernetes.leader.election.lock-namespace=lock-namespace", + "spring.cloud.kubernetes.leader.election.lock-name=lock-name", + "spring.cloud.kubernetes.leader.election.renew-deadline=2d", + "spring.cloud.kubernetes.leader.election.retry-period=3m", + "spring.cloud.kubernetes.leader.election.wait-after-renewal-failure=13m", + "spring.cloud.kubernetes.leader.election.use-config-map-as-lock=true") + .run(context -> { + LeaderElectionProperties properties = context.getBean(LeaderElectionProperties.class); + Assertions.assertThat(properties).isNotNull(); + Assertions.assertThat(properties.waitForPodReady()).isFalse(); + Assertions.assertThat(properties.publishEvents()).isFalse(); + Assertions.assertThat(properties.leaseDuration()).isEqualTo(Duration.ofSeconds(10)); + Assertions.assertThat(properties.lockNamespace()).isEqualTo("lock-namespace"); + Assertions.assertThat(properties.lockName()).isEqualTo("lock-name"); + Assertions.assertThat(properties.renewDeadline()).isEqualTo(Duration.ofDays(2)); + Assertions.assertThat(properties.retryPeriod()).isEqualTo(Duration.ofMinutes(3)); + Assertions.assertThat(properties.waitAfterRenewalFailure()).isEqualTo(Duration.ofMinutes(13)); + Assertions.assertThat(properties.useConfigMapAsLock()).isTrue(); + }); + } + + @EnableConfigurationProperties(LeaderElectionProperties.class) + @Configuration + static class Config { + + } + +} From 49b4499f487b186775a1769e5d64e875aecec39b Mon Sep 17 00:00:00 2001 From: wind57 Date: Tue, 21 Oct 2025 14:40:28 +0300 Subject: [PATCH 04/16] before tests in fabric8 Signed-off-by: wind57 --- .../election/CachedSingleThreadScheduler.java | 21 +- .../{PodReady.java => PodReadyRunner.java} | 54 +++- .../CachedSingleThreadSchedulerTest.java | 200 +++++++++++++++ .../leader/election/PodReadyRunnerTests.java | 237 ++++++++++++++++++ .../Fabric8LeaderAutoConfiguration.java | 2 + ...abric8LeaderElectionAutoConfiguration.java | 129 ++++++++++ .../Fabric8LeaderElectionCallbacks.java | 32 +++ ...derElectionCallbacksAutoConfiguration.java | 52 ++++ .../Fabric8LeaderElectionInfoContributor.java | 60 +++++ .../Fabric8LeaderElectionInitiator.java | 208 +++++++++++++++ ...ot.autoconfigure.AutoConfiguration.imports | 2 + .../leader/election/Fabric8LeaderApp.java | 64 +++++ .../Fabric8LeaderAutoConfigurationTests.java | 116 +++++++++ .../apps/ConfigurationWatcherBusKafkaIT.java | 2 - .../tests/commons/FixedPortsK3sContainer.java | 2 +- .../src/main/resources/setup/role.yaml | 6 +- 16 files changed, 1166 insertions(+), 21 deletions(-) rename spring-cloud-kubernetes-commons/src/main/java/org/springframework/cloud/kubernetes/commons/leader/election/{PodReady.java => PodReadyRunner.java} (52%) create mode 100644 spring-cloud-kubernetes-commons/src/test/java/org/springframework/cloud/kubernetes/commons/leader/election/CachedSingleThreadSchedulerTest.java create mode 100644 spring-cloud-kubernetes-commons/src/test/java/org/springframework/cloud/kubernetes/commons/leader/election/PodReadyRunnerTests.java create mode 100644 spring-cloud-kubernetes-fabric8-leader/src/main/java/org/springframework/cloud/kubernetes/fabric8/leader/election/Fabric8LeaderElectionAutoConfiguration.java create mode 100644 spring-cloud-kubernetes-fabric8-leader/src/main/java/org/springframework/cloud/kubernetes/fabric8/leader/election/Fabric8LeaderElectionCallbacks.java create mode 100644 spring-cloud-kubernetes-fabric8-leader/src/main/java/org/springframework/cloud/kubernetes/fabric8/leader/election/Fabric8LeaderElectionCallbacksAutoConfiguration.java create mode 100644 spring-cloud-kubernetes-fabric8-leader/src/main/java/org/springframework/cloud/kubernetes/fabric8/leader/election/Fabric8LeaderElectionInfoContributor.java create mode 100644 spring-cloud-kubernetes-fabric8-leader/src/main/java/org/springframework/cloud/kubernetes/fabric8/leader/election/Fabric8LeaderElectionInitiator.java create mode 100644 spring-cloud-kubernetes-fabric8-leader/src/test/java/org/springframework/cloud/kubernetes/fabric8/leader/election/Fabric8LeaderApp.java create mode 100644 spring-cloud-kubernetes-fabric8-leader/src/test/java/org/springframework/cloud/kubernetes/fabric8/leader/election/Fabric8LeaderAutoConfigurationTests.java diff --git a/spring-cloud-kubernetes-commons/src/main/java/org/springframework/cloud/kubernetes/commons/leader/election/CachedSingleThreadScheduler.java b/spring-cloud-kubernetes-commons/src/main/java/org/springframework/cloud/kubernetes/commons/leader/election/CachedSingleThreadScheduler.java index 1b485ff59f..32b141c6ef 100644 --- a/spring-cloud-kubernetes-commons/src/main/java/org/springframework/cloud/kubernetes/commons/leader/election/CachedSingleThreadScheduler.java +++ b/spring-cloud-kubernetes-commons/src/main/java/org/springframework/cloud/kubernetes/commons/leader/election/CachedSingleThreadScheduler.java @@ -24,31 +24,40 @@ import java.util.concurrent.locks.ReentrantLock; import jakarta.annotation.Nonnull; +import org.apache.commons.logging.LogFactory; + +import org.springframework.core.log.LogAccessor; /** - * This is taken from fabric8 with some minor changes (we need it, so it could be placed - * in the common package). A single thread scheduler that will shutdown itself when there - * are no more jobs running inside it. When all ScheduledFuture::cancel are called, the - * queue of tasks will be empty and there is an internal runnable that checks that. + * This is taken from fabric8 with some changes (we need it, so it could be placed in the + * common package). A single thread scheduler that will shutdown itself when there are no + * more jobs running inside it. When all ScheduledFuture::cancel are called, the queue of + * tasks will be empty and there is an internal runnable that checks that. * * @author wind57 */ public final class CachedSingleThreadScheduler { + private static final LogAccessor LOG = new LogAccessor(LogFactory.getLog(CachedSingleThreadScheduler.class)); + private final ReentrantLock lock = new ReentrantLock(); private final long ttlMillis; + private final String name; + private ScheduledThreadPoolExecutor executor; - public CachedSingleThreadScheduler(long ttlMillis) { + public CachedSingleThreadScheduler(String name, long ttlMillis) { this.ttlMillis = ttlMillis; + this.name = name; } public ScheduledFuture scheduleWithFixedDelay(Runnable command, long initialDelay, long delay, TimeUnit unit) { try { lock.lock(); this.startExecutor(); + LOG.debug(() -> "Scheduling command to run in : " + name); return this.executor.scheduleWithFixedDelay(command, initialDelay, delay, unit); } finally { @@ -60,6 +69,7 @@ public ScheduledFuture schedule(Runnable command, long delay, TimeUnit unit) try { lock.lock(); this.startExecutor(); + LOG.debug(() -> "Scheduling command to run in : " + name); return this.executor.schedule(command, delay, unit); } finally { @@ -81,6 +91,7 @@ private void shutdownCheck() { try { lock.lock(); if (this.executor.getQueue().isEmpty()) { + LOG.debug(() -> "Shutting down executor : " + name); this.executor.shutdownNow(); this.executor = null; } diff --git a/spring-cloud-kubernetes-commons/src/main/java/org/springframework/cloud/kubernetes/commons/leader/election/PodReady.java b/spring-cloud-kubernetes-commons/src/main/java/org/springframework/cloud/kubernetes/commons/leader/election/PodReadyRunner.java similarity index 52% rename from spring-cloud-kubernetes-commons/src/main/java/org/springframework/cloud/kubernetes/commons/leader/election/PodReady.java rename to spring-cloud-kubernetes-commons/src/main/java/org/springframework/cloud/kubernetes/commons/leader/election/PodReadyRunner.java index da2a61bc14..c98197728d 100644 --- a/spring-cloud-kubernetes-commons/src/main/java/org/springframework/cloud/kubernetes/commons/leader/election/PodReady.java +++ b/spring-cloud-kubernetes-commons/src/main/java/org/springframework/cloud/kubernetes/commons/leader/election/PodReadyRunner.java @@ -26,33 +26,50 @@ /** * @author wind57 */ -public final class PodReady { +public final class PodReadyRunner { - private static final LogAccessor LOG = new LogAccessor(PodReady.class); + private final String candidateIdentity; - private final CachedSingleThreadScheduler podReadyScheduler = new CachedSingleThreadScheduler( - TimeUnit.SECONDS.toMillis(10)); + private final String candidateNamespace; - public CompletableFuture podReady(BooleanSupplier podReadySupplier, String holderIdentity, - String podNamespace) { + public PodReadyRunner(String candidateIdentity, String candidateNamespace) { + this.candidateIdentity = candidateIdentity; + this.candidateNamespace = candidateNamespace; + } + + // how often the inner runnable runs, or how much is the scheduler kept alive + private static final long TTL_MILLIS = 100; + + private static final LogAccessor LOG = new LogAccessor(PodReadyRunner.class); + + private final CachedSingleThreadScheduler podReadyScheduler = new CachedSingleThreadScheduler("podReadyExecutor", + TTL_MILLIS); + + public CompletableFuture podReady(BooleanSupplier podReadySupplier) { CompletableFuture podReadyFuture = new CompletableFuture<>(); ScheduledFuture future = podReadyScheduler.scheduleWithFixedDelay(() -> { + if (podReadyFuture.isDone()) { + LOG.info(() -> "pod readiness is known, not running another cycle"); + return; + } + try { if (podReadySupplier.getAsBoolean()) { - LOG.info(() -> "Pod : " + holderIdentity + " in namespace : " + podNamespace + " is ready"); + LOG.info( + () -> "Pod : " + candidateIdentity + " in namespace : " + candidateNamespace + " is ready"); podReadyFuture.complete(null); } else { - LOG.debug(() -> "Pod : " + holderIdentity + " in namespace : " + podNamespace + " is not ready, " - + "will retry in one second"); + LOG.debug(() -> "Pod : " + candidateIdentity + " in namespace : " + candidateNamespace + + " is not ready, will retry in one second"); } } catch (Exception e) { LOG.error(() -> "exception waiting for pod : " + e.getMessage()); - LOG.error(() -> "leader election for " + holderIdentity + " was not successful"); + LOG.error(() -> "leader election for : " + candidateIdentity + " was not successful"); podReadyFuture.completeExceptionally(e); } @@ -60,6 +77,23 @@ public CompletableFuture podReady(BooleanSupplier podReadySupplier, String // cancel the future, thus shutting down the executor podReadyFuture.whenComplete((ok, nok) -> { + if (nok != null) { + if (podReadyFuture.isCancelled()) { + // something triggered us externally by calling + // CompletableFuture::cancel, + // need to shut down the readiness check + LOG.debug(() -> "canceling scheduled future because completable future was cancelled"); + } + else { + LOG.debug(() -> "canceling scheduled future because readiness failed"); + } + } + else { + LOG.debug(() -> "canceling scheduled future because readiness succeeded"); + } + + // no matter the outcome, we cancel the future and thus shut down the + // executor that runs it. future.cancel(true); }); diff --git a/spring-cloud-kubernetes-commons/src/test/java/org/springframework/cloud/kubernetes/commons/leader/election/CachedSingleThreadSchedulerTest.java b/spring-cloud-kubernetes-commons/src/test/java/org/springframework/cloud/kubernetes/commons/leader/election/CachedSingleThreadSchedulerTest.java new file mode 100644 index 0000000000..b099900913 --- /dev/null +++ b/spring-cloud-kubernetes-commons/src/test/java/org/springframework/cloud/kubernetes/commons/leader/election/CachedSingleThreadSchedulerTest.java @@ -0,0 +1,200 @@ +/* + * Copyright 2013-present the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.springframework.cloud.kubernetes.commons.leader.election; + +import java.time.Duration; +import java.util.concurrent.CompletableFuture; +import java.util.concurrent.ExecutorService; +import java.util.concurrent.Executors; +import java.util.concurrent.atomic.AtomicInteger; +import java.util.function.BooleanSupplier; + +import org.assertj.core.api.Assertions; +import org.awaitility.Awaitility; +import org.junit.jupiter.api.Test; +import org.junit.jupiter.api.extension.ExtendWith; + +import org.springframework.boot.test.context.SpringBootTest; +import org.springframework.boot.test.system.CapturedOutput; +import org.springframework.boot.test.system.OutputCaptureExtension; + +/** + * @author wind57 + */ +@ExtendWith(OutputCaptureExtension.class) +@SpringBootTest(properties = { "spring.cloud.config.enabled=false", + "logging.level.org.springframework.cloud.kubernetes.commons.leader.election=debug" }) +class CachedSingleThreadSchedulerTest { + + /** + * 
+	 *     - pod readiness passes after two attempts
+	 *     - we check that the executor is shutdown after readiness passes
+	 *
+ */ + @Test + void readinessPasses(CapturedOutput output) throws Exception { + + AtomicInteger counter = new AtomicInteger(); + + BooleanSupplier supplier = () -> { + if (counter.get() == 2) { + return true; + } + else { + counter.incrementAndGet(); + } + return false; + }; + + PodReadyRunner readyRunner = new PodReadyRunner("my-pod", "my-namespace"); + CompletableFuture ready = readyRunner.podReady(supplier); + ready.get(); + + String out = output.getOut(); + Assertions.assertThat(out).contains("Scheduling command to run in : podReadyExecutor"); + Assertions.assertThat(out) + .contains("Pod : my-pod in namespace : " + "my-namespace is not ready, will retry in one second"); + Assertions.assertThat(out).contains("Pod : my-pod in namespace : " + "my-namespace is ready"); + + // executor is shutting down + Awaitility.await() + .pollInterval(Duration.ofMillis(1)) + .atMost(Duration.ofSeconds(10)) + .until(() -> output.getOut().contains("Shutting down executor : podReadyExecutor")); + + Awaitility.await() + .pollInterval(Duration.ofMillis(1)) + .atMost(Duration.ofSeconds(10)) + .until(() -> output.getOut().contains("canceling scheduled future because readiness succeeded")); + } + + /** + * 
+	 *     - pod readiness fails after one attempt
+	 *     - we check that the executor is shutdown after that
+	 *
+ */ + @Test + void readinessFails(CapturedOutput output) throws Exception { + + AtomicInteger counter = new AtomicInteger(); + + BooleanSupplier supplier = () -> { + if (counter.get() == 1) { + throw new RuntimeException("just because"); + } + else { + counter.incrementAndGet(); + } + return false; + }; + + PodReadyRunner readyRunner = new PodReadyRunner("my-pod", "my-namespace"); + CompletableFuture ready = readyRunner.podReady(supplier); + + ExecutorService readyCheckExecutor = Executors.newSingleThreadExecutor(); + + boolean[] caught = new boolean[1]; + // just like Fabric8LeaderElectionInitiator does it + // ready.get is called in a different executor + readyCheckExecutor.submit(() -> { + try { + ready.get(); + } + catch (Exception e) { + caught[0] = true; + throw new RuntimeException(e); + } + }); + + // pod readiness is started + Awaitility.await() + .pollInterval(Duration.ofMillis(200)) + .atMost(Duration.ofSeconds(10)) + .until(() -> output.getOut().contains("Scheduling command to run in : podReadyExecutor")); + + // pod readiness progresses + Awaitility.await() + .pollInterval(Duration.ofMillis(200)) + .atMost(Duration.ofSeconds(10)) + .until(() -> output.getOut() + .contains("Pod : my-pod in namespace : " + "my-namespace is not ready, will retry in one second")); + + // executor is shutting down + Awaitility.await() + .pollInterval(Duration.ofMillis(1)) + .atMost(Duration.ofSeconds(10)) + .until(() -> output.getOut().contains("Shutting down executor : podReadyExecutor")); + + Awaitility.await() + .pollInterval(Duration.ofMillis(1)) + .atMost(Duration.ofSeconds(10)) + .until(() -> output.getOut().contains("canceling scheduled future because readiness failed")); + + Assertions.assertThat(caught[0]).isTrue(); + } + + /** + * 
+	 *     - pod readiness is not established
+	 *     - we cancel the future
+	 *     - we check that the executor is shutdown after that
+	 *
+ */ + @Test + void readinessCanceled(CapturedOutput output) throws Exception { + + BooleanSupplier supplier = () -> false; + + PodReadyRunner readyRunner = new PodReadyRunner("my-pod", "my-namespace"); + CompletableFuture ready = readyRunner.podReady(supplier); + + // sleep a few cycles of pod readiness check + Thread.sleep(2_000); + + // cancel must end the readiness check + ready.cancel(true); + + // pod readiness is started + Awaitility.await() + .pollInterval(Duration.ofMillis(200)) + .atMost(Duration.ofSeconds(10)) + .until(() -> output.getOut().contains("Scheduling command to run in : podReadyExecutor")); + + // pod readiness progresses + Awaitility.await() + .pollInterval(Duration.ofMillis(200)) + .atMost(Duration.ofSeconds(10)) + .until(() -> output.getOut() + .contains("Pod : my-pod in namespace : " + "my-namespace is not ready, will retry in one second")); + + // executor is shutting down + Awaitility.await() + .pollInterval(Duration.ofMillis(1)) + .atMost(Duration.ofSeconds(10)) + .until(() -> output.getOut().contains("Shutting down executor : podReadyExecutor")); + + Awaitility.await() + .pollInterval(Duration.ofMillis(1)) + .atMost(Duration.ofSeconds(10)) + .until(() -> output.getOut() + .contains("canceling scheduled future because completable future was cancelled")); + + } + +} diff --git a/spring-cloud-kubernetes-commons/src/test/java/org/springframework/cloud/kubernetes/commons/leader/election/PodReadyRunnerTests.java b/spring-cloud-kubernetes-commons/src/test/java/org/springframework/cloud/kubernetes/commons/leader/election/PodReadyRunnerTests.java new file mode 100644 index 0000000000..559e2e35cd --- /dev/null +++ b/spring-cloud-kubernetes-commons/src/test/java/org/springframework/cloud/kubernetes/commons/leader/election/PodReadyRunnerTests.java @@ -0,0 +1,237 @@ +/* + * Copyright 2013-present the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.springframework.cloud.kubernetes.commons.leader.election; + +import java.time.Duration; +import java.util.concurrent.CompletableFuture; +import java.util.concurrent.Executors; +import java.util.concurrent.ScheduledExecutorService; +import java.util.concurrent.TimeUnit; +import java.util.concurrent.atomic.AtomicInteger; +import java.util.function.BooleanSupplier; + +import org.junit.jupiter.api.Test; +import org.junit.jupiter.api.extension.ExtendWith; + +import org.springframework.boot.test.context.SpringBootTest; +import org.springframework.boot.test.system.CapturedOutput; +import org.springframework.boot.test.system.OutputCaptureExtension; + +import static org.assertj.core.api.Assertions.assertThat; +import static org.awaitility.Awaitility.await; + +/** + * @author wind57 + */ +@SpringBootTest(properties = { "logging.level.org.springframework.cloud.kubernetes.commons.leader.election=debug", + "spring.cloud.config.enabled=false" }) +@ExtendWith(OutputCaptureExtension.class) +class PodReadyRunnerTests { + + private final PodReadyRunner podReadyRunner = new PodReadyRunner("identity", "namespace"); + + /** + * 
+	 *     - readiness passes from the first cycle
+	 *     - assert that proper logging is in place
+	 *     - assert that executor is getting shutdown
+	 *
+ */ + @Test + void readinessOKFromTheFirstCycle(CapturedOutput output) throws Exception { + BooleanSupplier readinessSupplier = () -> true; + CompletableFuture readinessFuture = podReadyRunner.podReady(readinessSupplier); + readinessFuture.get(); + + assertThat(output.getOut()).contains("Pod : identity in namespace : namespace is ready"); + assertThat(output.getOut()).contains("canceling scheduled future because readiness succeeded"); + + await().atMost(Duration.ofSeconds(3)) + .pollInterval(Duration.ofMillis(200)) + .until(() -> output.getOut().contains("Shutting down executor : podReadyExecutor")); + } + + /** + * 
+	 *     - readiness passes from the second cycle
+	 *
+ */ + @Test + void readinessOKFromTheSecondCycle(CapturedOutput output) throws Exception { + AtomicInteger counter = new AtomicInteger(0); + BooleanSupplier readinessSupplier = () -> { + if (counter.get() == 0) { + counter.incrementAndGet(); + return false; + } + return true; + }; + CompletableFuture readinessFuture = podReadyRunner.podReady(readinessSupplier); + readinessFuture.get(); + + assertThat(output.getOut()) + .contains("Pod : identity in namespace : namespace is not ready, will retry in one second"); + assertThat(output.getOut()).contains("Pod : identity in namespace : namespace is ready"); + assertThat(output.getOut()).contains("canceling scheduled future because readiness succeeded"); + + await().atMost(Duration.ofSeconds(3)) + .pollInterval(Duration.ofMillis(200)) + .until(() -> output.getOut().contains("Shutting down executor : podReadyExecutor")); + } + + /** + * 
+	 *     - readiness throws an Exception in the second cycle
+	 *
+ */ + @Test + void readinessFailsOnTheSecondCycle(CapturedOutput output) { + AtomicInteger counter = new AtomicInteger(0); + BooleanSupplier readinessSupplier = () -> { + if (counter.get() == 0) { + counter.incrementAndGet(); + return false; + } + throw new RuntimeException("fail on the second cycle"); + }; + CompletableFuture readinessFuture = podReadyRunner.podReady(readinessSupplier); + boolean caught = false; + try { + readinessFuture.get(); + } + catch (Exception e) { + caught = true; + assertThat(output.getOut()) + .contains("Pod : identity in namespace : namespace is not ready, will retry in one second"); + assertThat(output.getOut()).contains("exception waiting for pod : fail on the second cycle"); + assertThat(output.getOut()).contains("leader election for : identity was not successful"); + assertThat(output.getOut()).contains("canceling scheduled future because readiness failed"); + + await().atMost(Duration.ofSeconds(3)) + .pollInterval(Duration.ofMillis(200)) + .until(() -> output.getOut().contains("Shutting down executor : podReadyExecutor")); + } + assertThat(caught).isTrue(); + } + + /** + * 
+	 *     - readiness throws an Exception in the second cycle
+	 *     - we chain one more thenApply and test it, just like
+	 *       Fabric8LeaderElectionInitiator does it.
+	 *
+ */ + @Test + void readinessFailsOnTheSecondCycleAttachNewPipeline(CapturedOutput output) { + AtomicInteger counter = new AtomicInteger(0); + BooleanSupplier readinessSupplier = () -> { + if (counter.get() == 0) { + counter.incrementAndGet(); + return false; + } + throw new RuntimeException("fail on the second cycle"); + }; + CompletableFuture podReadyFuture = podReadyRunner.podReady(readinessSupplier); + + CompletableFuture ready = podReadyFuture.whenComplete((ok, error) -> { + if (error != null) { + System.out.println("readiness failed and we caught that"); + } + else { + System.out.println("readiness succeeded"); + } + }); + + boolean caught = false; + try { + ready.get(); + } + catch (Exception e) { + caught = true; + assertThat(output.getOut()) + .contains("Pod : identity in namespace : namespace is not ready, will retry in one second"); + assertThat(output.getOut()).contains("exception waiting for pod : fail on the second cycle"); + assertThat(output.getOut()).contains("leader election for : identity was not successful"); + assertThat(output.getOut()).contains("readiness failed and we caught that"); + assertThat(output.getOut()).contains("canceling scheduled future because readiness failed"); + + await().atMost(Duration.ofSeconds(3)) + .pollInterval(Duration.ofMillis(200)) + .until(() -> output.getOut().contains("Shutting down executor : podReadyExecutor")); + } + assertThat(caught).isTrue(); + } + + /** + * 
+	 *     - readiness is canceled
+	 *     - we chain one more thenApply and test it, just like
+	 *       Fabric8LeaderElectionInitiator does it.
+	 *
+	 *     - this simulates when we issue cancel of the podReadyFuture from
+	 *     - pre-destroy code.
+	 *
+ */ + @Test + void readinessCanceledOnTheSecondCycleAttachNewPipeline(CapturedOutput output) throws Exception { + BooleanSupplier readinessSupplier = () -> false; + + CompletableFuture podReadyFuture = podReadyRunner.podReady(readinessSupplier); + + CompletableFuture ready = podReadyFuture.whenComplete((ok, error) -> { + if (error != null) { + System.out.println("readiness failed and we caught that"); + } + else { + System.out.println("readiness succeeded"); + } + }); + + // sleep a few cycles of pod readiness check + Thread.sleep(2_000); + + ScheduledExecutorService cancelScheduler = null; + + boolean caught = false; + // cancel podReady future in a different thread + cancelScheduler = Executors.newScheduledThreadPool(1); + cancelScheduler.scheduleWithFixedDelay(() -> podReadyFuture.cancel(true), 1, 1, TimeUnit.SECONDS); + + try { + ready.get(); + } + catch (Exception e) { + caught = true; + assertThat(output.getOut()) + .contains("Pod : identity in namespace : namespace is not ready, will retry in one second"); + // this is a cancel of the future, not an exception per se + assertThat(output.getOut()).doesNotContain("leader election for : identity was not successful"); + assertThat(output.getOut()).contains("readiness failed and we caught that"); + + assertThat(output.getOut()).contains("canceling scheduled future because completable future was cancelled"); + assertThat(output.getOut()).doesNotContain("canceling scheduled future because readiness failed"); + + await().atMost(Duration.ofSeconds(3)) + .pollInterval(Duration.ofMillis(200)) + .until(() -> output.getOut().contains("Shutting down executor : podReadyExecutor")); + } + assertThat(caught).isTrue(); + cancelScheduler.shutdownNow(); + + } + +} diff --git a/spring-cloud-kubernetes-fabric8-leader/src/main/java/org/springframework/cloud/kubernetes/fabric8/leader/Fabric8LeaderAutoConfiguration.java b/spring-cloud-kubernetes-fabric8-leader/src/main/java/org/springframework/cloud/kubernetes/fabric8/leader/Fabric8LeaderAutoConfiguration.java index bb04ed4275..01af2413a1 100644 --- a/spring-cloud-kubernetes-fabric8-leader/src/main/java/org/springframework/cloud/kubernetes/fabric8/leader/Fabric8LeaderAutoConfiguration.java +++ b/spring-cloud-kubernetes-fabric8-leader/src/main/java/org/springframework/cloud/kubernetes/fabric8/leader/Fabric8LeaderAutoConfiguration.java @@ -30,6 +30,7 @@ import org.springframework.cloud.kubernetes.commons.leader.LeaderInitiator; import org.springframework.cloud.kubernetes.commons.leader.LeaderProperties; import org.springframework.cloud.kubernetes.commons.leader.LeaderUtils; +import org.springframework.cloud.kubernetes.commons.leader.election.ConditionalOnLeaderElectionDisabled; import org.springframework.context.ApplicationEventPublisher; import org.springframework.context.annotation.Bean; import org.springframework.context.annotation.Configuration; @@ -45,6 +46,7 @@ @EnableConfigurationProperties(LeaderProperties.class) @ConditionalOnBean(KubernetesClient.class) @ConditionalOnProperty(value = "spring.cloud.kubernetes.leader.enabled", matchIfMissing = true) +@ConditionalOnLeaderElectionDisabled public class Fabric8LeaderAutoConfiguration { /* diff --git a/spring-cloud-kubernetes-fabric8-leader/src/main/java/org/springframework/cloud/kubernetes/fabric8/leader/election/Fabric8LeaderElectionAutoConfiguration.java b/spring-cloud-kubernetes-fabric8-leader/src/main/java/org/springframework/cloud/kubernetes/fabric8/leader/election/Fabric8LeaderElectionAutoConfiguration.java new file mode 100644 index 0000000000..02de8c4d15 --- /dev/null +++ b/spring-cloud-kubernetes-fabric8-leader/src/main/java/org/springframework/cloud/kubernetes/fabric8/leader/election/Fabric8LeaderElectionAutoConfiguration.java @@ -0,0 +1,129 @@ +/* + * Copyright 2013-present the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.springframework.cloud.kubernetes.fabric8.leader.election; + +import io.fabric8.kubernetes.api.model.APIResource; +import io.fabric8.kubernetes.api.model.APIResourceList; +import io.fabric8.kubernetes.api.model.GroupVersionForDiscovery; +import io.fabric8.kubernetes.client.KubernetesClient; +import io.fabric8.kubernetes.client.extended.leaderelection.LeaderElectionConfig; +import io.fabric8.kubernetes.client.extended.leaderelection.LeaderElectionConfigBuilder; +import io.fabric8.kubernetes.client.extended.leaderelection.resourcelock.ConfigMapLock; +import io.fabric8.kubernetes.client.extended.leaderelection.resourcelock.LeaseLock; +import io.fabric8.kubernetes.client.extended.leaderelection.resourcelock.Lock; + +import org.springframework.boot.actuate.info.InfoContributor; +import org.springframework.boot.autoconfigure.AutoConfigureAfter; +import org.springframework.boot.autoconfigure.condition.ConditionalOnBean; +import org.springframework.boot.autoconfigure.condition.ConditionalOnClass; +import org.springframework.boot.autoconfigure.condition.ConditionalOnCloudPlatform; +import org.springframework.boot.autoconfigure.condition.ConditionalOnMissingBean; +import org.springframework.boot.cloud.CloudPlatform; +import org.springframework.boot.context.properties.EnableConfigurationProperties; +import org.springframework.boot.health.autoconfigure.contributor.ConditionalOnEnabledHealthIndicator; +import org.springframework.cloud.kubernetes.commons.leader.election.ConditionalOnLeaderElectionEnabled; +import org.springframework.cloud.kubernetes.commons.leader.election.LeaderElectionProperties; +import org.springframework.context.annotation.Bean; +import org.springframework.context.annotation.Configuration; +import org.springframework.core.log.LogAccessor; + +import static org.springframework.cloud.kubernetes.commons.leader.LeaderUtils.COORDINATION_GROUP; +import static org.springframework.cloud.kubernetes.commons.leader.LeaderUtils.COORDINATION_VERSION; +import static org.springframework.cloud.kubernetes.commons.leader.LeaderUtils.LEASE; + +/** + * @author wind57 + */ +@Configuration(proxyBeanMethods = false) +@EnableConfigurationProperties(LeaderElectionProperties.class) +@ConditionalOnBean(KubernetesClient.class) +@ConditionalOnLeaderElectionEnabled +@ConditionalOnCloudPlatform(CloudPlatform.KUBERNETES) +@AutoConfigureAfter(Fabric8LeaderElectionCallbacksAutoConfiguration.class) +class Fabric8LeaderElectionAutoConfiguration { + + private static final String COORDINATION_VERSION_GROUP = COORDINATION_GROUP + "/" + COORDINATION_VERSION; + + private static final LogAccessor LOG = new LogAccessor(Fabric8LeaderElectionAutoConfiguration.class); + + @Bean + @ConditionalOnClass(InfoContributor.class) + @ConditionalOnEnabledHealthIndicator("leader.election") + Fabric8LeaderElectionInfoContributor leaderElectionInfoContributor(String holderIdentity, + LeaderElectionConfig leaderElectionConfig, KubernetesClient fabric8KubernetesClient) { + return new Fabric8LeaderElectionInfoContributor(holderIdentity, leaderElectionConfig, fabric8KubernetesClient); + } + + //TODO +// @Bean +// @ConditionalOnMissingBean +// Fabric8LeaderElectionInitiator fabric8LeaderElectionInitiator(String holderIdentity, String podNamespace, +// KubernetesClient fabric8KubernetesClient, LeaderElectionConfig fabric8LeaderElectionConfig, +// LeaderElectionProperties leaderElectionProperties) { +// return new Fabric8LeaderElectionInitiator(holderIdentity, podNamespace, fabric8KubernetesClient, +// fabric8LeaderElectionConfig, leaderElectionProperties); +// } + + @Bean + @ConditionalOnMissingBean + LeaderElectionConfig fabric8LeaderElectionConfig(LeaderElectionProperties properties, Lock lock, + Fabric8LeaderElectionCallbacks fabric8LeaderElectionCallbacks) { + return new LeaderElectionConfigBuilder() + .withReleaseOnCancel() + .withName("Spring k8s leader election") + .withLeaseDuration(properties.leaseDuration()) + .withLock(lock) + .withRenewDeadline(properties.renewDeadline()) + .withRetryPeriod(properties.retryPeriod()) + .withLeaderCallbacks(fabric8LeaderElectionCallbacks) + .build(); + } + + @Bean + @ConditionalOnMissingBean + Lock lock(KubernetesClient fabric8KubernetesClient, LeaderElectionProperties properties, String holderIdentity) { + boolean leaseSupported = fabric8KubernetesClient.getApiGroups() + .getGroups() + .stream() + .flatMap(x -> x.getVersions().stream()) + .map(GroupVersionForDiscovery::getGroupVersion) + .filter(COORDINATION_VERSION_GROUP::equals) + .findFirst() + .map(fabric8KubernetesClient::getApiResources) + .map(APIResourceList::getResources) + .map(x -> x.stream().map(APIResource::getKind)) + .flatMap(x -> x.filter(y -> y.equals(LEASE)).findFirst()) + .isPresent(); + + if (leaseSupported) { + if (properties.useConfigMapAsLock()) { + LOG.info(() -> "leases are supported on the cluster, but config map will be used " + + "(because 'spring.cloud.kubernetes.leader.election.use-config-map-as-lock=true')"); + return new ConfigMapLock(properties.lockNamespace(), properties.lockName(), holderIdentity); + } + else { + LOG.info(() -> "will use lease as the lock for leader election"); + return new LeaseLock(properties.lockNamespace(), properties.lockName(), holderIdentity); + } + } + else { + LOG.info(() -> "will use configmap as the lock for leader election"); + return new ConfigMapLock(properties.lockNamespace(), properties.lockName(), holderIdentity); + } + } + +} diff --git a/spring-cloud-kubernetes-fabric8-leader/src/main/java/org/springframework/cloud/kubernetes/fabric8/leader/election/Fabric8LeaderElectionCallbacks.java b/spring-cloud-kubernetes-fabric8-leader/src/main/java/org/springframework/cloud/kubernetes/fabric8/leader/election/Fabric8LeaderElectionCallbacks.java new file mode 100644 index 0000000000..910d7fb3a7 --- /dev/null +++ b/spring-cloud-kubernetes-fabric8-leader/src/main/java/org/springframework/cloud/kubernetes/fabric8/leader/election/Fabric8LeaderElectionCallbacks.java @@ -0,0 +1,32 @@ +/* + * Copyright 2013-2024 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.springframework.cloud.kubernetes.fabric8.leader.election; + +import java.util.function.Consumer; + +import io.fabric8.kubernetes.client.extended.leaderelection.LeaderCallbacks; + +/** + * @author wind57 + */ +final class Fabric8LeaderElectionCallbacks extends LeaderCallbacks { + + Fabric8LeaderElectionCallbacks(Runnable onStartLeading, Runnable onStopLeading, Consumer onNewLeader) { + super(onStartLeading, onStopLeading, onNewLeader); + } + +} diff --git a/spring-cloud-kubernetes-fabric8-leader/src/main/java/org/springframework/cloud/kubernetes/fabric8/leader/election/Fabric8LeaderElectionCallbacksAutoConfiguration.java b/spring-cloud-kubernetes-fabric8-leader/src/main/java/org/springframework/cloud/kubernetes/fabric8/leader/election/Fabric8LeaderElectionCallbacksAutoConfiguration.java new file mode 100644 index 0000000000..7e1faddede --- /dev/null +++ b/spring-cloud-kubernetes-fabric8-leader/src/main/java/org/springframework/cloud/kubernetes/fabric8/leader/election/Fabric8LeaderElectionCallbacksAutoConfiguration.java @@ -0,0 +1,52 @@ +/* + * Copyright 2013-present the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.springframework.cloud.kubernetes.fabric8.leader.election; + +import java.util.function.Consumer; + +import io.fabric8.kubernetes.client.KubernetesClient; + +import org.springframework.boot.autoconfigure.AutoConfigureAfter; +import org.springframework.boot.autoconfigure.condition.ConditionalOnBean; +import org.springframework.boot.autoconfigure.condition.ConditionalOnCloudPlatform; +import org.springframework.boot.autoconfigure.condition.ConditionalOnMissingBean; +import org.springframework.boot.cloud.CloudPlatform; +import org.springframework.boot.context.properties.EnableConfigurationProperties; +import org.springframework.cloud.kubernetes.commons.KubernetesCommonsAutoConfiguration; +import org.springframework.cloud.kubernetes.commons.leader.election.ConditionalOnLeaderElectionEnabled; +import org.springframework.cloud.kubernetes.commons.leader.election.LeaderElectionCallbacks; +import org.springframework.cloud.kubernetes.commons.leader.election.LeaderElectionProperties; +import org.springframework.cloud.kubernetes.fabric8.Fabric8AutoConfiguration; +import org.springframework.context.annotation.Bean; +import org.springframework.context.annotation.Configuration; + +@Configuration(proxyBeanMethods = false) +@EnableConfigurationProperties(LeaderElectionProperties.class) +@ConditionalOnBean(KubernetesClient.class) +@ConditionalOnCloudPlatform(CloudPlatform.KUBERNETES) +@ConditionalOnLeaderElectionEnabled +@AutoConfigureAfter({ Fabric8AutoConfiguration.class, KubernetesCommonsAutoConfiguration.class }) +final class Fabric8LeaderElectionCallbacksAutoConfiguration extends LeaderElectionCallbacks { + + @Bean + @ConditionalOnMissingBean + Fabric8LeaderElectionCallbacks fabric8LeaderElectionCallbacks(Runnable onStartLeadingCallback, + Runnable onStopLeadingCallback, Consumer onNewLeaderCallback) { + return new Fabric8LeaderElectionCallbacks(onStartLeadingCallback, onStopLeadingCallback, onNewLeaderCallback); + } + +} diff --git a/spring-cloud-kubernetes-fabric8-leader/src/main/java/org/springframework/cloud/kubernetes/fabric8/leader/election/Fabric8LeaderElectionInfoContributor.java b/spring-cloud-kubernetes-fabric8-leader/src/main/java/org/springframework/cloud/kubernetes/fabric8/leader/election/Fabric8LeaderElectionInfoContributor.java new file mode 100644 index 0000000000..706a2fbb4d --- /dev/null +++ b/spring-cloud-kubernetes-fabric8-leader/src/main/java/org/springframework/cloud/kubernetes/fabric8/leader/election/Fabric8LeaderElectionInfoContributor.java @@ -0,0 +1,60 @@ +/* + * Copyright 2013-present the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.springframework.cloud.kubernetes.fabric8.leader.election; + +import java.util.HashMap; +import java.util.Map; +import java.util.Optional; + +import io.fabric8.kubernetes.client.KubernetesClient; +import io.fabric8.kubernetes.client.extended.leaderelection.LeaderElectionConfig; + +import org.springframework.boot.actuate.info.Info; +import org.springframework.boot.actuate.info.InfoContributor; + +/** + * @author wind57 + */ +final class Fabric8LeaderElectionInfoContributor implements InfoContributor { + + private final String holderIdentity; + + private final LeaderElectionConfig leaderElectionConfig; + + private final KubernetesClient fabric8KubernetesClient; + + Fabric8LeaderElectionInfoContributor(String holderIdentity, LeaderElectionConfig leaderElectionConfig, + KubernetesClient fabric8KubernetesClient) { + this.holderIdentity = holderIdentity; + this.leaderElectionConfig = leaderElectionConfig; + this.fabric8KubernetesClient = fabric8KubernetesClient; + } + + @Override + public void contribute(Info.Builder builder) { + Map details = new HashMap<>(); + Optional.ofNullable(leaderElectionConfig.getLock().get(fabric8KubernetesClient)) + .ifPresentOrElse(leaderRecord -> { + boolean isLeader = holderIdentity.equals(leaderRecord.getHolderIdentity()); + details.put("leaderId", holderIdentity); + details.put("isLeader", isLeader); + }, () -> details.put("leaderId", "Unknown")); + + builder.withDetail("leaderElection", details); + } + +} diff --git a/spring-cloud-kubernetes-fabric8-leader/src/main/java/org/springframework/cloud/kubernetes/fabric8/leader/election/Fabric8LeaderElectionInitiator.java b/spring-cloud-kubernetes-fabric8-leader/src/main/java/org/springframework/cloud/kubernetes/fabric8/leader/election/Fabric8LeaderElectionInitiator.java new file mode 100644 index 0000000000..905ba95ae9 --- /dev/null +++ b/spring-cloud-kubernetes-fabric8-leader/src/main/java/org/springframework/cloud/kubernetes/fabric8/leader/election/Fabric8LeaderElectionInitiator.java @@ -0,0 +1,208 @@ +/* +* Copyright 2013-2024 the original author or authors. +* +* Licensed under the Apache License, Version 2.0 (the "License"); +* you may not use this file except in compliance with the License. +* You may obtain a copy of the License at +* +* https://www.apache.org/licenses/LICENSE-2.0 +* +* Unless required by applicable law or agreed to in writing, software +* distributed under the License is distributed on an "AS IS" BASIS, +* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +* See the License for the specific language governing permissions and +* limitations under the License. +*/ +package org.springframework.cloud.kubernetes.fabric8.leader.election; + +import java.util.concurrent.CancellationException; +import java.util.concurrent.CompletableFuture; +import java.util.concurrent.ExecutorService; +import java.util.concurrent.TimeUnit; +import java.util.function.BooleanSupplier; + +import io.fabric8.kubernetes.api.model.Pod; +import io.fabric8.kubernetes.client.KubernetesClient; +import io.fabric8.kubernetes.client.extended.leaderelection.LeaderElectionConfig; +import io.fabric8.kubernetes.client.extended.leaderelection.LeaderElector; +import io.fabric8.kubernetes.client.readiness.Readiness; +import jakarta.annotation.PostConstruct; +import jakarta.annotation.PreDestroy; + +import org.springframework.cloud.kubernetes.commons.leader.election.LeaderElectionProperties; +import org.springframework.cloud.kubernetes.commons.leader.election.PodReadyRunner; +import org.springframework.core.log.LogAccessor; + +import static java.util.concurrent.Executors.newSingleThreadExecutor; + +/** + * @author wind57 + */ + final class Fabric8LeaderElectionInitiator { + + private static final LogAccessor LOG = new LogAccessor(Fabric8LeaderElectionInitiator.class); + + private final PodReadyRunner podReadyRunner; + + private final String candidateIdentity; + + private final KubernetesClient fabric8KubernetesClient; + + private final LeaderElectionConfig leaderElectionConfig; + + private final LeaderElectionProperties leaderElectionProperties; + + private final boolean waitForPodReady; + + private final ExecutorService podReadyWaitingExecutor; + + private final BooleanSupplier podReadySupplier; + + private volatile CompletableFuture podReadyFuture; + + private volatile boolean destroyCalled = false; + + private volatile CompletableFuture leaderFuture; + + Fabric8LeaderElectionInitiator(String candidateIdentity, String candidateNamespace, + KubernetesClient fabric8KubernetesClient, LeaderElectionConfig leaderElectionConfig, + LeaderElectionProperties leaderElectionProperties) { + this.candidateIdentity = candidateIdentity; + this.fabric8KubernetesClient = fabric8KubernetesClient; + this.leaderElectionConfig = leaderElectionConfig; + this.leaderElectionProperties = leaderElectionProperties; + this.waitForPodReady = leaderElectionProperties.waitForPodReady(); + + this.podReadyWaitingExecutor = newSingleThreadExecutor(runnable -> + new Thread(runnable, "Fabric8LeaderElectionInitiator-" + candidateIdentity)); + + this.podReadySupplier = () -> { + Pod pod = fabric8KubernetesClient.pods().inNamespace(candidateNamespace).withName(candidateIdentity).get(); + return Readiness.isPodReady(pod); + }; + + this.podReadyRunner = new PodReadyRunner(candidateIdentity, candidateNamespace); + } + + /** + * 
+	 * 	We first try to see if we need to wait for the pod to be ready
+	 * 	before starting the leader election process.
+	 *
+ * + */ + @PostConstruct + void postConstruct() { + LOG.info(() -> "starting leader initiator : " + candidateIdentity); + + // wait until the pod is ready + if (waitForPodReady) { + LOG.info(() -> "will wait until pod " + candidateIdentity + " is ready"); + podReadyFuture = podReadyRunner.podReady(podReadySupplier); + } + else { + podReadyFuture = CompletableFuture.completedFuture(null); + } + + // wait in a different thread until the pod is ready + // and don't block the main application from starting + podReadyWaitingExecutor.submit(() -> { + if (waitForPodReady) { + + // if 'ready' is already completed at this point, thread will run this, + // otherwise it will attach the pipeline and move on to 'blockReadinessCheck' + CompletableFuture ready = podReadyFuture.whenComplete((ok, error) -> { + if (error != null) { + LOG.error(() -> "readiness failed for : " + candidateIdentity); + LOG.error(() -> "leader election for : " + candidateIdentity + " will not start"); + } + else { + LOG.info(() -> candidateIdentity + " is ready"); + startLeaderElection(); + } + }); + + blockReadinessCheck(ready); + + } + else { + startLeaderElection(); + } + }); + + } + + @PreDestroy + void preDestroy() { + destroyCalled = true; + LOG.info(() -> "preDestroy called in the leader initiator : " + candidateIdentity); + + if (podReadyFuture != null && !podReadyFuture.isDone()) { + // if the task is not running, this has no effect. + // if the task is running, calling this will also make sure + // that the caching executor will shut down too. + podReadyFuture.cancel(true); + } + + if (leaderFuture != null) { + LOG.info(() -> "leader will be canceled : " + candidateIdentity); + // needed to release the lock, in case we are holding it. + // fabric8 internally expects this one to be called + leaderFuture.cancel(true); + } + podReadyWaitingExecutor.shutdownNow(); + } + + private void startLeaderElection() { + leaderFuture = leaderElector(leaderElectionConfig, fabric8KubernetesClient).start(); + leaderFuture.whenComplete((ok, error) -> { + + if (ok != null) { + LOG.info(() -> "leaderFuture finished normally, will re-start it for : " + candidateIdentity); + startLeaderElection(); + return; + } + + if (error instanceof CancellationException) { + if (!destroyCalled) { + LOG.warn(() -> "renewal failed for : " + candidateIdentity + ", will re-start it after : " + + leaderElectionProperties.waitAfterRenewalFailure().toSeconds() + " seconds"); + sleep(); + startLeaderElection(); + } + } + else { + LOG.warn(() -> "leader election is over for : " + candidateIdentity); + } + + try { + leaderFuture.get(); + } catch (Exception e) { + LOG.warn(() -> "leader election failed for : " + candidateIdentity + ". Trying to recover..."); + } + }); + } + + private LeaderElector leaderElector(LeaderElectionConfig config, KubernetesClient fabric8KubernetesClient) { + return fabric8KubernetesClient.leaderElector().withConfig(config).build(); + } + + private void sleep() { + try { + TimeUnit.SECONDS.sleep(leaderElectionProperties.waitAfterRenewalFailure().toSeconds()); + } + catch (InterruptedException e) { + throw new RuntimeException(e); + } + } + + private void blockReadinessCheck(CompletableFuture ready) { + try { + ready.get(); + } + catch (Exception e) { + throw new RuntimeException(e); + } + } + +} diff --git a/spring-cloud-kubernetes-fabric8-leader/src/main/resources/META-INF/spring/org.springframework.boot.autoconfigure.AutoConfiguration.imports b/spring-cloud-kubernetes-fabric8-leader/src/main/resources/META-INF/spring/org.springframework.boot.autoconfigure.AutoConfiguration.imports index d612cac6dc..c94b99ab84 100644 --- a/spring-cloud-kubernetes-fabric8-leader/src/main/resources/META-INF/spring/org.springframework.boot.autoconfigure.AutoConfiguration.imports +++ b/spring-cloud-kubernetes-fabric8-leader/src/main/resources/META-INF/spring/org.springframework.boot.autoconfigure.AutoConfiguration.imports @@ -1 +1,3 @@ org.springframework.cloud.kubernetes.fabric8.leader.Fabric8LeaderAutoConfiguration +org.springframework.cloud.kubernetes.fabric8.leader.election.Fabric8LeaderElectionCallbacksAutoConfiguration +org.springframework.cloud.kubernetes.fabric8.leader.election.Fabric8LeaderElectionAutoConfiguration diff --git a/spring-cloud-kubernetes-fabric8-leader/src/test/java/org/springframework/cloud/kubernetes/fabric8/leader/election/Fabric8LeaderApp.java b/spring-cloud-kubernetes-fabric8-leader/src/test/java/org/springframework/cloud/kubernetes/fabric8/leader/election/Fabric8LeaderApp.java new file mode 100644 index 0000000000..f51818e22f --- /dev/null +++ b/spring-cloud-kubernetes-fabric8-leader/src/test/java/org/springframework/cloud/kubernetes/fabric8/leader/election/Fabric8LeaderApp.java @@ -0,0 +1,64 @@ +/* + * Copyright 2013-2024 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.springframework.cloud.kubernetes.fabric8.leader.election; + +import io.fabric8.kubernetes.client.KubernetesClient; +import io.fabric8.kubernetes.client.dsl.MixedOperation; +import io.fabric8.kubernetes.client.dsl.PodResource; +import io.fabric8.kubernetes.client.dsl.Resource; +import io.fabric8.kubernetes.client.dsl.internal.BaseOperation; +import io.fabric8.kubernetes.client.extended.leaderelection.resourcelock.Lock; +import org.mockito.Mockito; + +import org.springframework.context.annotation.Bean; +import org.springframework.context.annotation.Configuration; +import org.springframework.context.annotation.Primary; + +@Configuration +public class Fabric8LeaderApp { + + @SuppressWarnings({ "rawtypes", "unchecked" }) + @Bean + KubernetesClient kubernetesClient() { + KubernetesClient client = Mockito.mock(KubernetesClient.class); + Mockito.when(client.getNamespace()).thenReturn("a"); + + MixedOperation mixedOperation = Mockito.mock(MixedOperation.class); + Mockito.when(client.configMaps()).thenReturn(mixedOperation); + + PodResource podResource = Mockito.mock(PodResource.class); + Mockito.when(podResource.isReady()).thenReturn(true); + + Mockito.when(client.pods()).thenReturn(mixedOperation); + Mockito.when(mixedOperation.withName(Mockito.anyString())).thenReturn(podResource); + + Resource resource = Mockito.mock(Resource.class); + + BaseOperation baseOperation = Mockito.mock(BaseOperation.class); + Mockito.when(baseOperation.withName("leaders")).thenReturn(resource); + + Mockito.when(mixedOperation.inNamespace("a")).thenReturn(baseOperation); + return client; + } + + @Bean + @Primary + Lock lock() { + return Mockito.mock(Lock.class); + } + +} diff --git a/spring-cloud-kubernetes-fabric8-leader/src/test/java/org/springframework/cloud/kubernetes/fabric8/leader/election/Fabric8LeaderAutoConfigurationTests.java b/spring-cloud-kubernetes-fabric8-leader/src/test/java/org/springframework/cloud/kubernetes/fabric8/leader/election/Fabric8LeaderAutoConfigurationTests.java new file mode 100644 index 0000000000..6979c2d1d3 --- /dev/null +++ b/spring-cloud-kubernetes-fabric8-leader/src/test/java/org/springframework/cloud/kubernetes/fabric8/leader/election/Fabric8LeaderAutoConfigurationTests.java @@ -0,0 +1,116 @@ +/* + * Copyright 2013-present the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.springframework.cloud.kubernetes.fabric8.leader.election; + +import org.assertj.core.api.Assertions; +import org.junit.jupiter.api.Test; + +import org.springframework.boot.autoconfigure.AutoConfigurations; +import org.springframework.boot.test.context.runner.ApplicationContextRunner; +import org.springframework.cloud.kubernetes.fabric8.leader.Fabric8LeaderAutoConfiguration; +import org.springframework.cloud.kubernetes.fabric8.leader.Fabric8PodReadinessWatcher; + +/** + * tests that ensure 'spring.cloud.kubernetes.leader.election' enabled correct + * auto-configurations, when it is enabled/disabled. + * + * @author wind57 + */ +class Fabric8LeaderAutoConfigurationTests { + + /** + * 
+	 *     - spring.cloud.kubernetes.leader.election is not present
+	 *
+	 *     As such:
+	 *
+	 *     - Fabric8LeaderAutoConfiguration must be picked up
+	 *     - Fabric8LeaderElectionAutoConfiguration must not be picked up
+	 *
+ */ + @Test + void leaderElectionAnnotationMissing() { + new ApplicationContextRunner().withUserConfiguration(Fabric8LeaderApp.class) + .withConfiguration(AutoConfigurations.of(Fabric8LeaderAutoConfiguration.class, + Fabric8LeaderElectionAutoConfiguration.class, + Fabric8LeaderElectionCallbacksAutoConfiguration.class)) + .run(context -> { + + // this one comes from Fabric8LeaderElectionAutoConfiguration + Assertions.assertThat(context).doesNotHaveBean(Fabric8LeaderElectionInitiator.class); + + // this one comes from Fabric8LeaderAutoConfiguration + Assertions.assertThat(context).hasSingleBean(Fabric8PodReadinessWatcher.class); + }); + } + + /** + * 
+	 *     - spring.cloud.kubernetes.leader.election = false
+	 *
+	 *     As such:
+	 *
+	 *     - Fabric8LeaderAutoConfiguration must be picked up
+	 *     - Fabric8LeaderElectionAutoConfiguration must not be picked up
+	 *
+ */ + @Test + void leaderElectionAnnotationPresentEqualToFalse() { + new ApplicationContextRunner().withUserConfiguration(Fabric8LeaderApp.class) + .withConfiguration(AutoConfigurations.of(Fabric8LeaderAutoConfiguration.class, + Fabric8LeaderElectionAutoConfiguration.class, + Fabric8LeaderElectionCallbacksAutoConfiguration.class)) + .withPropertyValues("spring.cloud.kubernetes.leader.election.enabled=false") + .run(context -> { + + // this one comes from Fabric8LeaderElectionAutoConfiguration + Assertions.assertThat(context).doesNotHaveBean(Fabric8LeaderElectionInitiator.class); + + // this one comes from Fabric8LeaderAutoConfiguration + Assertions.assertThat(context).hasSingleBean(Fabric8PodReadinessWatcher.class); + }); + } + + /** + * 
+	 *     - spring.cloud.kubernetes.leader.election = false
+	 *
+	 *     As such:
+	 *
+	 *     - Fabric8LeaderAutoConfiguration must not be picked up
+	 *     - Fabric8LeaderElectionAutoConfiguration must be picked up
+	 *
+ */ + @Test + void leaderElectionAnnotationPresentEqualToTrue() { + new ApplicationContextRunner().withUserConfiguration(Fabric8LeaderApp.class) + .withConfiguration(AutoConfigurations.of(Fabric8LeaderAutoConfiguration.class, + Fabric8LeaderElectionAutoConfiguration.class, + Fabric8LeaderElectionCallbacksAutoConfiguration.class)) + .withPropertyValues("spring.cloud.kubernetes.leader.election.enabled=true", + "spring.main.cloud-platform=kubernetes") + .run(context -> { + + // this one comes from Fabric8LeaderElectionAutoConfiguration + Assertions.assertThat(context).hasSingleBean(Fabric8LeaderElectionInitiator.class); + + // this one comes from Fabric8LeaderAutoConfiguration + Assertions.assertThat(context).doesNotHaveBean(Fabric8PodReadinessWatcher.class); + }); + } + +} diff --git a/spring-cloud-kubernetes-integration-tests/spring-cloud-kubernetes-k8s-client-kafka-configmap-reload/kafka-configmap-test-app/src/test/java/org/springframework/cloud/kubernetes/configuration/watcher/multiple/apps/ConfigurationWatcherBusKafkaIT.java b/spring-cloud-kubernetes-integration-tests/spring-cloud-kubernetes-k8s-client-kafka-configmap-reload/kafka-configmap-test-app/src/test/java/org/springframework/cloud/kubernetes/configuration/watcher/multiple/apps/ConfigurationWatcherBusKafkaIT.java index f80e5c3313..aae56e7519 100644 --- a/spring-cloud-kubernetes-integration-tests/spring-cloud-kubernetes-k8s-client-kafka-configmap-reload/kafka-configmap-test-app/src/test/java/org/springframework/cloud/kubernetes/configuration/watcher/multiple/apps/ConfigurationWatcherBusKafkaIT.java +++ b/spring-cloud-kubernetes-integration-tests/spring-cloud-kubernetes-k8s-client-kafka-configmap-reload/kafka-configmap-test-app/src/test/java/org/springframework/cloud/kubernetes/configuration/watcher/multiple/apps/ConfigurationWatcherBusKafkaIT.java @@ -72,8 +72,6 @@ static void beforeAll() throws Exception { Commons.loadSpringCloudKubernetesImage(CONFIG_WATCHER_APP_IMAGE, K3S); Images.loadKafka(K3S); - - util = new Util(K3S); util.setUp(NAMESPACE); } diff --git a/spring-cloud-kubernetes-test-support/src/main/java/org/springframework/cloud/kubernetes/integration/tests/commons/FixedPortsK3sContainer.java b/spring-cloud-kubernetes-test-support/src/main/java/org/springframework/cloud/kubernetes/integration/tests/commons/FixedPortsK3sContainer.java index 88096eb2d8..2535eae8c7 100644 --- a/spring-cloud-kubernetes-test-support/src/main/java/org/springframework/cloud/kubernetes/integration/tests/commons/FixedPortsK3sContainer.java +++ b/spring-cloud-kubernetes-test-support/src/main/java/org/springframework/cloud/kubernetes/integration/tests/commons/FixedPortsK3sContainer.java @@ -37,7 +37,7 @@ final class FixedPortsK3sContainer extends K3sContainer { /** * Test containers exposed ports. */ - private static final int[] EXPOSED_PORTS = new int[] { 80, 6443, 8080, 8888, 9092, 32321 }; + private static final int[] EXPOSED_PORTS = new int[] { 80, 6443, 8080, 8888, 9092, 32321, 32322 }; /** * Rancher version to use for test-containers. diff --git a/spring-cloud-kubernetes-test-support/src/main/resources/setup/role.yaml b/spring-cloud-kubernetes-test-support/src/main/resources/setup/role.yaml index 4a397a6920..a159e008fc 100644 --- a/spring-cloud-kubernetes-test-support/src/main/resources/setup/role.yaml +++ b/spring-cloud-kubernetes-test-support/src/main/resources/setup/role.yaml @@ -4,6 +4,6 @@ metadata: namespace: default name: namespace-reader rules: - - apiGroups: ["", "extensions", "apps", "discovery.k8s.io"] - resources: ["configmaps", "pods", "services", "endpoints", "secrets", "endpointslices"] - verbs: ["get", "list", "watch"] + - apiGroups: ["", "extensions", "apps", "discovery.k8s.io", "coordination.k8s.io"] + resources: ["configmaps", "pods", "services", "endpoints", "secrets", "endpointslices", "leases"] + verbs: ["get", "list", "watch", "create", "update", "patch"] From d55c4b67dc6e3e5e9d3a180a997babfbd7781d54 Mon Sep 17 00:00:00 2001 From: wind57 Date: Wed, 22 Oct 2025 23:11:37 +0300 Subject: [PATCH 05/16] wip Signed-off-by: wind57 --- .../election/LeaderElectionCallbacks.java | 2 +- ...abric8LeaderElectionAutoConfiguration.java | 17 +- ...erElectionInfoContributorIsLeaderTest.java | 150 +++++++++++ ...lectionInfoContributorIsNotLeaderTest.java | 150 +++++++++++ ...ic8LeaderOldAndNewImplementationTests.java | 244 ++++++++++++++++++ .../it/Fabric8LeaderElectionSimpleITTest.java | 147 +++++++++++ .../src/test/resources/logback-test.xml | 6 + 7 files changed, 706 insertions(+), 10 deletions(-) create mode 100644 spring-cloud-kubernetes-fabric8-leader/src/test/java/org/springframework/cloud/kubernetes/fabric8/leader/election/Fabric8LeaderElectionInfoContributorIsLeaderTest.java create mode 100644 spring-cloud-kubernetes-fabric8-leader/src/test/java/org/springframework/cloud/kubernetes/fabric8/leader/election/Fabric8LeaderElectionInfoContributorIsNotLeaderTest.java create mode 100644 spring-cloud-kubernetes-fabric8-leader/src/test/java/org/springframework/cloud/kubernetes/fabric8/leader/election/Fabric8LeaderOldAndNewImplementationTests.java create mode 100644 spring-cloud-kubernetes-fabric8-leader/src/test/java/org/springframework/cloud/kubernetes/fabric8/leader/election/it/Fabric8LeaderElectionSimpleITTest.java create mode 100644 spring-cloud-kubernetes-fabric8-leader/src/test/resources/logback-test.xml diff --git a/spring-cloud-kubernetes-commons/src/main/java/org/springframework/cloud/kubernetes/commons/leader/election/LeaderElectionCallbacks.java b/spring-cloud-kubernetes-commons/src/main/java/org/springframework/cloud/kubernetes/commons/leader/election/LeaderElectionCallbacks.java index 0c8e75655a..dc919afd21 100644 --- a/spring-cloud-kubernetes-commons/src/main/java/org/springframework/cloud/kubernetes/commons/leader/election/LeaderElectionCallbacks.java +++ b/spring-cloud-kubernetes-commons/src/main/java/org/springframework/cloud/kubernetes/commons/leader/election/LeaderElectionCallbacks.java @@ -55,7 +55,7 @@ public final String podNamespace() { public final Runnable onStartLeadingCallback(ApplicationEventPublisher applicationEventPublisher, String holderIdentity, LeaderElectionProperties properties) { return () -> { - LOG.info(() -> "id : " + holderIdentity + " is now a leader"); + LOG.info(() -> holderIdentity + " is now a leader"); if (properties.publishEvents()) { applicationEventPublisher.publishEvent(new StartLeadingEvent(holderIdentity)); } diff --git a/spring-cloud-kubernetes-fabric8-leader/src/main/java/org/springframework/cloud/kubernetes/fabric8/leader/election/Fabric8LeaderElectionAutoConfiguration.java b/spring-cloud-kubernetes-fabric8-leader/src/main/java/org/springframework/cloud/kubernetes/fabric8/leader/election/Fabric8LeaderElectionAutoConfiguration.java index 02de8c4d15..0bec6c4ecc 100644 --- a/spring-cloud-kubernetes-fabric8-leader/src/main/java/org/springframework/cloud/kubernetes/fabric8/leader/election/Fabric8LeaderElectionAutoConfiguration.java +++ b/spring-cloud-kubernetes-fabric8-leader/src/main/java/org/springframework/cloud/kubernetes/fabric8/leader/election/Fabric8LeaderElectionAutoConfiguration.java @@ -68,15 +68,14 @@ Fabric8LeaderElectionInfoContributor leaderElectionInfoContributor(String holder return new Fabric8LeaderElectionInfoContributor(holderIdentity, leaderElectionConfig, fabric8KubernetesClient); } - //TODO -// @Bean -// @ConditionalOnMissingBean -// Fabric8LeaderElectionInitiator fabric8LeaderElectionInitiator(String holderIdentity, String podNamespace, -// KubernetesClient fabric8KubernetesClient, LeaderElectionConfig fabric8LeaderElectionConfig, -// LeaderElectionProperties leaderElectionProperties) { -// return new Fabric8LeaderElectionInitiator(holderIdentity, podNamespace, fabric8KubernetesClient, -// fabric8LeaderElectionConfig, leaderElectionProperties); -// } + @Bean + @ConditionalOnMissingBean + Fabric8LeaderElectionInitiator fabric8LeaderElectionInitiator(String holderIdentity, String podNamespace, + KubernetesClient fabric8KubernetesClient, LeaderElectionConfig fabric8LeaderElectionConfig, + LeaderElectionProperties leaderElectionProperties) { + return new Fabric8LeaderElectionInitiator(holderIdentity, podNamespace, fabric8KubernetesClient, + fabric8LeaderElectionConfig, leaderElectionProperties); + } @Bean @ConditionalOnMissingBean diff --git a/spring-cloud-kubernetes-fabric8-leader/src/test/java/org/springframework/cloud/kubernetes/fabric8/leader/election/Fabric8LeaderElectionInfoContributorIsLeaderTest.java b/spring-cloud-kubernetes-fabric8-leader/src/test/java/org/springframework/cloud/kubernetes/fabric8/leader/election/Fabric8LeaderElectionInfoContributorIsLeaderTest.java new file mode 100644 index 0000000000..1cb0b7525b --- /dev/null +++ b/spring-cloud-kubernetes-fabric8-leader/src/test/java/org/springframework/cloud/kubernetes/fabric8/leader/election/Fabric8LeaderElectionInfoContributorIsLeaderTest.java @@ -0,0 +1,150 @@ +/* + * Copyright 2013-2024 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.springframework.cloud.kubernetes.fabric8.leader.election; + +import java.time.ZonedDateTime; + +import io.fabric8.kubernetes.api.model.APIGroupList; +import io.fabric8.kubernetes.api.model.APIGroupListBuilder; +import io.fabric8.kubernetes.api.model.APIResourceBuilder; +import io.fabric8.kubernetes.api.model.APIResourceListBuilder; +import io.fabric8.kubernetes.api.model.GroupVersionForDiscoveryBuilder; +import io.fabric8.kubernetes.api.model.coordination.v1.Lease; +import io.fabric8.kubernetes.api.model.coordination.v1.LeaseBuilder; +import io.fabric8.kubernetes.api.model.coordination.v1.LeaseSpecBuilder; +import io.fabric8.kubernetes.client.KubernetesClient; +import io.fabric8.kubernetes.client.dsl.MixedOperation; +import io.fabric8.kubernetes.client.dsl.NonNamespaceOperation; +import io.fabric8.kubernetes.client.dsl.Resource; +import org.junit.jupiter.api.AfterAll; +import org.junit.jupiter.api.BeforeAll; +import org.junit.jupiter.api.Test; +import org.mockito.MockedStatic; +import org.mockito.Mockito; + +import org.springframework.beans.factory.annotation.Autowired; +import org.springframework.boot.test.context.SpringBootTest; +import org.springframework.boot.test.context.TestConfiguration; +import org.springframework.boot.test.web.server.LocalManagementPort; +import org.springframework.boot.webtestclient.AutoConfigureWebTestClient; +import org.springframework.cloud.kubernetes.commons.leader.LeaderUtils; +import org.springframework.context.annotation.Bean; +import org.springframework.context.annotation.Primary; +import org.springframework.http.MediaType; +import org.springframework.test.web.reactive.server.WebTestClient; + +/** + * @author wind57 + */ +@SpringBootTest(webEnvironment = SpringBootTest.WebEnvironment.RANDOM_PORT, + properties = { "spring.main.cloud-platform=KUBERNETES", "management.endpoints.web.exposure.include=info", + "management.endpoint.info.show-details=always", "management.info.kubernetes.enabled=true", + "spring.cloud.kubernetes.leader.election.enabled=true" }) +@AutoConfigureWebTestClient +class Fabric8LeaderElectionInfoContributorIsLeaderTest { + + private static final String HOLDER_IDENTITY = "leader"; + + @LocalManagementPort + private int port; + + @Autowired + private WebTestClient webClient; + + private static MockedStatic leaderUtilsMockedStatic; + + @BeforeAll + static void beforeAll() { + leaderUtilsMockedStatic = Mockito.mockStatic(LeaderUtils.class); + leaderUtilsMockedStatic.when(LeaderUtils::hostName).thenReturn(HOLDER_IDENTITY); + } + + @AfterAll + static void afterAll() { + leaderUtilsMockedStatic.close(); + } + + @Test + void infoEndpointIsLeaderTest() { + webClient.get() + .uri("http://localhost:{port}/actuator/info", port) + .accept(MediaType.APPLICATION_JSON) + .exchange() + .expectStatus() + .isOk() + .expectBody() + .jsonPath("leaderElection.isLeader") + .isEqualTo(true) + .jsonPath("leaderElection.leaderId") + .isEqualTo(HOLDER_IDENTITY); + } + + @TestConfiguration + static class Configuration { + + @Bean + @Primary + KubernetesClient mockKubernetesClient() { + KubernetesClient client = Mockito.mock(KubernetesClient.class); + mockForLeaseSupport(client); + mockForLeaderSupport(client); + return client; + } + + private void mockForLeaseSupport(KubernetesClient client) { + Mockito.when(client.getApiResources("coordination.k8s.io/v1")) + .thenReturn( + new APIResourceListBuilder().withResources(new APIResourceBuilder().withKind("Lease").build()) + .build()); + + APIGroupList apiGroupList = new APIGroupListBuilder().addNewGroup() + .withVersions(new GroupVersionForDiscoveryBuilder().withGroupVersion("coordination.k8s.io/v1").build()) + .endGroup() + .build(); + + Mockito.when(client.getApiGroups()).thenReturn(apiGroupList); + } + + @SuppressWarnings({ "unchecked", "rawtypes" }) + private void mockForLeaderSupport(KubernetesClient client) { + + Lease lease = new LeaseBuilder().withNewMetadata() + .withName("spring-k8s-leader-election-lock") + .endMetadata() + .withSpec(new LeaseSpecBuilder().withHolderIdentity(HOLDER_IDENTITY) + .withLeaseDurationSeconds(1) + .withAcquireTime(ZonedDateTime.now()) + .withRenewTime(ZonedDateTime.now()) + .withLeaseTransitions(1) + .build()) + .build(); + + MixedOperation mixedOperation = Mockito.mock(MixedOperation.class); + Mockito.when(client.resources(Lease.class)).thenReturn(mixedOperation); + + Resource resource = Mockito.mock(Resource.class); + Mockito.when(resource.get()).thenReturn(lease); + + NonNamespaceOperation nonNamespaceOperation = Mockito.mock(NonNamespaceOperation.class); + Mockito.when(mixedOperation.inNamespace("default")).thenReturn(nonNamespaceOperation); + Mockito.when(nonNamespaceOperation.withName("spring-k8s-leader-election-lock")).thenReturn(resource); + + } + + } + +} diff --git a/spring-cloud-kubernetes-fabric8-leader/src/test/java/org/springframework/cloud/kubernetes/fabric8/leader/election/Fabric8LeaderElectionInfoContributorIsNotLeaderTest.java b/spring-cloud-kubernetes-fabric8-leader/src/test/java/org/springframework/cloud/kubernetes/fabric8/leader/election/Fabric8LeaderElectionInfoContributorIsNotLeaderTest.java new file mode 100644 index 0000000000..069d5037f9 --- /dev/null +++ b/spring-cloud-kubernetes-fabric8-leader/src/test/java/org/springframework/cloud/kubernetes/fabric8/leader/election/Fabric8LeaderElectionInfoContributorIsNotLeaderTest.java @@ -0,0 +1,150 @@ +/* + * Copyright 2013-present the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.springframework.cloud.kubernetes.fabric8.leader.election; + +import java.time.ZonedDateTime; + +import io.fabric8.kubernetes.api.model.APIGroupList; +import io.fabric8.kubernetes.api.model.APIGroupListBuilder; +import io.fabric8.kubernetes.api.model.APIResourceBuilder; +import io.fabric8.kubernetes.api.model.APIResourceListBuilder; +import io.fabric8.kubernetes.api.model.GroupVersionForDiscoveryBuilder; +import io.fabric8.kubernetes.api.model.coordination.v1.Lease; +import io.fabric8.kubernetes.api.model.coordination.v1.LeaseBuilder; +import io.fabric8.kubernetes.api.model.coordination.v1.LeaseSpecBuilder; +import io.fabric8.kubernetes.client.KubernetesClient; +import io.fabric8.kubernetes.client.dsl.MixedOperation; +import io.fabric8.kubernetes.client.dsl.NonNamespaceOperation; +import io.fabric8.kubernetes.client.dsl.Resource; +import org.junit.jupiter.api.AfterAll; +import org.junit.jupiter.api.BeforeAll; +import org.junit.jupiter.api.Test; +import org.mockito.MockedStatic; +import org.mockito.Mockito; + +import org.springframework.beans.factory.annotation.Autowired; +import org.springframework.boot.test.context.SpringBootTest; +import org.springframework.boot.test.context.TestConfiguration; +import org.springframework.boot.test.web.server.LocalManagementPort; +import org.springframework.boot.webtestclient.AutoConfigureWebTestClient; +import org.springframework.cloud.kubernetes.commons.leader.LeaderUtils; +import org.springframework.context.annotation.Bean; +import org.springframework.context.annotation.Primary; +import org.springframework.http.MediaType; +import org.springframework.test.web.reactive.server.WebTestClient; + +/** + * @author wind57 + */ +@SpringBootTest(webEnvironment = SpringBootTest.WebEnvironment.RANDOM_PORT, + properties = { "spring.main.cloud-platform=KUBERNETES", "management.endpoints.web.exposure.include=info", + "management.endpoint.info.show-details=always", "management.info.kubernetes.enabled=true", + "spring.cloud.kubernetes.leader.election.enabled=true" }) +@AutoConfigureWebTestClient +class Fabric8LeaderElectionInfoContributorIsNotLeaderTest { + + private static final String HOLDER_IDENTITY = "leader"; + + @LocalManagementPort + private int port; + + @Autowired + private WebTestClient webClient; + + private static MockedStatic leaderUtilsMockedStatic; + + @BeforeAll + static void beforeAll() { + leaderUtilsMockedStatic = Mockito.mockStatic(LeaderUtils.class); + leaderUtilsMockedStatic.when(LeaderUtils::hostName).thenReturn("non-" + HOLDER_IDENTITY); + } + + @AfterAll + static void afterAll() { + leaderUtilsMockedStatic.close(); + } + + @Test + void infoEndpointIsNotLeaderTest() { + webClient.get() + .uri("http://localhost:{port}/actuator/info", port) + .accept(MediaType.APPLICATION_JSON) + .exchange() + .expectStatus() + .isOk() + .expectBody() + .jsonPath("leaderElection.isLeader") + .isEqualTo(false) + .jsonPath("leaderElection.leaderId") + .isEqualTo("non-" + HOLDER_IDENTITY); + } + + @TestConfiguration + static class Configuration { + + @Bean + @Primary + KubernetesClient mockKubernetesClient() { + KubernetesClient client = Mockito.mock(KubernetesClient.class); + mockForLeaseSupport(client); + mockForLeaderSupport(client); + return client; + } + + private void mockForLeaseSupport(KubernetesClient client) { + Mockito.when(client.getApiResources("coordination.k8s.io/v1")) + .thenReturn( + new APIResourceListBuilder().withResources(new APIResourceBuilder().withKind("Lease").build()) + .build()); + + APIGroupList apiGroupList = new APIGroupListBuilder().addNewGroup() + .withVersions(new GroupVersionForDiscoveryBuilder().withGroupVersion("coordination.k8s.io/v1").build()) + .endGroup() + .build(); + + Mockito.when(client.getApiGroups()).thenReturn(apiGroupList); + } + + @SuppressWarnings({ "unchecked", "rawtypes" }) + private void mockForLeaderSupport(KubernetesClient client) { + + Lease lease = new LeaseBuilder().withNewMetadata() + .withName("spring-k8s-leader-election-lock") + .endMetadata() + .withSpec(new LeaseSpecBuilder().withHolderIdentity(HOLDER_IDENTITY) + .withLeaseDurationSeconds(1) + .withAcquireTime(ZonedDateTime.now()) + .withRenewTime(ZonedDateTime.now()) + .withLeaseTransitions(1) + .build()) + .build(); + + MixedOperation mixedOperation = Mockito.mock(MixedOperation.class); + Mockito.when(client.resources(Lease.class)).thenReturn(mixedOperation); + + Resource resource = Mockito.mock(Resource.class); + Mockito.when(resource.get()).thenReturn(lease); + + NonNamespaceOperation nonNamespaceOperation = Mockito.mock(NonNamespaceOperation.class); + Mockito.when(mixedOperation.inNamespace("default")).thenReturn(nonNamespaceOperation); + Mockito.when(nonNamespaceOperation.withName("spring-k8s-leader-election-lock")).thenReturn(resource); + + } + + } + +} diff --git a/spring-cloud-kubernetes-fabric8-leader/src/test/java/org/springframework/cloud/kubernetes/fabric8/leader/election/Fabric8LeaderOldAndNewImplementationTests.java b/spring-cloud-kubernetes-fabric8-leader/src/test/java/org/springframework/cloud/kubernetes/fabric8/leader/election/Fabric8LeaderOldAndNewImplementationTests.java new file mode 100644 index 0000000000..e2e152ff44 --- /dev/null +++ b/spring-cloud-kubernetes-fabric8-leader/src/test/java/org/springframework/cloud/kubernetes/fabric8/leader/election/Fabric8LeaderOldAndNewImplementationTests.java @@ -0,0 +1,244 @@ +/* + * Copyright 2013-present the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.springframework.cloud.kubernetes.fabric8.leader.election; + +import io.fabric8.kubernetes.api.model.APIGroupList; +import io.fabric8.kubernetes.api.model.APIGroupListBuilder; +import io.fabric8.kubernetes.api.model.APIResourceBuilder; +import io.fabric8.kubernetes.api.model.APIResourceListBuilder; +import io.fabric8.kubernetes.api.model.ConfigMap; +import io.fabric8.kubernetes.api.model.GroupVersionForDiscoveryBuilder; +import io.fabric8.kubernetes.client.KubernetesClient; +import io.fabric8.kubernetes.client.dsl.MixedOperation; +import io.fabric8.kubernetes.client.dsl.NonNamespaceOperation; +import io.fabric8.kubernetes.client.dsl.PodResource; +import io.fabric8.kubernetes.client.dsl.Resource; +import org.junit.jupiter.api.Test; +import org.mockito.Mockito; + +import org.springframework.boot.autoconfigure.AutoConfigurations; +import org.springframework.boot.test.context.TestConfiguration; +import org.springframework.boot.test.context.runner.ApplicationContextRunner; +import org.springframework.cloud.kubernetes.commons.KubernetesCommonsAutoConfiguration; +import org.springframework.cloud.kubernetes.fabric8.Fabric8AutoConfiguration; +import org.springframework.cloud.kubernetes.fabric8.leader.Fabric8LeaderAutoConfiguration; +import org.springframework.context.annotation.Bean; + +import static org.assertj.core.api.Assertions.assertThat; + +/** + * Tests that prove that previous and new leader implementation works based on the flags + * we set. + * + * @author wind57 + */ +class Fabric8LeaderOldAndNewImplementationTests { + + private ApplicationContextRunner applicationContextRunner; + + /** + * 
+	 *     - 'spring.cloud.kubernetes.leader.enabled'           is not set
+	 *     - 'spring.cloud.kubernetes.leader.election.enabled'  is not set
+	 *
+	 *     As such :
+	 *
+	 *     - 'Fabric8LeaderAutoConfiguration'                   is active
+	 *     - 'Fabric8LeaderElectionAutoConfiguration'           is not active
+	 *
+ */ + @Test + void noFlagsSet() { + setup("spring.main.cloud-platform=KUBERNETES"); + applicationContextRunner.run(context -> { + assertThat(context).hasSingleBean(Fabric8LeaderAutoConfiguration.class); + assertThat(context).doesNotHaveBean(Fabric8LeaderElectionAutoConfiguration.class); + }); + } + + /** + * 
+	 *     - 'spring.cloud.kubernetes.leader.enabled'          =  true
+	 *     - 'spring.cloud.kubernetes.leader.election.enabled'    is not set
+	 *
+	 *     As such :
+	 *
+	 *     - 'Fabric8LeaderAutoConfiguration'                   is active
+	 *     - 'Fabric8LeaderElectionAutoConfiguration'           is not active
+	 *
+ */ + @Test + void oldImplementationEnabled() { + setup("spring.main.cloud-platform=KUBERNETES", "spring.cloud.kubernetes.leader.enabled=true"); + applicationContextRunner.run(context -> { + assertThat(context).hasSingleBean(Fabric8LeaderAutoConfiguration.class); + assertThat(context).doesNotHaveBean(Fabric8LeaderElectionAutoConfiguration.class); + }); + } + + /** + * 
+	 *     - 'spring.cloud.kubernetes.leader.enabled'          = false
+	 *     - 'spring.cloud.kubernetes.leader.election.enabled'   is not set
+	 *
+	 *     As such :
+	 *
+	 *     - 'Fabric8LeaderAutoConfiguration'                   is not active
+	 *     - 'Fabric8LeaderElectionAutoConfiguration'           is not active
+	 *
+ */ + @Test + void oldImplementationDisabled() { + setup("spring.main.cloud-platform=KUBERNETES", "spring.cloud.kubernetes.leader.enabled=false"); + applicationContextRunner.run(context -> { + assertThat(context).doesNotHaveBean(Fabric8LeaderAutoConfiguration.class); + assertThat(context).doesNotHaveBean(Fabric8LeaderElectionAutoConfiguration.class); + }); + } + + /** + * 
+	 *     - 'spring.cloud.kubernetes.leader.enabled'            is not set
+	 *     - 'spring.cloud.kubernetes.leader.election.enabled' = false
+	 *
+	 *     As such :
+	 *
+	 *     - 'Fabric8LeaderAutoConfiguration'                   is active
+	 *     - 'Fabric8LeaderElectionAutoConfiguration'           is not active
+	 *
+ */ + @Test + void newImplementationDisabled() { + setup("spring.main.cloud-platform=KUBERNETES", "spring.cloud.kubernetes.leader.election.enabled=false"); + applicationContextRunner.run(context -> { + assertThat(context).hasSingleBean(Fabric8LeaderAutoConfiguration.class); + assertThat(context).doesNotHaveBean(Fabric8LeaderElectionAutoConfiguration.class); + }); + } + + /** + * 
+	 *     - 'spring.cloud.kubernetes.leader.enabled'            is not set
+	 *     - 'spring.cloud.kubernetes.leader.election.enabled' = true
+	 *
+	 *     As such :
+	 *
+	 *     - 'Fabric8LeaderAutoConfiguration'                   is not active
+	 *     - 'Fabric8LeaderElectionAutoConfiguration'           is active
+	 *
+ */ + @Test + void newImplementationEnabled() { + setup("spring.main.cloud-platform=KUBERNETES", "spring.cloud.kubernetes.leader.election.enabled=true"); + applicationContextRunner.run(context -> { + assertThat(context).doesNotHaveBean(Fabric8LeaderAutoConfiguration.class); + assertThat(context).hasSingleBean(Fabric8LeaderElectionAutoConfiguration.class); + }); + } + + /** + * 
+	 *     - 'spring.cloud.kubernetes.leader.enabled'          = false
+	 *     - 'spring.cloud.kubernetes.leader.election.enabled' = false
+	 *
+	 *     As such :
+	 *
+	 *     - 'Fabric8LeaderAutoConfiguration'                   is not active
+	 *     - 'Fabric8LeaderElectionAutoConfiguration'           is not active
+	 *
+ */ + @Test + void bothDisabled() { + setup("spring.main.cloud-platform=KUBERNETES", "spring.cloud.kubernetes.leader.enabled=false", + "spring.cloud.kubernetes.leader.election.enabled=false"); + applicationContextRunner.run(context -> { + assertThat(context).doesNotHaveBean(Fabric8LeaderAutoConfiguration.class); + assertThat(context).doesNotHaveBean(Fabric8LeaderElectionAutoConfiguration.class); + }); + } + + /** + * 
+	 *     - 'spring.cloud.kubernetes.leader.enabled'          = true
+	 *     - 'spring.cloud.kubernetes.leader.election.enabled' = true
+	 *
+	 *     As such :
+	 *
+	 *     - 'Fabric8LeaderAutoConfiguration'                   is not active
+	 *     - 'Fabric8LeaderElectionAutoConfiguration'           is active
+	 *
+	 *     You can't enable both of them, only the new one will work.
+	 *
+ */ + @Test + void bothEnabled() { + setup("spring.main.cloud-platform=KUBERNETES", "spring.cloud.kubernetes.leader.enabled=true", + "spring.cloud.kubernetes.leader.election.enabled=true"); + applicationContextRunner.run(context -> { + assertThat(context).doesNotHaveBean(Fabric8LeaderAutoConfiguration.class); + assertThat(context).hasSingleBean(Fabric8LeaderElectionAutoConfiguration.class); + }); + } + + private void setup(String... properties) { + applicationContextRunner = new ApplicationContextRunner() + .withConfiguration(AutoConfigurations.of(Fabric8LeaderElectionCallbacksAutoConfiguration.class, + Fabric8AutoConfiguration.class, KubernetesCommonsAutoConfiguration.class, + Fabric8LeaderElectionAutoConfiguration.class, Fabric8LeaderAutoConfiguration.class)) + .withUserConfiguration(Fabric8LeaderOldAndNewImplementationTests.Configuration.class) + .withPropertyValues(properties); + } + + @TestConfiguration + static class Configuration { + + @Bean + @SuppressWarnings({ "rawtypes", "unchecked" }) + KubernetesClient mockKubernetesClient() { + KubernetesClient client = Mockito.mock(KubernetesClient.class); + + Mockito.when(client.getNamespace()).thenReturn("namespace"); + + MixedOperation mixedOperation = Mockito.mock(MixedOperation.class); + NonNamespaceOperation nonNamespaceOperation = Mockito.mock(NonNamespaceOperation.class); + Mockito.when(client.configMaps()).thenReturn(mixedOperation); + + Mockito.when(mixedOperation.inNamespace(Mockito.anyString())).thenReturn(nonNamespaceOperation); + Resource configMapResource = Mockito.mock(Resource.class); + Mockito.when(nonNamespaceOperation.withName(Mockito.anyString())).thenReturn(configMapResource); + + Mockito.when(client.pods()).thenReturn(mixedOperation); + PodResource podResource = Mockito.mock(PodResource.class); + Mockito.when(mixedOperation.withName(Mockito.anyString())).thenReturn(podResource); + + Mockito.when(client.getApiResources("coordination.k8s.io/v1")) + .thenReturn( + new APIResourceListBuilder().withResources(new APIResourceBuilder().withKind("Lease").build()) + .build()); + + APIGroupList apiGroupList = new APIGroupListBuilder().addNewGroup() + .withVersions(new GroupVersionForDiscoveryBuilder().withGroupVersion("coordination.k8s.io/v1").build()) + .endGroup() + .build(); + + Mockito.when(client.getApiGroups()).thenReturn(apiGroupList); + return client; + } + + } + +} diff --git a/spring-cloud-kubernetes-fabric8-leader/src/test/java/org/springframework/cloud/kubernetes/fabric8/leader/election/it/Fabric8LeaderElectionSimpleITTest.java b/spring-cloud-kubernetes-fabric8-leader/src/test/java/org/springframework/cloud/kubernetes/fabric8/leader/election/it/Fabric8LeaderElectionSimpleITTest.java new file mode 100644 index 0000000000..f9a765fa96 --- /dev/null +++ b/spring-cloud-kubernetes-fabric8-leader/src/test/java/org/springframework/cloud/kubernetes/fabric8/leader/election/it/Fabric8LeaderElectionSimpleITTest.java @@ -0,0 +1,147 @@ +/* + * Copyright 2013-present the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.springframework.cloud.kubernetes.fabric8.leader.election.it; + +import java.time.Duration; + +import io.fabric8.kubernetes.client.Config; +import io.fabric8.kubernetes.client.KubernetesClient; +import io.fabric8.kubernetes.client.KubernetesClientBuilder; +import org.awaitility.Awaitility; +import org.junit.jupiter.api.AfterAll; +import org.junit.jupiter.api.BeforeAll; +import org.junit.jupiter.api.Test; +import org.junit.jupiter.api.extension.ExtendWith; +import org.mockito.MockedStatic; +import org.mockito.Mockito; +import org.springframework.cloud.kubernetes.commons.leader.LeaderUtils; +import org.testcontainers.k3s.K3sContainer; + +import org.springframework.beans.factory.annotation.Autowired; +import org.springframework.boot.test.context.SpringBootTest; +import org.springframework.boot.test.context.TestConfiguration; +import org.springframework.boot.test.system.CapturedOutput; +import org.springframework.boot.test.system.OutputCaptureExtension; +import org.springframework.cloud.kubernetes.integration.tests.commons.Commons; +import org.springframework.context.annotation.Bean; +import org.springframework.context.annotation.Primary; + +import static org.assertj.core.api.Assertions.assertThat; + +/** + * A very simple test where we are the sole participant in the leader + * election and everything goes fine from start to end. It's a happy path + * scenario test. + * + * @author wind57 + */ +@SpringBootTest(webEnvironment = SpringBootTest.WebEnvironment.RANDOM_PORT, + properties = { "spring.main.cloud-platform=KUBERNETES", "spring.cloud.kubernetes.leader.election.enabled=true", + "spring.cloud.kubernetes.leader.election.wait-for-pod-ready=false" }) +@ExtendWith(OutputCaptureExtension.class) +class Fabric8LeaderElectionSimpleITTest { + + private static K3sContainer container; + + private static final MockedStatic LEADER_UTILS_MOCKED_STATIC = Mockito.mockStatic(LeaderUtils.class); + + @Autowired + private KubernetesClient kubernetesClient; + + @BeforeAll + static void beforeAll() { + container = Commons.container(); + container.start(); + + LEADER_UTILS_MOCKED_STATIC.when(LeaderUtils::hostName).thenReturn("simple-it"); + } + + @AfterAll + static void afterAll() { + container.stop(); + } + + @Test + void test(CapturedOutput output) { + + // 1. lease is used as the lock (comes from our code) + assertThat(output.getOut()).contains( + "will use lease as the lock for leader election"); + + // 2. we start leader initiator for our hostname (comes from our code) + assertThat(output.getOut()).contains( + "starting leader initiator : simple-it"); + + // 3. we try to acquire the lease (comes from fabric8 code) + assertThat(output.getOut()).contains( + "Attempting to acquire leader lease 'LeaseLock: default - spring-k8s-leader-election-lock (simple-it)'"); + + // 4. we are the leader (comes from our code) + assertThat(output.getOut()).contains("simple-it is now a leader"); + + // wait for a renewal + Awaitility.await() + .pollInterval(Duration.ofSeconds(1)) + .atMost(Duration.ofMinutes(1)) + .until(() -> output.getOut().contains("Attempting to renew leader lease")); + +// // all these logs happen before a renewal +// Assertions.assertThat(output.getOut()).contains("starting leader initiator"); +// Assertions.assertThat(output.getOut()).contains("Leader election started"); +// Assertions.assertThat(output.getOut()).contains("Successfully Acquired leader lease"); +// +// Lease lockLease = kubernetesClient.leases() +// .inNamespace("default") +// .withName("spring-k8s-leader-election-lock") +// .get(); +// ZonedDateTime currentAcquiredTime = lockLease.getSpec().getAcquireTime(); +// Assertions.assertThat(currentAcquiredTime).isNotNull(); +// Assertions.assertThat(lockLease.getSpec().getLeaseDurationSeconds()).isEqualTo(15); +// Assertions.assertThat(lockLease.getSpec().getLeaseTransitions()).isEqualTo(0); +// +// ZonedDateTime currentRenewalTime = lockLease.getSpec().getRenewTime(); +// Assertions.assertThat(currentRenewalTime).isNotNull(); +// +// // renew happened, we renew by default on every two seconds +// Awaitility.await() +// .pollInterval(Duration.ofSeconds(1)) +// .atMost(Duration.ofSeconds(4)) +// .until(() -> !(currentRenewalTime.equals(kubernetesClient.leases() +// .inNamespace("default") +// .withName("spring-k8s-leader-election-lock") +// .get() +// .getSpec() +// .getRenewTime()))); + } + + @TestConfiguration + static class LocalConfiguration { + + @Bean + @Primary + KubernetesClient client() { + String kubeConfigYaml = container.getKubeConfigYaml(); + Config config = Config.fromKubeconfig(kubeConfigYaml); + return new KubernetesClientBuilder().withConfig(config).build(); + } + + } + + // test with pod ready + // simulate that we lose leadership, must re-try + +} diff --git a/spring-cloud-kubernetes-fabric8-leader/src/test/resources/logback-test.xml b/spring-cloud-kubernetes-fabric8-leader/src/test/resources/logback-test.xml new file mode 100644 index 0000000000..55654605fa --- /dev/null +++ b/spring-cloud-kubernetes-fabric8-leader/src/test/resources/logback-test.xml @@ -0,0 +1,6 @@ + + + + + + From 723a1f73d5ddd822d62df6cbfe6af15441f63608 Mon Sep 17 00:00:00 2001 From: wind57 Date: Wed, 22 Oct 2025 23:23:45 +0300 Subject: [PATCH 06/16] wip Signed-off-by: wind57 --- .../it/Fabric8LeaderElectionSimpleITTest.java | 20 ++++++++++++------- 1 file changed, 13 insertions(+), 7 deletions(-) diff --git a/spring-cloud-kubernetes-fabric8-leader/src/test/java/org/springframework/cloud/kubernetes/fabric8/leader/election/it/Fabric8LeaderElectionSimpleITTest.java b/spring-cloud-kubernetes-fabric8-leader/src/test/java/org/springframework/cloud/kubernetes/fabric8/leader/election/it/Fabric8LeaderElectionSimpleITTest.java index f9a765fa96..f8c1cee785 100644 --- a/spring-cloud-kubernetes-fabric8-leader/src/test/java/org/springframework/cloud/kubernetes/fabric8/leader/election/it/Fabric8LeaderElectionSimpleITTest.java +++ b/spring-cloud-kubernetes-fabric8-leader/src/test/java/org/springframework/cloud/kubernetes/fabric8/leader/election/it/Fabric8LeaderElectionSimpleITTest.java @@ -78,6 +78,14 @@ static void afterAll() { @Test void test(CapturedOutput output) { + // we have become the leader + Awaitility.await() + .atMost(Duration.ofSeconds(10)) + .pollInterval(Duration.ofSeconds(1)) + .until(() -> output.getOut().contains("simple-it is the new leader")); + + // let's unwind some logs to see that the process is how we expect it to be + // 1. lease is used as the lock (comes from our code) assertThat(output.getOut()).contains( "will use lease as the lock for leader election"); @@ -91,13 +99,9 @@ void test(CapturedOutput output) { "Attempting to acquire leader lease 'LeaseLock: default - spring-k8s-leader-election-lock (simple-it)'"); // 4. we are the leader (comes from our code) - assertThat(output.getOut()).contains("simple-it is now a leader"); + assertThat(output.getOut()).contains("Leader changed from null to simple-it"); + - // wait for a renewal - Awaitility.await() - .pollInterval(Duration.ofSeconds(1)) - .atMost(Duration.ofMinutes(1)) - .until(() -> output.getOut().contains("Attempting to renew leader lease")); // // all these logs happen before a renewal // Assertions.assertThat(output.getOut()).contains("starting leader initiator"); @@ -125,7 +129,9 @@ void test(CapturedOutput output) { // .withName("spring-k8s-leader-election-lock") // .get() // .getSpec() -// .getRenewTime()))); +// .getRenewTime()))) + + } @TestConfiguration From cd1be17ed8dfbdb31a086c62b3fe4e1a8c0faa83 Mon Sep 17 00:00:00 2001 From: wind57 Date: Thu, 23 Oct 2025 10:51:07 +0300 Subject: [PATCH 07/16] wip Signed-off-by: wind57 --- .../election/Fabric8LeaderElectionInitiator.java | 4 ++-- .../it/Fabric8LeaderElectionSimpleITTest.java | 15 ++++++++++----- 2 files changed, 12 insertions(+), 7 deletions(-) diff --git a/spring-cloud-kubernetes-fabric8-leader/src/main/java/org/springframework/cloud/kubernetes/fabric8/leader/election/Fabric8LeaderElectionInitiator.java b/spring-cloud-kubernetes-fabric8-leader/src/main/java/org/springframework/cloud/kubernetes/fabric8/leader/election/Fabric8LeaderElectionInitiator.java index 905ba95ae9..af2ad8a5ca 100644 --- a/spring-cloud-kubernetes-fabric8-leader/src/main/java/org/springframework/cloud/kubernetes/fabric8/leader/election/Fabric8LeaderElectionInitiator.java +++ b/spring-cloud-kubernetes-fabric8-leader/src/main/java/org/springframework/cloud/kubernetes/fabric8/leader/election/Fabric8LeaderElectionInitiator.java @@ -135,7 +135,7 @@ void postConstruct() { @PreDestroy void preDestroy() { destroyCalled = true; - LOG.info(() -> "preDestroy called in the leader initiator : " + candidateIdentity); + LOG.info(() -> "preDestroy called on the leader initiator : " + candidateIdentity); if (podReadyFuture != null && !podReadyFuture.isDone()) { // if the task is not running, this has no effect. @@ -145,7 +145,7 @@ void preDestroy() { } if (leaderFuture != null) { - LOG.info(() -> "leader will be canceled : " + candidateIdentity); + LOG.info(() -> "leaderFuture will be canceled for : " + candidateIdentity); // needed to release the lock, in case we are holding it. // fabric8 internally expects this one to be called leaderFuture.cancel(true); diff --git a/spring-cloud-kubernetes-fabric8-leader/src/test/java/org/springframework/cloud/kubernetes/fabric8/leader/election/it/Fabric8LeaderElectionSimpleITTest.java b/spring-cloud-kubernetes-fabric8-leader/src/test/java/org/springframework/cloud/kubernetes/fabric8/leader/election/it/Fabric8LeaderElectionSimpleITTest.java index f8c1cee785..83cc9053c0 100644 --- a/spring-cloud-kubernetes-fabric8-leader/src/test/java/org/springframework/cloud/kubernetes/fabric8/leader/election/it/Fabric8LeaderElectionSimpleITTest.java +++ b/spring-cloud-kubernetes-fabric8-leader/src/test/java/org/springframework/cloud/kubernetes/fabric8/leader/election/it/Fabric8LeaderElectionSimpleITTest.java @@ -29,6 +29,7 @@ import org.mockito.MockedStatic; import org.mockito.Mockito; import org.springframework.cloud.kubernetes.commons.leader.LeaderUtils; +import org.springframework.cloud.kubernetes.commons.leader.election.LeaderElectionProperties; import org.testcontainers.k3s.K3sContainer; import org.springframework.beans.factory.annotation.Autowired; @@ -80,7 +81,7 @@ void test(CapturedOutput output) { // we have become the leader Awaitility.await() - .atMost(Duration.ofSeconds(10)) + .atMost(Duration.ofSeconds(60)) .pollInterval(Duration.ofSeconds(1)) .until(() -> output.getOut().contains("simple-it is the new leader")); @@ -101,12 +102,16 @@ void test(CapturedOutput output) { // 4. we are the leader (comes from our code) assertThat(output.getOut()).contains("Leader changed from null to simple-it"); + // 5. wait until a renewal happens (comes from fabric code) + // this one means that we have extended our leadership + Awaitility.await() + .atMost(Duration.ofSeconds(15)) + .pollInterval(Duration.ofSeconds(1)) + .until(() -> output.getOut().contains( + "Attempting to renew leader lease 'LeaseLock: default - spring-k8s-leader-election-lock (simple-it)'")); + -// // all these logs happen before a renewal -// Assertions.assertThat(output.getOut()).contains("starting leader initiator"); -// Assertions.assertThat(output.getOut()).contains("Leader election started"); -// Assertions.assertThat(output.getOut()).contains("Successfully Acquired leader lease"); // // Lease lockLease = kubernetesClient.leases() // .inNamespace("default") From 6c8db4c5d3296f3e810a90c146eb171b427f4360 Mon Sep 17 00:00:00 2001 From: wind57 Date: Thu, 23 Oct 2025 19:41:37 +0300 Subject: [PATCH 08/16] wip Signed-off-by: wind57 --- .../it/Fabric8LeaderElectionSimpleITTest.java | 17 ++++++++++------- 1 file changed, 10 insertions(+), 7 deletions(-) diff --git a/spring-cloud-kubernetes-fabric8-leader/src/test/java/org/springframework/cloud/kubernetes/fabric8/leader/election/it/Fabric8LeaderElectionSimpleITTest.java b/spring-cloud-kubernetes-fabric8-leader/src/test/java/org/springframework/cloud/kubernetes/fabric8/leader/election/it/Fabric8LeaderElectionSimpleITTest.java index 83cc9053c0..c7df875d5a 100644 --- a/spring-cloud-kubernetes-fabric8-leader/src/test/java/org/springframework/cloud/kubernetes/fabric8/leader/election/it/Fabric8LeaderElectionSimpleITTest.java +++ b/spring-cloud-kubernetes-fabric8-leader/src/test/java/org/springframework/cloud/kubernetes/fabric8/leader/election/it/Fabric8LeaderElectionSimpleITTest.java @@ -44,7 +44,7 @@ import static org.assertj.core.api.Assertions.assertThat; /** - * A very simple test where we are the sole participant in the leader + * A simple test where we are the sole participant in the leader * election and everything goes fine from start to end. It's a happy path * scenario test. * @@ -99,10 +99,14 @@ void test(CapturedOutput output) { assertThat(output.getOut()).contains( "Attempting to acquire leader lease 'LeaseLock: default - spring-k8s-leader-election-lock (simple-it)'"); - // 4. we are the leader (comes from our code) + // 4. lease has been acquired + assertThat(output.getOut()).contains( + "Acquired lease 'LeaseLock: default - spring-k8s-leader-election-lock (simple-it)'"); + + // 5. we are the leader (comes from our code) assertThat(output.getOut()).contains("Leader changed from null to simple-it"); - // 5. wait until a renewal happens (comes from fabric code) + // 6. wait until a renewal happens (comes from fabric code) // this one means that we have extended our leadership Awaitility.await() .atMost(Duration.ofSeconds(15)) @@ -112,11 +116,10 @@ void test(CapturedOutput output) { + + // -// Lease lockLease = kubernetesClient.leases() -// .inNamespace("default") -// .withName("spring-k8s-leader-election-lock") -// .get(); + // ZonedDateTime currentAcquiredTime = lockLease.getSpec().getAcquireTime(); // Assertions.assertThat(currentAcquiredTime).isNotNull(); // Assertions.assertThat(lockLease.getSpec().getLeaseDurationSeconds()).isEqualTo(15); From 933b7d2055614f4832d15b3c5b3b0991404d24bd Mon Sep 17 00:00:00 2001 From: wind57 Date: Wed, 29 Oct 2025 18:01:46 +0200 Subject: [PATCH 09/16] wip Signed-off-by: wind57 --- .../election/LeaderElectionCallbacks.java | 20 +-- .../leader/election/PodReadyRunner.java | 4 +- .../election/events/LeaderElectionEvent.java | 37 ++++ .../election/events/NewLeaderEvent.java | 11 +- .../election/events/StartLeadingEvent.java | 11 +- .../election/events/StopLeadingEvent.java | 11 +- .../leader/election/PodReadyRunnerTests.java | 8 +- ...abric8LeaderElectionAutoConfiguration.java | 45 +++-- .../Fabric8LeaderElectionCallbacks.java | 2 +- ...derElectionCallbacksAutoConfiguration.java | 2 +- .../Fabric8LeaderElectionInfoContributor.java | 12 +- .../Fabric8LeaderElectionInitiator.java | 64 +++---- .../leader/election/Fabric8LeaderApp.java | 2 +- .../Fabric8LeaderAutoConfigurationTests.java | 14 +- ...erElectionInfoContributorIsLeaderTest.java | 12 +- ...lectionInfoContributorIsNotLeaderTest.java | 10 +- ...ic8LeaderOldAndNewImplementationTests.java | 12 +- .../it/Fabric8LeaderElectionSimpleITTest.java | 161 ------------------ .../pom.xml | 5 +- .../pom.xml | 53 ++++++ .../election/AbstractLeaderElection.java | 127 ++++++++++++++ .../fabric8/leader/election/App.java | 29 ++++ ...ric8LeaderElectionReadinessCanceledIT.java | 90 ++++++++++ ...Fabric8LeaderElectionReadinessFailsIT.java | 94 ++++++++++ ...abric8LeaderElectionReadinessPassesIT.java | 92 ++++++++++ .../Fabric8LeaderElectionSimpleIT.java | 105 ++++++++++++ .../src/test/resources/logback-test.xml | 0 27 files changed, 745 insertions(+), 288 deletions(-) create mode 100644 spring-cloud-kubernetes-commons/src/main/java/org/springframework/cloud/kubernetes/commons/leader/election/events/LeaderElectionEvent.java delete mode 100644 spring-cloud-kubernetes-fabric8-leader/src/test/java/org/springframework/cloud/kubernetes/fabric8/leader/election/it/Fabric8LeaderElectionSimpleITTest.java create mode 100644 spring-cloud-kubernetes-integration-tests/spring-cloud-kubernetes-fabric8-leader-election/pom.xml create mode 100644 spring-cloud-kubernetes-integration-tests/spring-cloud-kubernetes-fabric8-leader-election/src/test/java/org/springframework/cloud/kubernetes/fabric8/leader/election/AbstractLeaderElection.java create mode 100644 spring-cloud-kubernetes-integration-tests/spring-cloud-kubernetes-fabric8-leader-election/src/test/java/org/springframework/cloud/kubernetes/fabric8/leader/election/App.java create mode 100644 spring-cloud-kubernetes-integration-tests/spring-cloud-kubernetes-fabric8-leader-election/src/test/java/org/springframework/cloud/kubernetes/fabric8/leader/election/Fabric8LeaderElectionReadinessCanceledIT.java create mode 100644 spring-cloud-kubernetes-integration-tests/spring-cloud-kubernetes-fabric8-leader-election/src/test/java/org/springframework/cloud/kubernetes/fabric8/leader/election/Fabric8LeaderElectionReadinessFailsIT.java create mode 100644 spring-cloud-kubernetes-integration-tests/spring-cloud-kubernetes-fabric8-leader-election/src/test/java/org/springframework/cloud/kubernetes/fabric8/leader/election/Fabric8LeaderElectionReadinessPassesIT.java create mode 100644 spring-cloud-kubernetes-integration-tests/spring-cloud-kubernetes-fabric8-leader-election/src/test/java/org/springframework/cloud/kubernetes/fabric8/leader/election/Fabric8LeaderElectionSimpleIT.java rename {spring-cloud-kubernetes-fabric8-leader => spring-cloud-kubernetes-integration-tests/spring-cloud-kubernetes-fabric8-leader-election}/src/test/resources/logback-test.xml (100%) diff --git a/spring-cloud-kubernetes-commons/src/main/java/org/springframework/cloud/kubernetes/commons/leader/election/LeaderElectionCallbacks.java b/spring-cloud-kubernetes-commons/src/main/java/org/springframework/cloud/kubernetes/commons/leader/election/LeaderElectionCallbacks.java index dc919afd21..a1c463c0b8 100644 --- a/spring-cloud-kubernetes-commons/src/main/java/org/springframework/cloud/kubernetes/commons/leader/election/LeaderElectionCallbacks.java +++ b/spring-cloud-kubernetes-commons/src/main/java/org/springframework/cloud/kubernetes/commons/leader/election/LeaderElectionCallbacks.java @@ -38,7 +38,7 @@ public class LeaderElectionCallbacks { private static final LogAccessor LOG = new LogAccessor(LeaderElectionCallbacks.class); @Bean - public final String holderIdentity() throws UnknownHostException { + public final String candidateIdentity() throws UnknownHostException { String podHostName = LeaderUtils.hostName(); LOG.debug(() -> "using pod hostname : " + podHostName); return podHostName; @@ -53,22 +53,22 @@ public final String podNamespace() { @Bean public final Runnable onStartLeadingCallback(ApplicationEventPublisher applicationEventPublisher, - String holderIdentity, LeaderElectionProperties properties) { + String candidateIdentity, LeaderElectionProperties properties) { return () -> { - LOG.info(() -> holderIdentity + " is now a leader"); + LOG.info(() -> candidateIdentity + " is now a leader"); if (properties.publishEvents()) { - applicationEventPublisher.publishEvent(new StartLeadingEvent(holderIdentity)); + applicationEventPublisher.publishEvent(new StartLeadingEvent(candidateIdentity)); } }; } @Bean public final Runnable onStopLeadingCallback(ApplicationEventPublisher applicationEventPublisher, - String holderIdentity, LeaderElectionProperties properties) { + String candidateIdentity, LeaderElectionProperties properties) { return () -> { - LOG.info(() -> "id : " + holderIdentity + " stopped being a leader"); + LOG.info(() -> "id : " + candidateIdentity + " stopped being a leader"); if (properties.publishEvents()) { - applicationEventPublisher.publishEvent(new StopLeadingEvent(holderIdentity)); + applicationEventPublisher.publishEvent(new StopLeadingEvent(candidateIdentity)); } }; } @@ -76,10 +76,10 @@ public final Runnable onStopLeadingCallback(ApplicationEventPublisher applicatio @Bean public final Consumer onNewLeaderCallback(ApplicationEventPublisher applicationEventPublisher, LeaderElectionProperties properties) { - return holderIdentity -> { - LOG.info(() -> "id : " + holderIdentity + " is the new leader"); + return candidateIdentity -> { + LOG.info(() -> "id : " + candidateIdentity + " is the new leader"); if (properties.publishEvents()) { - applicationEventPublisher.publishEvent(new NewLeaderEvent(holderIdentity)); + applicationEventPublisher.publishEvent(new NewLeaderEvent(candidateIdentity)); } }; } diff --git a/spring-cloud-kubernetes-commons/src/main/java/org/springframework/cloud/kubernetes/commons/leader/election/PodReadyRunner.java b/spring-cloud-kubernetes-commons/src/main/java/org/springframework/cloud/kubernetes/commons/leader/election/PodReadyRunner.java index c98197728d..a9051362b9 100644 --- a/spring-cloud-kubernetes-commons/src/main/java/org/springframework/cloud/kubernetes/commons/leader/election/PodReadyRunner.java +++ b/spring-cloud-kubernetes-commons/src/main/java/org/springframework/cloud/kubernetes/commons/leader/election/PodReadyRunner.java @@ -68,8 +68,8 @@ public CompletableFuture podReady(BooleanSupplier podReadySupplier) { } } catch (Exception e) { - LOG.error(() -> "exception waiting for pod : " + e.getMessage()); - LOG.error(() -> "leader election for : " + candidateIdentity + " was not successful"); + LOG.error(() -> "exception waiting for pod : " + candidateIdentity); + LOG.error(() -> "pod readiness for : " + candidateIdentity + " failed with : " + e.getMessage()); podReadyFuture.completeExceptionally(e); } diff --git a/spring-cloud-kubernetes-commons/src/main/java/org/springframework/cloud/kubernetes/commons/leader/election/events/LeaderElectionEvent.java b/spring-cloud-kubernetes-commons/src/main/java/org/springframework/cloud/kubernetes/commons/leader/election/events/LeaderElectionEvent.java new file mode 100644 index 0000000000..90919f26e6 --- /dev/null +++ b/spring-cloud-kubernetes-commons/src/main/java/org/springframework/cloud/kubernetes/commons/leader/election/events/LeaderElectionEvent.java @@ -0,0 +1,37 @@ +/* + * Copyright 2013-present the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.springframework.cloud.kubernetes.commons.leader.election.events; + +import org.springframework.context.ApplicationEvent; + +/** + * @author wind57 + */ +abstract class LeaderElectionEvent extends ApplicationEvent { + + private final String candidateIdentity; + + LeaderElectionEvent(Object source) { + super(source); + candidateIdentity = (String) source; + } + + public String candidateIdentity() { + return candidateIdentity; + } + +} diff --git a/spring-cloud-kubernetes-commons/src/main/java/org/springframework/cloud/kubernetes/commons/leader/election/events/NewLeaderEvent.java b/spring-cloud-kubernetes-commons/src/main/java/org/springframework/cloud/kubernetes/commons/leader/election/events/NewLeaderEvent.java index 727dce8ba4..7d50ddf930 100644 --- a/spring-cloud-kubernetes-commons/src/main/java/org/springframework/cloud/kubernetes/commons/leader/election/events/NewLeaderEvent.java +++ b/spring-cloud-kubernetes-commons/src/main/java/org/springframework/cloud/kubernetes/commons/leader/election/events/NewLeaderEvent.java @@ -16,19 +16,10 @@ package org.springframework.cloud.kubernetes.commons.leader.election.events; -import org.springframework.context.ApplicationEvent; - -public final class NewLeaderEvent extends ApplicationEvent { - - private final String holderIdentity; +public final class NewLeaderEvent extends LeaderElectionEvent { public NewLeaderEvent(Object source) { super(source); - holderIdentity = (String) source; - } - - public String holderIdentity() { - return holderIdentity; } } diff --git a/spring-cloud-kubernetes-commons/src/main/java/org/springframework/cloud/kubernetes/commons/leader/election/events/StartLeadingEvent.java b/spring-cloud-kubernetes-commons/src/main/java/org/springframework/cloud/kubernetes/commons/leader/election/events/StartLeadingEvent.java index 2d580d7013..4fb682493d 100644 --- a/spring-cloud-kubernetes-commons/src/main/java/org/springframework/cloud/kubernetes/commons/leader/election/events/StartLeadingEvent.java +++ b/spring-cloud-kubernetes-commons/src/main/java/org/springframework/cloud/kubernetes/commons/leader/election/events/StartLeadingEvent.java @@ -16,22 +16,13 @@ package org.springframework.cloud.kubernetes.commons.leader.election.events; -import org.springframework.context.ApplicationEvent; - /** * @author wind57 */ -public final class StartLeadingEvent extends ApplicationEvent { - - private final String holderIdentity; +public final class StartLeadingEvent extends LeaderElectionEvent { public StartLeadingEvent(Object source) { super(source); - holderIdentity = (String) source; - } - - public String holderIdentity() { - return holderIdentity; } } diff --git a/spring-cloud-kubernetes-commons/src/main/java/org/springframework/cloud/kubernetes/commons/leader/election/events/StopLeadingEvent.java b/spring-cloud-kubernetes-commons/src/main/java/org/springframework/cloud/kubernetes/commons/leader/election/events/StopLeadingEvent.java index ee0bc1abbe..9d9dce466f 100644 --- a/spring-cloud-kubernetes-commons/src/main/java/org/springframework/cloud/kubernetes/commons/leader/election/events/StopLeadingEvent.java +++ b/spring-cloud-kubernetes-commons/src/main/java/org/springframework/cloud/kubernetes/commons/leader/election/events/StopLeadingEvent.java @@ -16,22 +16,13 @@ package org.springframework.cloud.kubernetes.commons.leader.election.events; -import org.springframework.context.ApplicationEvent; - /** * @author wind57 */ -public final class StopLeadingEvent extends ApplicationEvent { - - private final String holderIdentity; +public final class StopLeadingEvent extends LeaderElectionEvent { public StopLeadingEvent(Object source) { super(source); - holderIdentity = (String) source; - } - - public String holderIdentity() { - return holderIdentity; } } diff --git a/spring-cloud-kubernetes-commons/src/test/java/org/springframework/cloud/kubernetes/commons/leader/election/PodReadyRunnerTests.java b/spring-cloud-kubernetes-commons/src/test/java/org/springframework/cloud/kubernetes/commons/leader/election/PodReadyRunnerTests.java index 559e2e35cd..61c8493a6f 100644 --- a/spring-cloud-kubernetes-commons/src/test/java/org/springframework/cloud/kubernetes/commons/leader/election/PodReadyRunnerTests.java +++ b/spring-cloud-kubernetes-commons/src/test/java/org/springframework/cloud/kubernetes/commons/leader/election/PodReadyRunnerTests.java @@ -117,8 +117,8 @@ void readinessFailsOnTheSecondCycle(CapturedOutput output) { caught = true; assertThat(output.getOut()) .contains("Pod : identity in namespace : namespace is not ready, will retry in one second"); - assertThat(output.getOut()).contains("exception waiting for pod : fail on the second cycle"); - assertThat(output.getOut()).contains("leader election for : identity was not successful"); + assertThat(output.getOut()).contains("exception waiting for pod : identity"); + assertThat(output.getOut()).contains("pod readiness for : identity failed with : fail on the second cycle"); assertThat(output.getOut()).contains("canceling scheduled future because readiness failed"); await().atMost(Duration.ofSeconds(3)) @@ -164,8 +164,8 @@ void readinessFailsOnTheSecondCycleAttachNewPipeline(CapturedOutput output) { caught = true; assertThat(output.getOut()) .contains("Pod : identity in namespace : namespace is not ready, will retry in one second"); - assertThat(output.getOut()).contains("exception waiting for pod : fail on the second cycle"); - assertThat(output.getOut()).contains("leader election for : identity was not successful"); + assertThat(output.getOut()).contains("exception waiting for pod : identity"); + assertThat(output.getOut()).contains("pod readiness for : identity failed with : fail on the second cycle"); assertThat(output.getOut()).contains("readiness failed and we caught that"); assertThat(output.getOut()).contains("canceling scheduled future because readiness failed"); diff --git a/spring-cloud-kubernetes-fabric8-leader/src/main/java/org/springframework/cloud/kubernetes/fabric8/leader/election/Fabric8LeaderElectionAutoConfiguration.java b/spring-cloud-kubernetes-fabric8-leader/src/main/java/org/springframework/cloud/kubernetes/fabric8/leader/election/Fabric8LeaderElectionAutoConfiguration.java index 0bec6c4ecc..56f7d5231d 100644 --- a/spring-cloud-kubernetes-fabric8-leader/src/main/java/org/springframework/cloud/kubernetes/fabric8/leader/election/Fabric8LeaderElectionAutoConfiguration.java +++ b/spring-cloud-kubernetes-fabric8-leader/src/main/java/org/springframework/cloud/kubernetes/fabric8/leader/election/Fabric8LeaderElectionAutoConfiguration.java @@ -16,15 +16,19 @@ package org.springframework.cloud.kubernetes.fabric8.leader.election; +import java.util.function.BooleanSupplier; + import io.fabric8.kubernetes.api.model.APIResource; import io.fabric8.kubernetes.api.model.APIResourceList; import io.fabric8.kubernetes.api.model.GroupVersionForDiscovery; +import io.fabric8.kubernetes.api.model.Pod; import io.fabric8.kubernetes.client.KubernetesClient; import io.fabric8.kubernetes.client.extended.leaderelection.LeaderElectionConfig; import io.fabric8.kubernetes.client.extended.leaderelection.LeaderElectionConfigBuilder; import io.fabric8.kubernetes.client.extended.leaderelection.resourcelock.ConfigMapLock; import io.fabric8.kubernetes.client.extended.leaderelection.resourcelock.LeaseLock; import io.fabric8.kubernetes.client.extended.leaderelection.resourcelock.Lock; +import io.fabric8.kubernetes.client.readiness.Readiness; import org.springframework.boot.actuate.info.InfoContributor; import org.springframework.boot.autoconfigure.AutoConfigureAfter; @@ -63,26 +67,35 @@ class Fabric8LeaderElectionAutoConfiguration { @Bean @ConditionalOnClass(InfoContributor.class) @ConditionalOnEnabledHealthIndicator("leader.election") - Fabric8LeaderElectionInfoContributor leaderElectionInfoContributor(String holderIdentity, - LeaderElectionConfig leaderElectionConfig, KubernetesClient fabric8KubernetesClient) { - return new Fabric8LeaderElectionInfoContributor(holderIdentity, leaderElectionConfig, fabric8KubernetesClient); + Fabric8LeaderElectionInfoContributor leaderElectionInfoContributor(String candidateIdentity, + LeaderElectionConfig leaderElectionConfig, KubernetesClient fabric8KubernetesClient) { + return new Fabric8LeaderElectionInfoContributor(candidateIdentity, leaderElectionConfig, + fabric8KubernetesClient); } @Bean @ConditionalOnMissingBean - Fabric8LeaderElectionInitiator fabric8LeaderElectionInitiator(String holderIdentity, String podNamespace, - KubernetesClient fabric8KubernetesClient, LeaderElectionConfig fabric8LeaderElectionConfig, - LeaderElectionProperties leaderElectionProperties) { - return new Fabric8LeaderElectionInitiator(holderIdentity, podNamespace, fabric8KubernetesClient, - fabric8LeaderElectionConfig, leaderElectionProperties); + Fabric8LeaderElectionInitiator fabric8LeaderElectionInitiator(String candidateIdentity, String podNamespace, + KubernetesClient fabric8KubernetesClient, LeaderElectionConfig fabric8LeaderElectionConfig, + LeaderElectionProperties leaderElectionProperties, BooleanSupplier podReadySupplier) { + return new Fabric8LeaderElectionInitiator(candidateIdentity, podNamespace, fabric8KubernetesClient, + fabric8LeaderElectionConfig, leaderElectionProperties, podReadySupplier); + } + + @Bean + BooleanSupplier podReadySupplier(KubernetesClient fabric8KubernetesClient, String candidateIdentity, + String podNamespace) { + return () -> { + Pod pod = fabric8KubernetesClient.pods().inNamespace(podNamespace).withName(candidateIdentity).get(); + return Readiness.isPodReady(pod); + }; } @Bean @ConditionalOnMissingBean LeaderElectionConfig fabric8LeaderElectionConfig(LeaderElectionProperties properties, Lock lock, - Fabric8LeaderElectionCallbacks fabric8LeaderElectionCallbacks) { - return new LeaderElectionConfigBuilder() - .withReleaseOnCancel() + Fabric8LeaderElectionCallbacks fabric8LeaderElectionCallbacks) { + return new LeaderElectionConfigBuilder().withReleaseOnCancel() .withName("Spring k8s leader election") .withLeaseDuration(properties.leaseDuration()) .withLock(lock) @@ -94,7 +107,7 @@ LeaderElectionConfig fabric8LeaderElectionConfig(LeaderElectionProperties proper @Bean @ConditionalOnMissingBean - Lock lock(KubernetesClient fabric8KubernetesClient, LeaderElectionProperties properties, String holderIdentity) { + Lock lock(KubernetesClient fabric8KubernetesClient, LeaderElectionProperties properties, String candidateIdentity) { boolean leaseSupported = fabric8KubernetesClient.getApiGroups() .getGroups() .stream() @@ -111,17 +124,17 @@ Lock lock(KubernetesClient fabric8KubernetesClient, LeaderElectionProperties pro if (leaseSupported) { if (properties.useConfigMapAsLock()) { LOG.info(() -> "leases are supported on the cluster, but config map will be used " - + "(because 'spring.cloud.kubernetes.leader.election.use-config-map-as-lock=true')"); - return new ConfigMapLock(properties.lockNamespace(), properties.lockName(), holderIdentity); + + "(because 'spring.cloud.kubernetes.leader.election.use-config-map-as-lock=true')"); + return new ConfigMapLock(properties.lockNamespace(), properties.lockName(), candidateIdentity); } else { LOG.info(() -> "will use lease as the lock for leader election"); - return new LeaseLock(properties.lockNamespace(), properties.lockName(), holderIdentity); + return new LeaseLock(properties.lockNamespace(), properties.lockName(), candidateIdentity); } } else { LOG.info(() -> "will use configmap as the lock for leader election"); - return new ConfigMapLock(properties.lockNamespace(), properties.lockName(), holderIdentity); + return new ConfigMapLock(properties.lockNamespace(), properties.lockName(), candidateIdentity); } } diff --git a/spring-cloud-kubernetes-fabric8-leader/src/main/java/org/springframework/cloud/kubernetes/fabric8/leader/election/Fabric8LeaderElectionCallbacks.java b/spring-cloud-kubernetes-fabric8-leader/src/main/java/org/springframework/cloud/kubernetes/fabric8/leader/election/Fabric8LeaderElectionCallbacks.java index 910d7fb3a7..16966cb981 100644 --- a/spring-cloud-kubernetes-fabric8-leader/src/main/java/org/springframework/cloud/kubernetes/fabric8/leader/election/Fabric8LeaderElectionCallbacks.java +++ b/spring-cloud-kubernetes-fabric8-leader/src/main/java/org/springframework/cloud/kubernetes/fabric8/leader/election/Fabric8LeaderElectionCallbacks.java @@ -1,5 +1,5 @@ /* - * Copyright 2013-2024 the original author or authors. + * Copyright 2013-present the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. diff --git a/spring-cloud-kubernetes-fabric8-leader/src/main/java/org/springframework/cloud/kubernetes/fabric8/leader/election/Fabric8LeaderElectionCallbacksAutoConfiguration.java b/spring-cloud-kubernetes-fabric8-leader/src/main/java/org/springframework/cloud/kubernetes/fabric8/leader/election/Fabric8LeaderElectionCallbacksAutoConfiguration.java index 7e1faddede..14f4255985 100644 --- a/spring-cloud-kubernetes-fabric8-leader/src/main/java/org/springframework/cloud/kubernetes/fabric8/leader/election/Fabric8LeaderElectionCallbacksAutoConfiguration.java +++ b/spring-cloud-kubernetes-fabric8-leader/src/main/java/org/springframework/cloud/kubernetes/fabric8/leader/election/Fabric8LeaderElectionCallbacksAutoConfiguration.java @@ -45,7 +45,7 @@ final class Fabric8LeaderElectionCallbacksAutoConfiguration extends LeaderElecti @Bean @ConditionalOnMissingBean Fabric8LeaderElectionCallbacks fabric8LeaderElectionCallbacks(Runnable onStartLeadingCallback, - Runnable onStopLeadingCallback, Consumer onNewLeaderCallback) { + Runnable onStopLeadingCallback, Consumer onNewLeaderCallback) { return new Fabric8LeaderElectionCallbacks(onStartLeadingCallback, onStopLeadingCallback, onNewLeaderCallback); } diff --git a/spring-cloud-kubernetes-fabric8-leader/src/main/java/org/springframework/cloud/kubernetes/fabric8/leader/election/Fabric8LeaderElectionInfoContributor.java b/spring-cloud-kubernetes-fabric8-leader/src/main/java/org/springframework/cloud/kubernetes/fabric8/leader/election/Fabric8LeaderElectionInfoContributor.java index 706a2fbb4d..8852ce8777 100644 --- a/spring-cloud-kubernetes-fabric8-leader/src/main/java/org/springframework/cloud/kubernetes/fabric8/leader/election/Fabric8LeaderElectionInfoContributor.java +++ b/spring-cloud-kubernetes-fabric8-leader/src/main/java/org/springframework/cloud/kubernetes/fabric8/leader/election/Fabric8LeaderElectionInfoContributor.java @@ -31,15 +31,15 @@ */ final class Fabric8LeaderElectionInfoContributor implements InfoContributor { - private final String holderIdentity; + private final String candidateIdentity; private final LeaderElectionConfig leaderElectionConfig; private final KubernetesClient fabric8KubernetesClient; - Fabric8LeaderElectionInfoContributor(String holderIdentity, LeaderElectionConfig leaderElectionConfig, - KubernetesClient fabric8KubernetesClient) { - this.holderIdentity = holderIdentity; + Fabric8LeaderElectionInfoContributor(String candidateIdentity, LeaderElectionConfig leaderElectionConfig, + KubernetesClient fabric8KubernetesClient) { + this.candidateIdentity = candidateIdentity; this.leaderElectionConfig = leaderElectionConfig; this.fabric8KubernetesClient = fabric8KubernetesClient; } @@ -49,8 +49,8 @@ public void contribute(Info.Builder builder) { Map details = new HashMap<>(); Optional.ofNullable(leaderElectionConfig.getLock().get(fabric8KubernetesClient)) .ifPresentOrElse(leaderRecord -> { - boolean isLeader = holderIdentity.equals(leaderRecord.getHolderIdentity()); - details.put("leaderId", holderIdentity); + boolean isLeader = candidateIdentity.equals(leaderRecord.getHolderIdentity()); + details.put("leaderId", candidateIdentity); details.put("isLeader", isLeader); }, () -> details.put("leaderId", "Unknown")); diff --git a/spring-cloud-kubernetes-fabric8-leader/src/main/java/org/springframework/cloud/kubernetes/fabric8/leader/election/Fabric8LeaderElectionInitiator.java b/spring-cloud-kubernetes-fabric8-leader/src/main/java/org/springframework/cloud/kubernetes/fabric8/leader/election/Fabric8LeaderElectionInitiator.java index af2ad8a5ca..81702b3ea7 100644 --- a/spring-cloud-kubernetes-fabric8-leader/src/main/java/org/springframework/cloud/kubernetes/fabric8/leader/election/Fabric8LeaderElectionInitiator.java +++ b/spring-cloud-kubernetes-fabric8-leader/src/main/java/org/springframework/cloud/kubernetes/fabric8/leader/election/Fabric8LeaderElectionInitiator.java @@ -1,18 +1,19 @@ /* -* Copyright 2013-2024 the original author or authors. -* -* Licensed under the Apache License, Version 2.0 (the "License"); -* you may not use this file except in compliance with the License. -* You may obtain a copy of the License at -* -* https://www.apache.org/licenses/LICENSE-2.0 -* -* Unless required by applicable law or agreed to in writing, software -* distributed under the License is distributed on an "AS IS" BASIS, -* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -* See the License for the specific language governing permissions and -* limitations under the License. -*/ + * Copyright 2013-present the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + package org.springframework.cloud.kubernetes.fabric8.leader.election; import java.util.concurrent.CancellationException; @@ -21,11 +22,9 @@ import java.util.concurrent.TimeUnit; import java.util.function.BooleanSupplier; -import io.fabric8.kubernetes.api.model.Pod; import io.fabric8.kubernetes.client.KubernetesClient; import io.fabric8.kubernetes.client.extended.leaderelection.LeaderElectionConfig; import io.fabric8.kubernetes.client.extended.leaderelection.LeaderElector; -import io.fabric8.kubernetes.client.readiness.Readiness; import jakarta.annotation.PostConstruct; import jakarta.annotation.PreDestroy; @@ -38,7 +37,7 @@ /** * @author wind57 */ - final class Fabric8LeaderElectionInitiator { +final class Fabric8LeaderElectionInitiator { private static final LogAccessor LOG = new LogAccessor(Fabric8LeaderElectionInitiator.class); @@ -66,20 +65,16 @@ final class Fabric8LeaderElectionInitiator { Fabric8LeaderElectionInitiator(String candidateIdentity, String candidateNamespace, KubernetesClient fabric8KubernetesClient, LeaderElectionConfig leaderElectionConfig, - LeaderElectionProperties leaderElectionProperties) { + LeaderElectionProperties leaderElectionProperties, BooleanSupplier podReadySupplier) { this.candidateIdentity = candidateIdentity; this.fabric8KubernetesClient = fabric8KubernetesClient; this.leaderElectionConfig = leaderElectionConfig; this.leaderElectionProperties = leaderElectionProperties; this.waitForPodReady = leaderElectionProperties.waitForPodReady(); + this.podReadySupplier = podReadySupplier; - this.podReadyWaitingExecutor = newSingleThreadExecutor(runnable -> - new Thread(runnable, "Fabric8LeaderElectionInitiator-" + candidateIdentity)); - - this.podReadySupplier = () -> { - Pod pod = fabric8KubernetesClient.pods().inNamespace(candidateNamespace).withName(candidateIdentity).get(); - return Readiness.isPodReady(pod); - }; + this.podReadyWaitingExecutor = newSingleThreadExecutor( + runnable -> new Thread(runnable, "Fabric8LeaderElectionInitiator-" + candidateIdentity)); this.podReadyRunner = new PodReadyRunner(candidateIdentity, candidateNamespace); } @@ -110,11 +105,12 @@ void postConstruct() { if (waitForPodReady) { // if 'ready' is already completed at this point, thread will run this, - // otherwise it will attach the pipeline and move on to 'blockReadinessCheck' + // otherwise it will attach the pipeline and move on to + // 'blockReadinessCheck' CompletableFuture ready = podReadyFuture.whenComplete((ok, error) -> { if (error != null) { - LOG.error(() -> "readiness failed for : " + candidateIdentity); - LOG.error(() -> "leader election for : " + candidateIdentity + " will not start"); + LOG.error(() -> "readiness failed for : " + candidateIdentity + + ", leader election will not start"); } else { LOG.info(() -> candidateIdentity + " is ready"); @@ -141,6 +137,7 @@ void preDestroy() { // if the task is not running, this has no effect. // if the task is running, calling this will also make sure // that the caching executor will shut down too. + LOG.debug(() -> "podReadyFuture will be canceled for : " + candidateIdentity); podReadyFuture.cancel(true); } @@ -148,9 +145,13 @@ void preDestroy() { LOG.info(() -> "leaderFuture will be canceled for : " + candidateIdentity); // needed to release the lock, in case we are holding it. // fabric8 internally expects this one to be called + LOG.debug(() -> "leaderFuture will be canceled for : " + candidateIdentity); leaderFuture.cancel(true); } - podReadyWaitingExecutor.shutdownNow(); + if (!podReadyWaitingExecutor.isShutdown()) { + LOG.debug(() -> "podReadyWaitingExecutor will be shutdown for : " + candidateIdentity); + podReadyWaitingExecutor.shutdownNow(); + } } private void startLeaderElection() { @@ -166,7 +167,7 @@ private void startLeaderElection() { if (error instanceof CancellationException) { if (!destroyCalled) { LOG.warn(() -> "renewal failed for : " + candidateIdentity + ", will re-start it after : " - + leaderElectionProperties.waitAfterRenewalFailure().toSeconds() + " seconds"); + + leaderElectionProperties.waitAfterRenewalFailure().toSeconds() + " seconds"); sleep(); startLeaderElection(); } @@ -177,7 +178,8 @@ private void startLeaderElection() { try { leaderFuture.get(); - } catch (Exception e) { + } + catch (Exception e) { LOG.warn(() -> "leader election failed for : " + candidateIdentity + ". Trying to recover..."); } }); diff --git a/spring-cloud-kubernetes-fabric8-leader/src/test/java/org/springframework/cloud/kubernetes/fabric8/leader/election/Fabric8LeaderApp.java b/spring-cloud-kubernetes-fabric8-leader/src/test/java/org/springframework/cloud/kubernetes/fabric8/leader/election/Fabric8LeaderApp.java index f51818e22f..fa224c425a 100644 --- a/spring-cloud-kubernetes-fabric8-leader/src/test/java/org/springframework/cloud/kubernetes/fabric8/leader/election/Fabric8LeaderApp.java +++ b/spring-cloud-kubernetes-fabric8-leader/src/test/java/org/springframework/cloud/kubernetes/fabric8/leader/election/Fabric8LeaderApp.java @@ -1,5 +1,5 @@ /* - * Copyright 2013-2024 the original author or authors. + * Copyright 2013-present the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. diff --git a/spring-cloud-kubernetes-fabric8-leader/src/test/java/org/springframework/cloud/kubernetes/fabric8/leader/election/Fabric8LeaderAutoConfigurationTests.java b/spring-cloud-kubernetes-fabric8-leader/src/test/java/org/springframework/cloud/kubernetes/fabric8/leader/election/Fabric8LeaderAutoConfigurationTests.java index 6979c2d1d3..f878317c98 100644 --- a/spring-cloud-kubernetes-fabric8-leader/src/test/java/org/springframework/cloud/kubernetes/fabric8/leader/election/Fabric8LeaderAutoConfigurationTests.java +++ b/spring-cloud-kubernetes-fabric8-leader/src/test/java/org/springframework/cloud/kubernetes/fabric8/leader/election/Fabric8LeaderAutoConfigurationTests.java @@ -46,8 +46,8 @@ class Fabric8LeaderAutoConfigurationTests { void leaderElectionAnnotationMissing() { new ApplicationContextRunner().withUserConfiguration(Fabric8LeaderApp.class) .withConfiguration(AutoConfigurations.of(Fabric8LeaderAutoConfiguration.class, - Fabric8LeaderElectionAutoConfiguration.class, - Fabric8LeaderElectionCallbacksAutoConfiguration.class)) + Fabric8LeaderElectionAutoConfiguration.class, + Fabric8LeaderElectionCallbacksAutoConfiguration.class)) .run(context -> { // this one comes from Fabric8LeaderElectionAutoConfiguration @@ -72,8 +72,8 @@ void leaderElectionAnnotationMissing() { void leaderElectionAnnotationPresentEqualToFalse() { new ApplicationContextRunner().withUserConfiguration(Fabric8LeaderApp.class) .withConfiguration(AutoConfigurations.of(Fabric8LeaderAutoConfiguration.class, - Fabric8LeaderElectionAutoConfiguration.class, - Fabric8LeaderElectionCallbacksAutoConfiguration.class)) + Fabric8LeaderElectionAutoConfiguration.class, + Fabric8LeaderElectionCallbacksAutoConfiguration.class)) .withPropertyValues("spring.cloud.kubernetes.leader.election.enabled=false") .run(context -> { @@ -99,10 +99,10 @@ void leaderElectionAnnotationPresentEqualToFalse() { void leaderElectionAnnotationPresentEqualToTrue() { new ApplicationContextRunner().withUserConfiguration(Fabric8LeaderApp.class) .withConfiguration(AutoConfigurations.of(Fabric8LeaderAutoConfiguration.class, - Fabric8LeaderElectionAutoConfiguration.class, - Fabric8LeaderElectionCallbacksAutoConfiguration.class)) + Fabric8LeaderElectionAutoConfiguration.class, + Fabric8LeaderElectionCallbacksAutoConfiguration.class)) .withPropertyValues("spring.cloud.kubernetes.leader.election.enabled=true", - "spring.main.cloud-platform=kubernetes") + "spring.main.cloud-platform=kubernetes") .run(context -> { // this one comes from Fabric8LeaderElectionAutoConfiguration diff --git a/spring-cloud-kubernetes-fabric8-leader/src/test/java/org/springframework/cloud/kubernetes/fabric8/leader/election/Fabric8LeaderElectionInfoContributorIsLeaderTest.java b/spring-cloud-kubernetes-fabric8-leader/src/test/java/org/springframework/cloud/kubernetes/fabric8/leader/election/Fabric8LeaderElectionInfoContributorIsLeaderTest.java index 1cb0b7525b..007e4f7c87 100644 --- a/spring-cloud-kubernetes-fabric8-leader/src/test/java/org/springframework/cloud/kubernetes/fabric8/leader/election/Fabric8LeaderElectionInfoContributorIsLeaderTest.java +++ b/spring-cloud-kubernetes-fabric8-leader/src/test/java/org/springframework/cloud/kubernetes/fabric8/leader/election/Fabric8LeaderElectionInfoContributorIsLeaderTest.java @@ -1,5 +1,5 @@ /* - * Copyright 2013-2024 the original author or authors. + * Copyright 2013-present the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. @@ -51,9 +51,9 @@ * @author wind57 */ @SpringBootTest(webEnvironment = SpringBootTest.WebEnvironment.RANDOM_PORT, - properties = { "spring.main.cloud-platform=KUBERNETES", "management.endpoints.web.exposure.include=info", - "management.endpoint.info.show-details=always", "management.info.kubernetes.enabled=true", - "spring.cloud.kubernetes.leader.election.enabled=true" }) + properties = { "spring.main.cloud-platform=KUBERNETES", "management.endpoints.web.exposure.include=info", + "management.endpoint.info.show-details=always", "management.info.kubernetes.enabled=true", + "spring.cloud.kubernetes.leader.election.enabled=true" }) @AutoConfigureWebTestClient class Fabric8LeaderElectionInfoContributorIsLeaderTest { @@ -108,8 +108,8 @@ KubernetesClient mockKubernetesClient() { private void mockForLeaseSupport(KubernetesClient client) { Mockito.when(client.getApiResources("coordination.k8s.io/v1")) .thenReturn( - new APIResourceListBuilder().withResources(new APIResourceBuilder().withKind("Lease").build()) - .build()); + new APIResourceListBuilder().withResources(new APIResourceBuilder().withKind("Lease").build()) + .build()); APIGroupList apiGroupList = new APIGroupListBuilder().addNewGroup() .withVersions(new GroupVersionForDiscoveryBuilder().withGroupVersion("coordination.k8s.io/v1").build()) diff --git a/spring-cloud-kubernetes-fabric8-leader/src/test/java/org/springframework/cloud/kubernetes/fabric8/leader/election/Fabric8LeaderElectionInfoContributorIsNotLeaderTest.java b/spring-cloud-kubernetes-fabric8-leader/src/test/java/org/springframework/cloud/kubernetes/fabric8/leader/election/Fabric8LeaderElectionInfoContributorIsNotLeaderTest.java index 069d5037f9..53d0dec9d8 100644 --- a/spring-cloud-kubernetes-fabric8-leader/src/test/java/org/springframework/cloud/kubernetes/fabric8/leader/election/Fabric8LeaderElectionInfoContributorIsNotLeaderTest.java +++ b/spring-cloud-kubernetes-fabric8-leader/src/test/java/org/springframework/cloud/kubernetes/fabric8/leader/election/Fabric8LeaderElectionInfoContributorIsNotLeaderTest.java @@ -51,9 +51,9 @@ * @author wind57 */ @SpringBootTest(webEnvironment = SpringBootTest.WebEnvironment.RANDOM_PORT, - properties = { "spring.main.cloud-platform=KUBERNETES", "management.endpoints.web.exposure.include=info", - "management.endpoint.info.show-details=always", "management.info.kubernetes.enabled=true", - "spring.cloud.kubernetes.leader.election.enabled=true" }) + properties = { "spring.main.cloud-platform=KUBERNETES", "management.endpoints.web.exposure.include=info", + "management.endpoint.info.show-details=always", "management.info.kubernetes.enabled=true", + "spring.cloud.kubernetes.leader.election.enabled=true" }) @AutoConfigureWebTestClient class Fabric8LeaderElectionInfoContributorIsNotLeaderTest { @@ -108,8 +108,8 @@ KubernetesClient mockKubernetesClient() { private void mockForLeaseSupport(KubernetesClient client) { Mockito.when(client.getApiResources("coordination.k8s.io/v1")) .thenReturn( - new APIResourceListBuilder().withResources(new APIResourceBuilder().withKind("Lease").build()) - .build()); + new APIResourceListBuilder().withResources(new APIResourceBuilder().withKind("Lease").build()) + .build()); APIGroupList apiGroupList = new APIGroupListBuilder().addNewGroup() .withVersions(new GroupVersionForDiscoveryBuilder().withGroupVersion("coordination.k8s.io/v1").build()) diff --git a/spring-cloud-kubernetes-fabric8-leader/src/test/java/org/springframework/cloud/kubernetes/fabric8/leader/election/Fabric8LeaderOldAndNewImplementationTests.java b/spring-cloud-kubernetes-fabric8-leader/src/test/java/org/springframework/cloud/kubernetes/fabric8/leader/election/Fabric8LeaderOldAndNewImplementationTests.java index e2e152ff44..589ed75fe3 100644 --- a/spring-cloud-kubernetes-fabric8-leader/src/test/java/org/springframework/cloud/kubernetes/fabric8/leader/election/Fabric8LeaderOldAndNewImplementationTests.java +++ b/spring-cloud-kubernetes-fabric8-leader/src/test/java/org/springframework/cloud/kubernetes/fabric8/leader/election/Fabric8LeaderOldAndNewImplementationTests.java @@ -164,7 +164,7 @@ void newImplementationEnabled() { @Test void bothDisabled() { setup("spring.main.cloud-platform=KUBERNETES", "spring.cloud.kubernetes.leader.enabled=false", - "spring.cloud.kubernetes.leader.election.enabled=false"); + "spring.cloud.kubernetes.leader.election.enabled=false"); applicationContextRunner.run(context -> { assertThat(context).doesNotHaveBean(Fabric8LeaderAutoConfiguration.class); assertThat(context).doesNotHaveBean(Fabric8LeaderElectionAutoConfiguration.class); @@ -187,7 +187,7 @@ void bothDisabled() { @Test void bothEnabled() { setup("spring.main.cloud-platform=KUBERNETES", "spring.cloud.kubernetes.leader.enabled=true", - "spring.cloud.kubernetes.leader.election.enabled=true"); + "spring.cloud.kubernetes.leader.election.enabled=true"); applicationContextRunner.run(context -> { assertThat(context).doesNotHaveBean(Fabric8LeaderAutoConfiguration.class); assertThat(context).hasSingleBean(Fabric8LeaderElectionAutoConfiguration.class); @@ -197,8 +197,8 @@ void bothEnabled() { private void setup(String... properties) { applicationContextRunner = new ApplicationContextRunner() .withConfiguration(AutoConfigurations.of(Fabric8LeaderElectionCallbacksAutoConfiguration.class, - Fabric8AutoConfiguration.class, KubernetesCommonsAutoConfiguration.class, - Fabric8LeaderElectionAutoConfiguration.class, Fabric8LeaderAutoConfiguration.class)) + Fabric8AutoConfiguration.class, KubernetesCommonsAutoConfiguration.class, + Fabric8LeaderElectionAutoConfiguration.class, Fabric8LeaderAutoConfiguration.class)) .withUserConfiguration(Fabric8LeaderOldAndNewImplementationTests.Configuration.class) .withPropertyValues(properties); } @@ -227,8 +227,8 @@ KubernetesClient mockKubernetesClient() { Mockito.when(client.getApiResources("coordination.k8s.io/v1")) .thenReturn( - new APIResourceListBuilder().withResources(new APIResourceBuilder().withKind("Lease").build()) - .build()); + new APIResourceListBuilder().withResources(new APIResourceBuilder().withKind("Lease").build()) + .build()); APIGroupList apiGroupList = new APIGroupListBuilder().addNewGroup() .withVersions(new GroupVersionForDiscoveryBuilder().withGroupVersion("coordination.k8s.io/v1").build()) diff --git a/spring-cloud-kubernetes-fabric8-leader/src/test/java/org/springframework/cloud/kubernetes/fabric8/leader/election/it/Fabric8LeaderElectionSimpleITTest.java b/spring-cloud-kubernetes-fabric8-leader/src/test/java/org/springframework/cloud/kubernetes/fabric8/leader/election/it/Fabric8LeaderElectionSimpleITTest.java deleted file mode 100644 index c7df875d5a..0000000000 --- a/spring-cloud-kubernetes-fabric8-leader/src/test/java/org/springframework/cloud/kubernetes/fabric8/leader/election/it/Fabric8LeaderElectionSimpleITTest.java +++ /dev/null @@ -1,161 +0,0 @@ -/* - * Copyright 2013-present the original author or authors. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * https://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -package org.springframework.cloud.kubernetes.fabric8.leader.election.it; - -import java.time.Duration; - -import io.fabric8.kubernetes.client.Config; -import io.fabric8.kubernetes.client.KubernetesClient; -import io.fabric8.kubernetes.client.KubernetesClientBuilder; -import org.awaitility.Awaitility; -import org.junit.jupiter.api.AfterAll; -import org.junit.jupiter.api.BeforeAll; -import org.junit.jupiter.api.Test; -import org.junit.jupiter.api.extension.ExtendWith; -import org.mockito.MockedStatic; -import org.mockito.Mockito; -import org.springframework.cloud.kubernetes.commons.leader.LeaderUtils; -import org.springframework.cloud.kubernetes.commons.leader.election.LeaderElectionProperties; -import org.testcontainers.k3s.K3sContainer; - -import org.springframework.beans.factory.annotation.Autowired; -import org.springframework.boot.test.context.SpringBootTest; -import org.springframework.boot.test.context.TestConfiguration; -import org.springframework.boot.test.system.CapturedOutput; -import org.springframework.boot.test.system.OutputCaptureExtension; -import org.springframework.cloud.kubernetes.integration.tests.commons.Commons; -import org.springframework.context.annotation.Bean; -import org.springframework.context.annotation.Primary; - -import static org.assertj.core.api.Assertions.assertThat; - -/** - * A simple test where we are the sole participant in the leader - * election and everything goes fine from start to end. It's a happy path - * scenario test. - * - * @author wind57 - */ -@SpringBootTest(webEnvironment = SpringBootTest.WebEnvironment.RANDOM_PORT, - properties = { "spring.main.cloud-platform=KUBERNETES", "spring.cloud.kubernetes.leader.election.enabled=true", - "spring.cloud.kubernetes.leader.election.wait-for-pod-ready=false" }) -@ExtendWith(OutputCaptureExtension.class) -class Fabric8LeaderElectionSimpleITTest { - - private static K3sContainer container; - - private static final MockedStatic LEADER_UTILS_MOCKED_STATIC = Mockito.mockStatic(LeaderUtils.class); - - @Autowired - private KubernetesClient kubernetesClient; - - @BeforeAll - static void beforeAll() { - container = Commons.container(); - container.start(); - - LEADER_UTILS_MOCKED_STATIC.when(LeaderUtils::hostName).thenReturn("simple-it"); - } - - @AfterAll - static void afterAll() { - container.stop(); - } - - @Test - void test(CapturedOutput output) { - - // we have become the leader - Awaitility.await() - .atMost(Duration.ofSeconds(60)) - .pollInterval(Duration.ofSeconds(1)) - .until(() -> output.getOut().contains("simple-it is the new leader")); - - // let's unwind some logs to see that the process is how we expect it to be - - // 1. lease is used as the lock (comes from our code) - assertThat(output.getOut()).contains( - "will use lease as the lock for leader election"); - - // 2. we start leader initiator for our hostname (comes from our code) - assertThat(output.getOut()).contains( - "starting leader initiator : simple-it"); - - // 3. we try to acquire the lease (comes from fabric8 code) - assertThat(output.getOut()).contains( - "Attempting to acquire leader lease 'LeaseLock: default - spring-k8s-leader-election-lock (simple-it)'"); - - // 4. lease has been acquired - assertThat(output.getOut()).contains( - "Acquired lease 'LeaseLock: default - spring-k8s-leader-election-lock (simple-it)'"); - - // 5. we are the leader (comes from our code) - assertThat(output.getOut()).contains("Leader changed from null to simple-it"); - - // 6. wait until a renewal happens (comes from fabric code) - // this one means that we have extended our leadership - Awaitility.await() - .atMost(Duration.ofSeconds(15)) - .pollInterval(Duration.ofSeconds(1)) - .until(() -> output.getOut().contains( - "Attempting to renew leader lease 'LeaseLock: default - spring-k8s-leader-election-lock (simple-it)'")); - - - - - -// - -// ZonedDateTime currentAcquiredTime = lockLease.getSpec().getAcquireTime(); -// Assertions.assertThat(currentAcquiredTime).isNotNull(); -// Assertions.assertThat(lockLease.getSpec().getLeaseDurationSeconds()).isEqualTo(15); -// Assertions.assertThat(lockLease.getSpec().getLeaseTransitions()).isEqualTo(0); -// -// ZonedDateTime currentRenewalTime = lockLease.getSpec().getRenewTime(); -// Assertions.assertThat(currentRenewalTime).isNotNull(); -// -// // renew happened, we renew by default on every two seconds -// Awaitility.await() -// .pollInterval(Duration.ofSeconds(1)) -// .atMost(Duration.ofSeconds(4)) -// .until(() -> !(currentRenewalTime.equals(kubernetesClient.leases() -// .inNamespace("default") -// .withName("spring-k8s-leader-election-lock") -// .get() -// .getSpec() -// .getRenewTime()))) - - - } - - @TestConfiguration - static class LocalConfiguration { - - @Bean - @Primary - KubernetesClient client() { - String kubeConfigYaml = container.getKubeConfigYaml(); - Config config = Config.fromKubeconfig(kubeConfigYaml); - return new KubernetesClientBuilder().withConfig(config).build(); - } - - } - - // test with pod ready - // simulate that we lose leadership, must re-try - -} diff --git a/spring-cloud-kubernetes-integration-tests/pom.xml b/spring-cloud-kubernetes-integration-tests/pom.xml index 7cc3edfb62..03eb3a6df2 100644 --- a/spring-cloud-kubernetes-integration-tests/pom.xml +++ b/spring-cloud-kubernetes-integration-tests/pom.xml @@ -113,5 +113,8 @@ spring-cloud-kubernetes-k8s-client-rabbitmq-secret-reload - + + spring-cloud-kubernetes-fabric8-leader-election + + diff --git a/spring-cloud-kubernetes-integration-tests/spring-cloud-kubernetes-fabric8-leader-election/pom.xml b/spring-cloud-kubernetes-integration-tests/spring-cloud-kubernetes-fabric8-leader-election/pom.xml new file mode 100644 index 0000000000..306a60aa77 --- /dev/null +++ b/spring-cloud-kubernetes-integration-tests/spring-cloud-kubernetes-fabric8-leader-election/pom.xml @@ -0,0 +1,53 @@ + + + 4.0.0 + + org.springframework.cloud + spring-cloud-kubernetes-integration-tests + 5.0.0-SNAPSHOT + + + + true + true + + + spring-cloud-kubernetes-fabric8-leader-election + + + + org.springframework.boot + spring-boot-starter-test + test + + + org.springframework.cloud + spring-cloud-kubernetes-test-support + test + + + org.springframework.cloud + spring-cloud-kubernetes-fabric8-leader + + + org.springframework.boot + spring-boot-starter-actuator + + + + + + + ../src/main/resources + true + + + src/main/resources + true + + + + + diff --git a/spring-cloud-kubernetes-integration-tests/spring-cloud-kubernetes-fabric8-leader-election/src/test/java/org/springframework/cloud/kubernetes/fabric8/leader/election/AbstractLeaderElection.java b/spring-cloud-kubernetes-integration-tests/spring-cloud-kubernetes-fabric8-leader-election/src/test/java/org/springframework/cloud/kubernetes/fabric8/leader/election/AbstractLeaderElection.java new file mode 100644 index 0000000000..12e1fde241 --- /dev/null +++ b/spring-cloud-kubernetes-integration-tests/spring-cloud-kubernetes-fabric8-leader-election/src/test/java/org/springframework/cloud/kubernetes/fabric8/leader/election/AbstractLeaderElection.java @@ -0,0 +1,127 @@ +/* + * Copyright 2013-present the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.springframework.cloud.kubernetes.fabric8.leader.election; + +import java.util.concurrent.TimeUnit; +import java.util.concurrent.atomic.AtomicInteger; +import java.util.function.BooleanSupplier; + +import io.fabric8.kubernetes.client.Config; +import io.fabric8.kubernetes.client.KubernetesClient; +import io.fabric8.kubernetes.client.KubernetesClientBuilder; +import org.junit.jupiter.api.AfterEach; +import org.junit.jupiter.api.extension.ExtendWith; +import org.mockito.MockedStatic; +import org.mockito.Mockito; +import org.springframework.boot.autoconfigure.condition.ConditionalOnProperty; +import org.springframework.boot.test.context.SpringBootTest; +import org.springframework.boot.test.system.OutputCaptureExtension; +import org.testcontainers.k3s.K3sContainer; + +import org.springframework.beans.factory.annotation.Autowired; +import org.springframework.boot.test.context.TestConfiguration; +import org.springframework.cloud.kubernetes.commons.leader.LeaderUtils; +import org.springframework.cloud.kubernetes.integration.tests.commons.Commons; +import org.springframework.context.annotation.Bean; +import org.springframework.context.annotation.Primary; + +/** + * @author wind57 + */ +@ExtendWith(OutputCaptureExtension.class) +@SpringBootTest(webEnvironment = SpringBootTest.WebEnvironment.RANDOM_PORT, + properties = { "spring.main.cloud-platform=KUBERNETES", "spring.cloud.kubernetes.leader.election.enabled=true", + "logging.level.org.springframework.cloud.kubernetes.commons.leader.election=debug", + "logging.level.org.springframework.cloud.kubernetes.fabric8.leader.election=debug" }, + classes = { App.class, AbstractLeaderElection.LocalConfiguration.class }) +abstract class AbstractLeaderElection { + + private static K3sContainer container; + + private static final MockedStatic LEADER_UTILS_MOCKED_STATIC = Mockito.mockStatic(LeaderUtils.class); + + @Autowired + KubernetesClient kubernetesClient; + + static void beforeAll(String candidateIdentity) { + container = Commons.container(); + container.start(); + + LEADER_UTILS_MOCKED_STATIC.when(LeaderUtils::hostName).thenReturn(candidateIdentity); + } + + @AfterEach + void afterEach() { + kubernetesClient.leases() + .inNamespace("default") + .withName("spring-k8s-leader-election-lock") + .withTimeout(10, TimeUnit.SECONDS) + .delete(); + } + + @TestConfiguration + static class LocalConfiguration { + + @Bean + @Primary + KubernetesClient client() { + String kubeConfigYaml = container.getKubeConfigYaml(); + Config config = Config.fromKubeconfig(kubeConfigYaml); + return new KubernetesClientBuilder().withConfig(config).build(); + } + + // readiness passes after 2 retries + @Bean + @Primary + @ConditionalOnProperty(value = "readiness.passes", havingValue = "true", matchIfMissing = false) + BooleanSupplier readinessSupplierPasses() { + AtomicInteger counter = new AtomicInteger(0); + return () -> { + if (counter.get() != 2) { + counter.incrementAndGet(); + return false; + } + return true; + }; + } + + // readiness fails after 2 retries + @Bean + @Primary + @ConditionalOnProperty(value = "readiness.fails", havingValue = "true", matchIfMissing = false) + BooleanSupplier readinessSupplierFails() { + AtomicInteger counter = new AtomicInteger(0); + return () -> { + if (counter.get() != 2) { + counter.incrementAndGet(); + return false; + } + throw new RuntimeException("readiness fails"); + }; + } + + // readiness fails after 2 retries + @Bean + @Primary + @ConditionalOnProperty(value = "readiness.cycle.false", havingValue = "true", matchIfMissing = false) + BooleanSupplier readinessCycleFalse() { + return () -> false; + } + + } + +} diff --git a/spring-cloud-kubernetes-integration-tests/spring-cloud-kubernetes-fabric8-leader-election/src/test/java/org/springframework/cloud/kubernetes/fabric8/leader/election/App.java b/spring-cloud-kubernetes-integration-tests/spring-cloud-kubernetes-fabric8-leader-election/src/test/java/org/springframework/cloud/kubernetes/fabric8/leader/election/App.java new file mode 100644 index 0000000000..524cadb768 --- /dev/null +++ b/spring-cloud-kubernetes-integration-tests/spring-cloud-kubernetes-fabric8-leader-election/src/test/java/org/springframework/cloud/kubernetes/fabric8/leader/election/App.java @@ -0,0 +1,29 @@ +/* + * Copyright 2013-present the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.springframework.cloud.kubernetes.fabric8.leader.election; + +import org.springframework.boot.SpringApplication; +import org.springframework.boot.autoconfigure.SpringBootApplication; + +@SpringBootApplication +public class App { + + public static void main(String[] args) { + SpringApplication.run(App.class, args); + } + +} diff --git a/spring-cloud-kubernetes-integration-tests/spring-cloud-kubernetes-fabric8-leader-election/src/test/java/org/springframework/cloud/kubernetes/fabric8/leader/election/Fabric8LeaderElectionReadinessCanceledIT.java b/spring-cloud-kubernetes-integration-tests/spring-cloud-kubernetes-fabric8-leader-election/src/test/java/org/springframework/cloud/kubernetes/fabric8/leader/election/Fabric8LeaderElectionReadinessCanceledIT.java new file mode 100644 index 0000000000..b3a0d2059f --- /dev/null +++ b/spring-cloud-kubernetes-integration-tests/spring-cloud-kubernetes-fabric8-leader-election/src/test/java/org/springframework/cloud/kubernetes/fabric8/leader/election/Fabric8LeaderElectionReadinessCanceledIT.java @@ -0,0 +1,90 @@ +/* + * Copyright 2013-present the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.springframework.cloud.kubernetes.fabric8.leader.election; + +import org.awaitility.Awaitility; +import org.junit.jupiter.api.BeforeAll; +import org.junit.jupiter.api.Test; + +import org.springframework.beans.factory.annotation.Autowired; +import org.springframework.boot.test.system.CapturedOutput; +import org.springframework.test.context.TestPropertySource; + +import java.time.Duration; + +import static org.assertj.core.api.Assertions.assertThat; + +/** + * Readiness is canceled. This is the case when pod is shut down gracefully + * + * @author wind57 + */ +@TestPropertySource(properties = { + "readiness.cycle.false=true", + "spring.cloud.kubernetes.leader.election.wait-for-pod-ready=true" +}) +class Fabric8LeaderElectionReadinessCanceledIT extends AbstractLeaderElection { + + @Autowired + Fabric8LeaderElectionInitiator initiator; + + @BeforeAll + static void beforeAll() { + AbstractLeaderElection.beforeAll("canceled-readiness-it"); + } + + @Test + void test(CapturedOutput output) { + + // we are trying readiness at least once + Awaitility.await() + .atMost(Duration.ofSeconds(60)) + .pollInterval(Duration.ofSeconds(1)) + .until(() -> output.getOut().contains( + "Pod : canceled-readiness-it in namespace : default is not ready, will retry in one second" + )); + + initiator.preDestroy(); + + try { + Thread.sleep(2_000); + } catch (InterruptedException e) { + throw new RuntimeException(e); + } + + // 1. preDestroy logs what it will do + assertThat(output.getOut()).contains("podReadyFuture will be canceled for : canceled-readiness-it"); + + // 2. readiness failed + assertThat(output.getOut()).contains( + "readiness failed for : canceled-readiness-it, leader election will not start"); + + // 3. will cancel the future that is supposed to do the readiness + assertThat(output.getOut()).contains( + "canceling scheduled future because completable future was cancelled"); + + // 4. podReadyWaitingExecutor is shut down also + assertThat(output.getOut()).contains( + "podReadyWaitingExecutor will be shutdown for : canceled-readiness-it"); + + // 5. the scheduled executor where pod readiness is checked is shut down also + assertThat(output.getOut()).contains( + "Shutting down executor : podReadyExecutor"); + + } + +} diff --git a/spring-cloud-kubernetes-integration-tests/spring-cloud-kubernetes-fabric8-leader-election/src/test/java/org/springframework/cloud/kubernetes/fabric8/leader/election/Fabric8LeaderElectionReadinessFailsIT.java b/spring-cloud-kubernetes-integration-tests/spring-cloud-kubernetes-fabric8-leader-election/src/test/java/org/springframework/cloud/kubernetes/fabric8/leader/election/Fabric8LeaderElectionReadinessFailsIT.java new file mode 100644 index 0000000000..bf8a7a77b3 --- /dev/null +++ b/spring-cloud-kubernetes-integration-tests/spring-cloud-kubernetes-fabric8-leader-election/src/test/java/org/springframework/cloud/kubernetes/fabric8/leader/election/Fabric8LeaderElectionReadinessFailsIT.java @@ -0,0 +1,94 @@ +/* + * Copyright 2013-present the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.springframework.cloud.kubernetes.fabric8.leader.election; + +import java.time.Duration; + +import org.awaitility.Awaitility; +import org.junit.jupiter.api.BeforeAll; +import org.junit.jupiter.api.Test; + +import org.springframework.test.context.TestPropertySource; +import org.springframework.boot.test.system.CapturedOutput; + +import static org.assertj.core.api.Assertions.assertThat; + +/** + * Readiness fails with an Exception, and we don't establish leadership + * + * @author wind57 + */ +@TestPropertySource(properties = {"readiness.fails=true", + "spring.cloud.kubernetes.leader.election.wait-for-pod-ready=true" +}) +class Fabric8LeaderElectionReadinessFailsIT extends AbstractLeaderElection { + + @BeforeAll + static void beforeAll() { + AbstractLeaderElection.beforeAll("readiness-fails-simple-it"); + } + + /* 
+	 *     - readiness fails after 2 seconds
+	 *     - leader election process is not started at all
+	 *
+ */ + @Test + void test(CapturedOutput output) { + // we do not start leader election at all + Awaitility.await() + .atMost(Duration.ofSeconds(60)) + .pollInterval(Duration.ofSeconds(1)) + .until(() -> output.getOut().contains("leader election for : readiness-fails-simple-it will not start")); + + // let's unwind some logs to see that the process is how we expect it to be + + // 1. lease is used as the lock (comes from our code) + assertThat(output.getOut()).contains("will use lease as the lock for leader election"); + + // 2. leader initiator is started + assertThat(output.getOut()).contains("starting leader initiator : readiness-fails-simple-it"); + + // 3. wait for when pod is ready (we mock this one) + assertThat(output.getOut()).contains("will wait until pod readiness-fails-simple-it is ready"); + + // 4. we run readiness check in podReadyExecutor + assertThat(output.getOut()).contains("Scheduling command to run in : podReadyExecutor"); + + // 5. pod fails on the first two attempts + assertThat(output.getOut()) + .contains("Pod : readiness-fails-simple-it in namespace : default is not ready, will retry in one second"); + + // 6. readiness fails + assertThat(output.getOut()).contains("exception waiting for pod : readiness-fails-simple-it"); + + // 7. readiness failed + assertThat(output.getOut()).contains( + "pod readiness for : readiness-fails-simple-it failed with : readiness fails"); + + // 8. we shut down the executor + assertThat(output.getOut()).contains("canceling scheduled future because readiness failed"); + + // 9. leader election did not even start properly + assertThat(output.getOut()).contains( + "pod readiness for : readiness-fails-simple-it failed with : readiness fails"); + + // 10. executor is shutdown, even when readiness failed + assertThat(output.getOut()).contains("Shutting down executor : podReadyExecutor"); + } + +} diff --git a/spring-cloud-kubernetes-integration-tests/spring-cloud-kubernetes-fabric8-leader-election/src/test/java/org/springframework/cloud/kubernetes/fabric8/leader/election/Fabric8LeaderElectionReadinessPassesIT.java b/spring-cloud-kubernetes-integration-tests/spring-cloud-kubernetes-fabric8-leader-election/src/test/java/org/springframework/cloud/kubernetes/fabric8/leader/election/Fabric8LeaderElectionReadinessPassesIT.java new file mode 100644 index 0000000000..6c89d21255 --- /dev/null +++ b/spring-cloud-kubernetes-integration-tests/spring-cloud-kubernetes-fabric8-leader-election/src/test/java/org/springframework/cloud/kubernetes/fabric8/leader/election/Fabric8LeaderElectionReadinessPassesIT.java @@ -0,0 +1,92 @@ +/* + * Copyright 2013-present the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.springframework.cloud.kubernetes.fabric8.leader.election; + +import java.time.Duration; + +import org.awaitility.Awaitility; +import org.junit.jupiter.api.BeforeAll; +import org.junit.jupiter.api.Test; + +import org.springframework.test.context.TestPropertySource; +import org.springframework.boot.test.system.CapturedOutput; + +import static org.assertj.core.api.Assertions.assertThat; + +/** + * Readiness passes and we establish leadership + * + * @author wind57 + */ +@TestPropertySource(properties = { "readiness.passes=true", + "spring.cloud.kubernetes.leader.election.wait-for-pod-ready=true" }) +class Fabric8LeaderElectionReadinessPassesIT extends AbstractLeaderElection { + + @BeforeAll + static void beforeAll() { + AbstractLeaderElection.beforeAll("readiness-passes-simple-it"); + } + + /** + * 
+	 *     - readiness passes after 2 seconds
+	 *     - leader election process happens after that
+	 *
+ */ + @Test + void test(CapturedOutput output) { + // we have become the leader + Awaitility.await() + .atMost(Duration.ofSeconds(60)) + .pollInterval(Duration.ofSeconds(1)) + .until(() -> output.getOut().contains("readiness-passes-simple-it is the new leader")); + + // let's unwind some logs to see that the process is how we expect it to be + + // 1. lease is used as the lock (comes from our code) + assertThat(output.getOut()).contains("will use lease as the lock for leader election"); + + // 2. leader initiator is started + assertThat(output.getOut()).contains("starting leader initiator : readiness-passes-simple-it"); + + // 3. wait for when pod is ready (we mock this one) + assertThat(output.getOut()).contains("will wait until pod readiness-passes-simple-it is ready"); + + // 4. we run readiness check in podReadyExecutor + assertThat(output.getOut()).contains("Scheduling command to run in : podReadyExecutor"); + + // 5. pod fails on the first two attempts + assertThat(output.getOut()) + .contains("Pod : readiness-passes-simple-it in namespace : default is not ready, will retry in one second"); + + // 6. readiness passes and pod is ready + assertThat(output.getOut()).contains("Pod : readiness-passes-simple-it in namespace : default is ready"); + + // 7. we cancel the scheduled future because we do not need it anymore + assertThat(output.getOut()).contains("canceling scheduled future because readiness succeeded"); + + // 8. executor is shutdown + assertThat(output.getOut()).contains("Shutting down executor : podReadyExecutor"); + + // 9. pod is now ready + assertThat(output.getOut()).contains("readiness-passes-simple-it is ready"); + + // 10. we are the leader + assertThat(output.getOut()).contains("Leader changed from null to readiness-passes-simple-it"); + } + +} diff --git a/spring-cloud-kubernetes-integration-tests/spring-cloud-kubernetes-fabric8-leader-election/src/test/java/org/springframework/cloud/kubernetes/fabric8/leader/election/Fabric8LeaderElectionSimpleIT.java b/spring-cloud-kubernetes-integration-tests/spring-cloud-kubernetes-fabric8-leader-election/src/test/java/org/springframework/cloud/kubernetes/fabric8/leader/election/Fabric8LeaderElectionSimpleIT.java new file mode 100644 index 0000000000..467a384686 --- /dev/null +++ b/spring-cloud-kubernetes-integration-tests/spring-cloud-kubernetes-fabric8-leader-election/src/test/java/org/springframework/cloud/kubernetes/fabric8/leader/election/Fabric8LeaderElectionSimpleIT.java @@ -0,0 +1,105 @@ +/* + * Copyright 2013-present the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.springframework.cloud.kubernetes.fabric8.leader.election; + +import java.time.Duration; +import java.time.ZonedDateTime; + +import io.fabric8.kubernetes.api.model.coordination.v1.Lease; +import org.awaitility.Awaitility; +import org.junit.jupiter.api.BeforeAll; +import org.junit.jupiter.api.Test; + +import org.springframework.boot.test.system.CapturedOutput; +import org.springframework.test.context.TestPropertySource; + +import static org.assertj.core.api.Assertions.assertThat; + +/** + * A simple test where we are the sole participant in the leader election and everything + * goes fine from start to end. It's a happy path scenario test. + * + * @author wind57 + */ +@TestPropertySource(properties = "spring.cloud.kubernetes.leader.election.wait-for-pod-ready=false") +class Fabric8LeaderElectionSimpleIT extends AbstractLeaderElection { + + @BeforeAll + static void beforeAll() { + AbstractLeaderElection.beforeAll("simple-it"); + } + + @Test + void test(CapturedOutput output) { + + // we have become the leader + Awaitility.await() + .atMost(Duration.ofSeconds(60)) + .pollInterval(Duration.ofSeconds(1)) + .until(() -> output.getOut().contains("simple-it is the new leader")); + + // let's unwind some logs to see that the process is how we expect it to be + + // 1. lease is used as the lock (comes from our code) + assertThat(output.getOut()).contains("will use lease as the lock for leader election"); + + // 2. we start leader initiator for our hostname (comes from our code) + assertThat(output.getOut()).contains("starting leader initiator : simple-it"); + + // 3. we try to acquire the lease (comes from fabric8 code) + assertThat(output.getOut()).contains( + "Attempting to acquire leader lease 'LeaseLock: default - spring-k8s-leader-election-lock (simple-it)'"); + + // 4. lease has been acquired + assertThat(output.getOut()) + .contains("Acquired lease 'LeaseLock: default - spring-k8s-leader-election-lock (simple-it)'"); + + // 5. we are the leader (comes from fabric8 code) + assertThat(output.getOut()).contains("Leader changed from null to simple-it"); + + // 6. wait until a renewal happens (comes from fabric code) + // this one means that we have extended our leadership + Awaitility.await() + .atMost(Duration.ofSeconds(30)) + .pollInterval(Duration.ofSeconds(1)) + .until(() -> output.getOut() + .contains( + "Attempting to renew leader lease 'LeaseLock: default - spring-k8s-leader-election-lock (simple-it)'")); + + Lease lockLease = getLease(); + + ZonedDateTime currentAcquiredTime = lockLease.getSpec().getAcquireTime(); + assertThat(currentAcquiredTime).isNotNull(); + assertThat(lockLease.getSpec().getLeaseDurationSeconds()).isEqualTo(15); + assertThat(lockLease.getSpec().getLeaseTransitions()).isEqualTo(0); + + ZonedDateTime currentRenewalTime = lockLease.getSpec().getRenewTime(); + assertThat(currentRenewalTime).isNotNull(); + + // 7. renewal happens + Awaitility.await().pollInterval(Duration.ofSeconds(1)).atMost(Duration.ofSeconds(4)).until(() -> { + ZonedDateTime newRenewalTime = getLease().getSpec().getRenewTime(); + return newRenewalTime.isAfter(currentRenewalTime); + }); + + } + + private Lease getLease() { + return kubernetesClient.leases().inNamespace("default").withName("spring-k8s-leader-election-lock").get(); + } + +} diff --git a/spring-cloud-kubernetes-fabric8-leader/src/test/resources/logback-test.xml b/spring-cloud-kubernetes-integration-tests/spring-cloud-kubernetes-fabric8-leader-election/src/test/resources/logback-test.xml similarity index 100% rename from spring-cloud-kubernetes-fabric8-leader/src/test/resources/logback-test.xml rename to spring-cloud-kubernetes-integration-tests/spring-cloud-kubernetes-fabric8-leader-election/src/test/resources/logback-test.xml From dc56d269007a415c976fac160e2af28e73d01583 Mon Sep 17 00:00:00 2001 From: wind57 Date: Thu, 30 Oct 2025 23:31:14 +0200 Subject: [PATCH 10/16] wip Signed-off-by: wind57 --- .../Fabric8LeaderElectionInitiator.java | 4 +- .../election/AbstractLeaderElection.java | 14 +++---- ...ric8LeaderElectionReadinessCanceledIT.java | 39 ++++++++++--------- ...Fabric8LeaderElectionReadinessFailsIT.java | 31 ++++++++------- ...abric8LeaderElectionReadinessPassesIT.java | 13 ++++--- 5 files changed, 54 insertions(+), 47 deletions(-) diff --git a/spring-cloud-kubernetes-fabric8-leader/src/main/java/org/springframework/cloud/kubernetes/fabric8/leader/election/Fabric8LeaderElectionInitiator.java b/spring-cloud-kubernetes-fabric8-leader/src/main/java/org/springframework/cloud/kubernetes/fabric8/leader/election/Fabric8LeaderElectionInitiator.java index 81702b3ea7..2831363eba 100644 --- a/spring-cloud-kubernetes-fabric8-leader/src/main/java/org/springframework/cloud/kubernetes/fabric8/leader/election/Fabric8LeaderElectionInitiator.java +++ b/spring-cloud-kubernetes-fabric8-leader/src/main/java/org/springframework/cloud/kubernetes/fabric8/leader/election/Fabric8LeaderElectionInitiator.java @@ -109,8 +109,8 @@ void postConstruct() { // 'blockReadinessCheck' CompletableFuture ready = podReadyFuture.whenComplete((ok, error) -> { if (error != null) { - LOG.error(() -> "readiness failed for : " + candidateIdentity + - ", leader election will not start"); + LOG.error(() -> "readiness failed for : " + candidateIdentity + + ", leader election will not start"); } else { LOG.info(() -> candidateIdentity + " is ready"); diff --git a/spring-cloud-kubernetes-integration-tests/spring-cloud-kubernetes-fabric8-leader-election/src/test/java/org/springframework/cloud/kubernetes/fabric8/leader/election/AbstractLeaderElection.java b/spring-cloud-kubernetes-integration-tests/spring-cloud-kubernetes-fabric8-leader-election/src/test/java/org/springframework/cloud/kubernetes/fabric8/leader/election/AbstractLeaderElection.java index 12e1fde241..76fd821db8 100644 --- a/spring-cloud-kubernetes-integration-tests/spring-cloud-kubernetes-fabric8-leader-election/src/test/java/org/springframework/cloud/kubernetes/fabric8/leader/election/AbstractLeaderElection.java +++ b/spring-cloud-kubernetes-integration-tests/spring-cloud-kubernetes-fabric8-leader-election/src/test/java/org/springframework/cloud/kubernetes/fabric8/leader/election/AbstractLeaderElection.java @@ -27,13 +27,13 @@ import org.junit.jupiter.api.extension.ExtendWith; import org.mockito.MockedStatic; import org.mockito.Mockito; -import org.springframework.boot.autoconfigure.condition.ConditionalOnProperty; -import org.springframework.boot.test.context.SpringBootTest; -import org.springframework.boot.test.system.OutputCaptureExtension; import org.testcontainers.k3s.K3sContainer; import org.springframework.beans.factory.annotation.Autowired; +import org.springframework.boot.autoconfigure.condition.ConditionalOnProperty; +import org.springframework.boot.test.context.SpringBootTest; import org.springframework.boot.test.context.TestConfiguration; +import org.springframework.boot.test.system.OutputCaptureExtension; import org.springframework.cloud.kubernetes.commons.leader.LeaderUtils; import org.springframework.cloud.kubernetes.integration.tests.commons.Commons; import org.springframework.context.annotation.Bean; @@ -44,10 +44,10 @@ */ @ExtendWith(OutputCaptureExtension.class) @SpringBootTest(webEnvironment = SpringBootTest.WebEnvironment.RANDOM_PORT, - properties = { "spring.main.cloud-platform=KUBERNETES", "spring.cloud.kubernetes.leader.election.enabled=true", - "logging.level.org.springframework.cloud.kubernetes.commons.leader.election=debug", - "logging.level.org.springframework.cloud.kubernetes.fabric8.leader.election=debug" }, - classes = { App.class, AbstractLeaderElection.LocalConfiguration.class }) + properties = { "spring.main.cloud-platform=KUBERNETES", "spring.cloud.kubernetes.leader.election.enabled=true", + "logging.level.org.springframework.cloud.kubernetes.commons.leader.election=debug", + "logging.level.org.springframework.cloud.kubernetes.fabric8.leader.election=debug" }, + classes = { App.class, AbstractLeaderElection.LocalConfiguration.class }) abstract class AbstractLeaderElection { private static K3sContainer container; diff --git a/spring-cloud-kubernetes-integration-tests/spring-cloud-kubernetes-fabric8-leader-election/src/test/java/org/springframework/cloud/kubernetes/fabric8/leader/election/Fabric8LeaderElectionReadinessCanceledIT.java b/spring-cloud-kubernetes-integration-tests/spring-cloud-kubernetes-fabric8-leader-election/src/test/java/org/springframework/cloud/kubernetes/fabric8/leader/election/Fabric8LeaderElectionReadinessCanceledIT.java index b3a0d2059f..f9cbf0b231 100644 --- a/spring-cloud-kubernetes-integration-tests/spring-cloud-kubernetes-fabric8-leader-election/src/test/java/org/springframework/cloud/kubernetes/fabric8/leader/election/Fabric8LeaderElectionReadinessCanceledIT.java +++ b/spring-cloud-kubernetes-integration-tests/spring-cloud-kubernetes-fabric8-leader-election/src/test/java/org/springframework/cloud/kubernetes/fabric8/leader/election/Fabric8LeaderElectionReadinessCanceledIT.java @@ -16,6 +16,8 @@ package org.springframework.cloud.kubernetes.fabric8.leader.election; +import java.time.Duration; + import org.awaitility.Awaitility; import org.junit.jupiter.api.BeforeAll; import org.junit.jupiter.api.Test; @@ -24,8 +26,6 @@ import org.springframework.boot.test.system.CapturedOutput; import org.springframework.test.context.TestPropertySource; -import java.time.Duration; - import static org.assertj.core.api.Assertions.assertThat; /** @@ -33,10 +33,8 @@ * * @author wind57 */ -@TestPropertySource(properties = { - "readiness.cycle.false=true", - "spring.cloud.kubernetes.leader.election.wait-for-pod-ready=true" -}) +@TestPropertySource(properties = { "readiness.cycle.false=true", + "spring.cloud.kubernetes.leader.election.wait-for-pod-ready=true" }) class Fabric8LeaderElectionReadinessCanceledIT extends AbstractLeaderElection { @Autowired @@ -54,36 +52,39 @@ void test(CapturedOutput output) { Awaitility.await() .atMost(Duration.ofSeconds(60)) .pollInterval(Duration.ofSeconds(1)) - .until(() -> output.getOut().contains( - "Pod : canceled-readiness-it in namespace : default is not ready, will retry in one second" - )); + .until(() -> output.getOut() + .contains("Pod : canceled-readiness-it in namespace : default is not ready, will retry in one second")); initiator.preDestroy(); try { Thread.sleep(2_000); - } catch (InterruptedException e) { + } + catch (InterruptedException e) { throw new RuntimeException(e); } - // 1. preDestroy logs what it will do + // 1. preDestroy method logs what it will do assertThat(output.getOut()).contains("podReadyFuture will be canceled for : canceled-readiness-it"); // 2. readiness failed - assertThat(output.getOut()).contains( - "readiness failed for : canceled-readiness-it, leader election will not start"); + assertThat(output.getOut()) + .contains("readiness failed for : canceled-readiness-it, leader election will not start"); // 3. will cancel the future that is supposed to do the readiness - assertThat(output.getOut()).contains( - "canceling scheduled future because completable future was cancelled"); + assertThat(output.getOut()).contains("canceling scheduled future because completable future was cancelled"); // 4. podReadyWaitingExecutor is shut down also - assertThat(output.getOut()).contains( - "podReadyWaitingExecutor will be shutdown for : canceled-readiness-it"); + assertThat(output.getOut()).contains("podReadyWaitingExecutor will be shutdown for : canceled-readiness-it"); // 5. the scheduled executor where pod readiness is checked is shut down also - assertThat(output.getOut()).contains( - "Shutting down executor : podReadyExecutor"); + Awaitility.await() + .atMost(Duration.ofSeconds(2)) + .pollInterval(Duration.ofMillis(100)) + .until(() -> output.getOut().contains("Shutting down executor : podReadyExecutor")); + + // 6. leader election is not started, since readiness does not finish + assertThat(output.getOut()).doesNotContain("leaderFuture will be canceled for"); } diff --git a/spring-cloud-kubernetes-integration-tests/spring-cloud-kubernetes-fabric8-leader-election/src/test/java/org/springframework/cloud/kubernetes/fabric8/leader/election/Fabric8LeaderElectionReadinessFailsIT.java b/spring-cloud-kubernetes-integration-tests/spring-cloud-kubernetes-fabric8-leader-election/src/test/java/org/springframework/cloud/kubernetes/fabric8/leader/election/Fabric8LeaderElectionReadinessFailsIT.java index bf8a7a77b3..ff6b158129 100644 --- a/spring-cloud-kubernetes-integration-tests/spring-cloud-kubernetes-fabric8-leader-election/src/test/java/org/springframework/cloud/kubernetes/fabric8/leader/election/Fabric8LeaderElectionReadinessFailsIT.java +++ b/spring-cloud-kubernetes-integration-tests/spring-cloud-kubernetes-fabric8-leader-election/src/test/java/org/springframework/cloud/kubernetes/fabric8/leader/election/Fabric8LeaderElectionReadinessFailsIT.java @@ -22,8 +22,8 @@ import org.junit.jupiter.api.BeforeAll; import org.junit.jupiter.api.Test; -import org.springframework.test.context.TestPropertySource; import org.springframework.boot.test.system.CapturedOutput; +import org.springframework.test.context.TestPropertySource; import static org.assertj.core.api.Assertions.assertThat; @@ -32,9 +32,8 @@ * * @author wind57 */ -@TestPropertySource(properties = {"readiness.fails=true", - "spring.cloud.kubernetes.leader.election.wait-for-pod-ready=true" -}) +@TestPropertySource( + properties = { "readiness.fails=true", "spring.cloud.kubernetes.leader.election.wait-for-pod-ready=true" }) class Fabric8LeaderElectionReadinessFailsIT extends AbstractLeaderElection { @BeforeAll @@ -42,10 +41,9 @@ static void beforeAll() { AbstractLeaderElection.beforeAll("readiness-fails-simple-it"); } - /* 
-	 *     - readiness fails after 2 seconds
-	 *     - leader election process is not started at all
-	 *
+ /* + * 
 - readiness fails after 2 seconds - leader election process is not started at
+	 * all
*/ @Test void test(CapturedOutput output) { @@ -53,7 +51,8 @@ void test(CapturedOutput output) { Awaitility.await() .atMost(Duration.ofSeconds(60)) .pollInterval(Duration.ofSeconds(1)) - .until(() -> output.getOut().contains("leader election for : readiness-fails-simple-it will not start")); + .until(() -> output.getOut() + .contains("readiness failed for : readiness-fails-simple-it, leader election will not start")); // let's unwind some logs to see that the process is how we expect it to be @@ -77,18 +76,22 @@ void test(CapturedOutput output) { assertThat(output.getOut()).contains("exception waiting for pod : readiness-fails-simple-it"); // 7. readiness failed - assertThat(output.getOut()).contains( - "pod readiness for : readiness-fails-simple-it failed with : readiness fails"); + assertThat(output.getOut()) + .contains("pod readiness for : readiness-fails-simple-it failed with : readiness fails"); // 8. we shut down the executor assertThat(output.getOut()).contains("canceling scheduled future because readiness failed"); // 9. leader election did not even start properly - assertThat(output.getOut()).contains( - "pod readiness for : readiness-fails-simple-it failed with : readiness fails"); + assertThat(output.getOut()) + .contains("pod readiness for : readiness-fails-simple-it failed with : readiness fails"); // 10. executor is shutdown, even when readiness failed - assertThat(output.getOut()).contains("Shutting down executor : podReadyExecutor"); + Awaitility.await() + .atMost(Duration.ofSeconds(2)) + .pollInterval(Duration.ofMillis(100)) + .until(() -> output.getOut().contains("Shutting down executor : podReadyExecutor")); + } } diff --git a/spring-cloud-kubernetes-integration-tests/spring-cloud-kubernetes-fabric8-leader-election/src/test/java/org/springframework/cloud/kubernetes/fabric8/leader/election/Fabric8LeaderElectionReadinessPassesIT.java b/spring-cloud-kubernetes-integration-tests/spring-cloud-kubernetes-fabric8-leader-election/src/test/java/org/springframework/cloud/kubernetes/fabric8/leader/election/Fabric8LeaderElectionReadinessPassesIT.java index 6c89d21255..58a6fa34fe 100644 --- a/spring-cloud-kubernetes-integration-tests/spring-cloud-kubernetes-fabric8-leader-election/src/test/java/org/springframework/cloud/kubernetes/fabric8/leader/election/Fabric8LeaderElectionReadinessPassesIT.java +++ b/spring-cloud-kubernetes-integration-tests/spring-cloud-kubernetes-fabric8-leader-election/src/test/java/org/springframework/cloud/kubernetes/fabric8/leader/election/Fabric8LeaderElectionReadinessPassesIT.java @@ -22,8 +22,8 @@ import org.junit.jupiter.api.BeforeAll; import org.junit.jupiter.api.Test; -import org.springframework.test.context.TestPropertySource; import org.springframework.boot.test.system.CapturedOutput; +import org.springframework.test.context.TestPropertySource; import static org.assertj.core.api.Assertions.assertThat; @@ -32,9 +32,9 @@ * * @author wind57 */ -@TestPropertySource(properties = { "readiness.passes=true", - "spring.cloud.kubernetes.leader.election.wait-for-pod-ready=true" }) -class Fabric8LeaderElectionReadinessPassesIT extends AbstractLeaderElection { +@TestPropertySource( + properties = { "readiness.passes=true", "spring.cloud.kubernetes.leader.election.wait-for-pod-ready=true" }) +class Fabric8LeaderElectionReadinessPassesIT extends AbstractLeaderElection { @BeforeAll static void beforeAll() { @@ -80,7 +80,10 @@ void test(CapturedOutput output) { assertThat(output.getOut()).contains("canceling scheduled future because readiness succeeded"); // 8. executor is shutdown - assertThat(output.getOut()).contains("Shutting down executor : podReadyExecutor"); + Awaitility.await() + .atMost(Duration.ofSeconds(2)) + .pollInterval(Duration.ofMillis(100)) + .until(() -> output.getOut().contains("Shutting down executor : podReadyExecutor")); // 9. pod is now ready assertThat(output.getOut()).contains("readiness-passes-simple-it is ready"); From dc7e6f66053ef05366396b8b6b593b3468720e8c Mon Sep 17 00:00:00 2001 From: wind57 Date: Sat, 1 Nov 2025 22:07:12 +0200 Subject: [PATCH 11/16] fix Signed-off-by: wind57 --- .../cloud/kubernetes/client/ActuatorDisabledHealthTest.java | 2 +- .../cloud/kubernetes/client/ActuatorEnabledHealthTest.java | 2 +- .../LabeledConfigMapWithPrefixTests.java | 2 +- .../LabeledConfigMapWithProfileTests.java | 2 +- .../LabeledSecretWithPrefixTests.java | 2 +- .../NamedConfigMapWithPrefixTests.java | 2 +- .../NamedConfigMapWithProfileTests.java | 2 +- .../named_secret_with_prefix/NamedSecretWithPrefixTests.java | 2 +- .../named_secret_with_profile/NamedSecretWithProfileTests.java | 2 +- .../SingleSourceMultipleFilesTests.java | 2 +- .../sources_order/ConfigDataRetryableSourcesOrderTests.java | 2 +- .../config/applications/sources_order/SourcesOrderTests.java | 2 +- ...otstrapKubernetesClientSanitizeConfigpropsEndpointTests.java | 2 +- .../BootstrapKubernetesClientSanitizeEnvEndpointTests.java | 2 +- .../ConfigDataFabric8ConfigpropsEndpointTests.java | 2 +- .../ConfigDataKubernetesClientSanitizeEnvEndpointTests.java | 2 +- .../DiscoveryServerIntegrationAppsEndpointTests.java | 2 +- .../DiscoveryServerIntegrationAppsNameEndpointTests.java | 2 +- .../DiscoveryServerIntegrationInstanceEndpointTests.java | 2 +- .../cloud/kubernetes/discoveryserver/HeartbeatTests.java | 2 +- .../kubernetes/fabric8/Fabric8HealthIndicatorDisabledTest.java | 2 +- .../kubernetes/fabric8/Fabric8InsideHealthIndicatorTest.java | 2 +- .../kubernetes/fabric8/Fabric8InsideInfoContributorTest.java | 2 +- .../kubernetes/fabric8/Fabric8NotInsideHealthIndicatorTest.java | 2 +- .../kubernetes/fabric8/Fabric8NotInsideInfoContributorTest.java | 2 +- .../cloud/kubernetes/fabric8/config/HealthIndicatorTest.java | 2 +- .../kubernetes/fabric8/config/actuator/DisabledHealthTests.java | 2 +- .../kubernetes/fabric8/config/actuator/EnabledHealthTests.java | 2 +- .../BootstrapMultipleYamlDocumentsDevActiveTests.java | 2 +- .../BootstrapMultipleYamlDocumentsNoActiveProfileTests.java | 2 +- .../ConfigDataMultipleYamlDocumentsDevActiveTests.java | 2 +- .../ConfigDataMultipleYamlDocumentsNoActiveProfileTests.java | 2 +- .../fabric8/config/config_map_source/ConfigMapSource.java | 2 +- .../ConfigMapsWithProfilesNoActiveProfile.java | 2 +- .../ConfigMapsWithProfileExpression.java | 2 +- .../config/config_map_with_profiles/ConfigMapsWithProfiles.java | 2 +- .../fabric8/config/config_maps_mixed/ConfigMapsMixed.java | 2 +- .../ConfigMapsWithActiveProfilesName.java | 2 +- .../config_maps_without_profiles/ConfigMapsWithoutProfiles.java | 2 +- .../LabeledConfigMapWithPrefix.java | 2 +- .../labeled_secret_with_prefix/LabeledSecretWithPrefix.java | 2 +- .../multiple_configmaps_sources/MultipleConfigMapsSources.java | 2 +- .../fabric8/config/multiple_secrets/MultipleSecrets.java | 2 +- .../named_config_map_with_prefix/NamedConfigMapWithPrefix.java | 2 +- .../NamedConfigMapWithProfile.java | 2 +- .../config/named_secret_with_prefix/NamedSecretWithPrefix.java | 2 +- .../named_secret_with_profile/NamedSecretWithProfile.java | 2 +- .../config/retryable_sources_order/RetryableSourcesOrder.java | 2 +- .../BootstrapFabric8SanitizeConfigpropsEndpointTests.java | 2 +- .../BootstrapFabric8SanitizeEnvEndpointTests.java | 2 +- .../ConfigDataFabric8ConfigpropsEndpointTests.java | 2 +- .../ConfigDataFabric8SanitizeEnvEndpointTests.java | 2 +- .../fabric8/config/secrets_with_labels/SecretsWithLabels.java | 2 +- .../single_source_multiple_files/SingleSourceMultipleFiles.java | 2 +- .../kubernetes/fabric8/config/sources_order/SourcesOrder.java | 2 +- .../fabric8/leader/Fabric8LeaderAutoConfigurationTests.java | 2 +- 56 files changed, 56 insertions(+), 56 deletions(-) diff --git a/spring-cloud-kubernetes-client-autoconfig/src/test/java/org/springframework/cloud/kubernetes/client/ActuatorDisabledHealthTest.java b/spring-cloud-kubernetes-client-autoconfig/src/test/java/org/springframework/cloud/kubernetes/client/ActuatorDisabledHealthTest.java index 0e5ed47c03..cf003746d4 100644 --- a/spring-cloud-kubernetes-client-autoconfig/src/test/java/org/springframework/cloud/kubernetes/client/ActuatorDisabledHealthTest.java +++ b/spring-cloud-kubernetes-client-autoconfig/src/test/java/org/springframework/cloud/kubernetes/client/ActuatorDisabledHealthTest.java @@ -23,7 +23,7 @@ import org.springframework.boot.health.registry.ReactiveHealthContributorRegistry; import org.springframework.boot.test.context.SpringBootTest; import org.springframework.boot.test.web.server.LocalManagementPort; -import org.springframework.boot.webtestclient.AutoConfigureWebTestClient; +import org.springframework.boot.webtestclient.autoconfigure.AutoConfigureWebTestClient; import org.springframework.http.MediaType; import org.springframework.test.web.reactive.server.WebTestClient; diff --git a/spring-cloud-kubernetes-client-autoconfig/src/test/java/org/springframework/cloud/kubernetes/client/ActuatorEnabledHealthTest.java b/spring-cloud-kubernetes-client-autoconfig/src/test/java/org/springframework/cloud/kubernetes/client/ActuatorEnabledHealthTest.java index a138489cd3..355c2fb2d0 100644 --- a/spring-cloud-kubernetes-client-autoconfig/src/test/java/org/springframework/cloud/kubernetes/client/ActuatorEnabledHealthTest.java +++ b/spring-cloud-kubernetes-client-autoconfig/src/test/java/org/springframework/cloud/kubernetes/client/ActuatorEnabledHealthTest.java @@ -23,7 +23,7 @@ import org.springframework.boot.health.registry.HealthContributorRegistry; import org.springframework.boot.test.context.SpringBootTest; import org.springframework.boot.test.web.server.LocalManagementPort; -import org.springframework.boot.webtestclient.AutoConfigureWebTestClient; +import org.springframework.boot.webtestclient.autoconfigure.AutoConfigureWebTestClient; import org.springframework.http.MediaType; import org.springframework.test.web.reactive.server.WebTestClient; diff --git a/spring-cloud-kubernetes-client-config/src/test/java/org/springframework/cloud/kubernetes/client/config/applications/labeled_config_map_with_prefix/LabeledConfigMapWithPrefixTests.java b/spring-cloud-kubernetes-client-config/src/test/java/org/springframework/cloud/kubernetes/client/config/applications/labeled_config_map_with_prefix/LabeledConfigMapWithPrefixTests.java index 31efe63574..c1341049be 100644 --- a/spring-cloud-kubernetes-client-config/src/test/java/org/springframework/cloud/kubernetes/client/config/applications/labeled_config_map_with_prefix/LabeledConfigMapWithPrefixTests.java +++ b/spring-cloud-kubernetes-client-config/src/test/java/org/springframework/cloud/kubernetes/client/config/applications/labeled_config_map_with_prefix/LabeledConfigMapWithPrefixTests.java @@ -23,7 +23,7 @@ import org.junit.jupiter.api.Test; import org.springframework.beans.factory.annotation.Autowired; -import org.springframework.boot.webtestclient.AutoConfigureWebTestClient; +import org.springframework.boot.webtestclient.autoconfigure.AutoConfigureWebTestClient; import org.springframework.test.web.reactive.server.WebTestClient; /** diff --git a/spring-cloud-kubernetes-client-config/src/test/java/org/springframework/cloud/kubernetes/client/config/applications/labeled_config_map_with_profile/LabeledConfigMapWithProfileTests.java b/spring-cloud-kubernetes-client-config/src/test/java/org/springframework/cloud/kubernetes/client/config/applications/labeled_config_map_with_profile/LabeledConfigMapWithProfileTests.java index 8998ce570c..09ec47cda9 100644 --- a/spring-cloud-kubernetes-client-config/src/test/java/org/springframework/cloud/kubernetes/client/config/applications/labeled_config_map_with_profile/LabeledConfigMapWithProfileTests.java +++ b/spring-cloud-kubernetes-client-config/src/test/java/org/springframework/cloud/kubernetes/client/config/applications/labeled_config_map_with_profile/LabeledConfigMapWithProfileTests.java @@ -22,7 +22,7 @@ import org.junit.jupiter.api.Test; import org.springframework.beans.factory.annotation.Autowired; -import org.springframework.boot.webtestclient.AutoConfigureWebTestClient; +import org.springframework.boot.webtestclient.autoconfigure.AutoConfigureWebTestClient; import org.springframework.cloud.kubernetes.client.config.applications.labeled_config_map_with_profile.properties.Blue; import org.springframework.cloud.kubernetes.client.config.applications.labeled_config_map_with_profile.properties.Green; import org.springframework.cloud.kubernetes.client.config.applications.labeled_config_map_with_profile.properties.GreenK8s; diff --git a/spring-cloud-kubernetes-client-config/src/test/java/org/springframework/cloud/kubernetes/client/config/applications/labeled_secret_with_prefix/LabeledSecretWithPrefixTests.java b/spring-cloud-kubernetes-client-config/src/test/java/org/springframework/cloud/kubernetes/client/config/applications/labeled_secret_with_prefix/LabeledSecretWithPrefixTests.java index 1113dba710..e0785b694d 100644 --- a/spring-cloud-kubernetes-client-config/src/test/java/org/springframework/cloud/kubernetes/client/config/applications/labeled_secret_with_prefix/LabeledSecretWithPrefixTests.java +++ b/spring-cloud-kubernetes-client-config/src/test/java/org/springframework/cloud/kubernetes/client/config/applications/labeled_secret_with_prefix/LabeledSecretWithPrefixTests.java @@ -23,7 +23,7 @@ import org.junit.jupiter.api.Test; import org.springframework.beans.factory.annotation.Autowired; -import org.springframework.boot.webtestclient.AutoConfigureWebTestClient; +import org.springframework.boot.webtestclient.autoconfigure.AutoConfigureWebTestClient; import org.springframework.cloud.kubernetes.client.config.bootstrap.stubs.LabeledSecretWithPrefixConfigurationStub; import org.springframework.test.context.TestPropertySource; import org.springframework.test.web.reactive.server.WebTestClient; diff --git a/spring-cloud-kubernetes-client-config/src/test/java/org/springframework/cloud/kubernetes/client/config/applications/named_config_map_with_prefix/NamedConfigMapWithPrefixTests.java b/spring-cloud-kubernetes-client-config/src/test/java/org/springframework/cloud/kubernetes/client/config/applications/named_config_map_with_prefix/NamedConfigMapWithPrefixTests.java index ad98979a76..84cd4c6498 100644 --- a/spring-cloud-kubernetes-client-config/src/test/java/org/springframework/cloud/kubernetes/client/config/applications/named_config_map_with_prefix/NamedConfigMapWithPrefixTests.java +++ b/spring-cloud-kubernetes-client-config/src/test/java/org/springframework/cloud/kubernetes/client/config/applications/named_config_map_with_prefix/NamedConfigMapWithPrefixTests.java @@ -23,7 +23,7 @@ import org.junit.jupiter.api.Test; import org.springframework.beans.factory.annotation.Autowired; -import org.springframework.boot.webtestclient.AutoConfigureWebTestClient; +import org.springframework.boot.webtestclient.autoconfigure.AutoConfigureWebTestClient; import org.springframework.test.web.reactive.server.WebTestClient; /** diff --git a/spring-cloud-kubernetes-client-config/src/test/java/org/springframework/cloud/kubernetes/client/config/applications/named_config_map_with_profile/NamedConfigMapWithProfileTests.java b/spring-cloud-kubernetes-client-config/src/test/java/org/springframework/cloud/kubernetes/client/config/applications/named_config_map_with_profile/NamedConfigMapWithProfileTests.java index e1bb00df41..e83f03eed2 100644 --- a/spring-cloud-kubernetes-client-config/src/test/java/org/springframework/cloud/kubernetes/client/config/applications/named_config_map_with_profile/NamedConfigMapWithProfileTests.java +++ b/spring-cloud-kubernetes-client-config/src/test/java/org/springframework/cloud/kubernetes/client/config/applications/named_config_map_with_profile/NamedConfigMapWithProfileTests.java @@ -23,7 +23,7 @@ import org.junit.jupiter.api.Test; import org.springframework.beans.factory.annotation.Autowired; -import org.springframework.boot.webtestclient.AutoConfigureWebTestClient; +import org.springframework.boot.webtestclient.autoconfigure.AutoConfigureWebTestClient; import org.springframework.test.web.reactive.server.WebTestClient; /** diff --git a/spring-cloud-kubernetes-client-config/src/test/java/org/springframework/cloud/kubernetes/client/config/applications/named_secret_with_prefix/NamedSecretWithPrefixTests.java b/spring-cloud-kubernetes-client-config/src/test/java/org/springframework/cloud/kubernetes/client/config/applications/named_secret_with_prefix/NamedSecretWithPrefixTests.java index f46aa8a393..41ebc2dd7b 100644 --- a/spring-cloud-kubernetes-client-config/src/test/java/org/springframework/cloud/kubernetes/client/config/applications/named_secret_with_prefix/NamedSecretWithPrefixTests.java +++ b/spring-cloud-kubernetes-client-config/src/test/java/org/springframework/cloud/kubernetes/client/config/applications/named_secret_with_prefix/NamedSecretWithPrefixTests.java @@ -23,7 +23,7 @@ import org.junit.jupiter.api.Test; import org.springframework.beans.factory.annotation.Autowired; -import org.springframework.boot.webtestclient.AutoConfigureWebTestClient; +import org.springframework.boot.webtestclient.autoconfigure.AutoConfigureWebTestClient; import org.springframework.test.context.TestPropertySource; import org.springframework.test.web.reactive.server.WebTestClient; diff --git a/spring-cloud-kubernetes-client-config/src/test/java/org/springframework/cloud/kubernetes/client/config/applications/named_secret_with_profile/NamedSecretWithProfileTests.java b/spring-cloud-kubernetes-client-config/src/test/java/org/springframework/cloud/kubernetes/client/config/applications/named_secret_with_profile/NamedSecretWithProfileTests.java index c94122b46a..7f1a8d2982 100644 --- a/spring-cloud-kubernetes-client-config/src/test/java/org/springframework/cloud/kubernetes/client/config/applications/named_secret_with_profile/NamedSecretWithProfileTests.java +++ b/spring-cloud-kubernetes-client-config/src/test/java/org/springframework/cloud/kubernetes/client/config/applications/named_secret_with_profile/NamedSecretWithProfileTests.java @@ -23,7 +23,7 @@ import org.junit.jupiter.api.Test; import org.springframework.beans.factory.annotation.Autowired; -import org.springframework.boot.webtestclient.AutoConfigureWebTestClient; +import org.springframework.boot.webtestclient.autoconfigure.AutoConfigureWebTestClient; import org.springframework.test.context.TestPropertySource; import org.springframework.test.web.reactive.server.WebTestClient; diff --git a/spring-cloud-kubernetes-client-config/src/test/java/org/springframework/cloud/kubernetes/client/config/applications/single_source_multiple_files/SingleSourceMultipleFilesTests.java b/spring-cloud-kubernetes-client-config/src/test/java/org/springframework/cloud/kubernetes/client/config/applications/single_source_multiple_files/SingleSourceMultipleFilesTests.java index 1b49f8f243..b8956fb761 100644 --- a/spring-cloud-kubernetes-client-config/src/test/java/org/springframework/cloud/kubernetes/client/config/applications/single_source_multiple_files/SingleSourceMultipleFilesTests.java +++ b/spring-cloud-kubernetes-client-config/src/test/java/org/springframework/cloud/kubernetes/client/config/applications/single_source_multiple_files/SingleSourceMultipleFilesTests.java @@ -23,7 +23,7 @@ import org.junit.jupiter.api.Test; import org.springframework.beans.factory.annotation.Autowired; -import org.springframework.boot.webtestclient.AutoConfigureWebTestClient; +import org.springframework.boot.webtestclient.autoconfigure.AutoConfigureWebTestClient; import org.springframework.test.web.reactive.server.WebTestClient; /** diff --git a/spring-cloud-kubernetes-client-config/src/test/java/org/springframework/cloud/kubernetes/client/config/applications/sources_order/ConfigDataRetryableSourcesOrderTests.java b/spring-cloud-kubernetes-client-config/src/test/java/org/springframework/cloud/kubernetes/client/config/applications/sources_order/ConfigDataRetryableSourcesOrderTests.java index de10c38628..137796c469 100644 --- a/spring-cloud-kubernetes-client-config/src/test/java/org/springframework/cloud/kubernetes/client/config/applications/sources_order/ConfigDataRetryableSourcesOrderTests.java +++ b/spring-cloud-kubernetes-client-config/src/test/java/org/springframework/cloud/kubernetes/client/config/applications/sources_order/ConfigDataRetryableSourcesOrderTests.java @@ -26,7 +26,7 @@ import org.mockito.Mockito; import org.springframework.boot.test.context.SpringBootTest; -import org.springframework.boot.webtestclient.AutoConfigureWebTestClient; +import org.springframework.boot.webtestclient.autoconfigure.AutoConfigureWebTestClient; import org.springframework.cloud.kubernetes.client.KubernetesClientUtils; import org.springframework.test.context.junit.jupiter.SpringExtension; diff --git a/spring-cloud-kubernetes-client-config/src/test/java/org/springframework/cloud/kubernetes/client/config/applications/sources_order/SourcesOrderTests.java b/spring-cloud-kubernetes-client-config/src/test/java/org/springframework/cloud/kubernetes/client/config/applications/sources_order/SourcesOrderTests.java index 55f50312ae..1bfcc4784c 100644 --- a/spring-cloud-kubernetes-client-config/src/test/java/org/springframework/cloud/kubernetes/client/config/applications/sources_order/SourcesOrderTests.java +++ b/spring-cloud-kubernetes-client-config/src/test/java/org/springframework/cloud/kubernetes/client/config/applications/sources_order/SourcesOrderTests.java @@ -31,7 +31,7 @@ import org.springframework.beans.factory.annotation.Autowired; import org.springframework.boot.test.context.SpringBootTest; -import org.springframework.boot.webtestclient.AutoConfigureWebTestClient; +import org.springframework.boot.webtestclient.autoconfigure.AutoConfigureWebTestClient; import org.springframework.test.context.junit.jupiter.SpringExtension; import org.springframework.test.web.reactive.server.WebTestClient; diff --git a/spring-cloud-kubernetes-client-config/src/test/java/org/springframework/cloud/kubernetes/client/config/sanitize_secrets/BootstrapKubernetesClientSanitizeConfigpropsEndpointTests.java b/spring-cloud-kubernetes-client-config/src/test/java/org/springframework/cloud/kubernetes/client/config/sanitize_secrets/BootstrapKubernetesClientSanitizeConfigpropsEndpointTests.java index 18787f1637..e0a50959fa 100644 --- a/spring-cloud-kubernetes-client-config/src/test/java/org/springframework/cloud/kubernetes/client/config/sanitize_secrets/BootstrapKubernetesClientSanitizeConfigpropsEndpointTests.java +++ b/spring-cloud-kubernetes-client-config/src/test/java/org/springframework/cloud/kubernetes/client/config/sanitize_secrets/BootstrapKubernetesClientSanitizeConfigpropsEndpointTests.java @@ -23,7 +23,7 @@ import org.springframework.boot.actuate.endpoint.SanitizableData; import org.springframework.boot.test.context.SpringBootTest; import org.springframework.boot.test.web.server.LocalManagementPort; -import org.springframework.boot.webtestclient.AutoConfigureWebTestClient; +import org.springframework.boot.webtestclient.autoconfigure.AutoConfigureWebTestClient; import org.springframework.http.MediaType; import org.springframework.test.web.reactive.server.WebTestClient; diff --git a/spring-cloud-kubernetes-client-config/src/test/java/org/springframework/cloud/kubernetes/client/config/sanitize_secrets/BootstrapKubernetesClientSanitizeEnvEndpointTests.java b/spring-cloud-kubernetes-client-config/src/test/java/org/springframework/cloud/kubernetes/client/config/sanitize_secrets/BootstrapKubernetesClientSanitizeEnvEndpointTests.java index d266dfb2bf..1b1221e8fa 100644 --- a/spring-cloud-kubernetes-client-config/src/test/java/org/springframework/cloud/kubernetes/client/config/sanitize_secrets/BootstrapKubernetesClientSanitizeEnvEndpointTests.java +++ b/spring-cloud-kubernetes-client-config/src/test/java/org/springframework/cloud/kubernetes/client/config/sanitize_secrets/BootstrapKubernetesClientSanitizeEnvEndpointTests.java @@ -23,7 +23,7 @@ import org.springframework.boot.actuate.endpoint.SanitizableData; import org.springframework.boot.test.context.SpringBootTest; import org.springframework.boot.test.web.server.LocalManagementPort; -import org.springframework.boot.webtestclient.AutoConfigureWebTestClient; +import org.springframework.boot.webtestclient.autoconfigure.AutoConfigureWebTestClient; import org.springframework.http.MediaType; import org.springframework.test.web.reactive.server.WebTestClient; diff --git a/spring-cloud-kubernetes-client-config/src/test/java/org/springframework/cloud/kubernetes/client/config/sanitize_secrets/ConfigDataFabric8ConfigpropsEndpointTests.java b/spring-cloud-kubernetes-client-config/src/test/java/org/springframework/cloud/kubernetes/client/config/sanitize_secrets/ConfigDataFabric8ConfigpropsEndpointTests.java index b5dd4730e8..2c86c4076c 100644 --- a/spring-cloud-kubernetes-client-config/src/test/java/org/springframework/cloud/kubernetes/client/config/sanitize_secrets/ConfigDataFabric8ConfigpropsEndpointTests.java +++ b/spring-cloud-kubernetes-client-config/src/test/java/org/springframework/cloud/kubernetes/client/config/sanitize_secrets/ConfigDataFabric8ConfigpropsEndpointTests.java @@ -23,7 +23,7 @@ import org.springframework.boot.actuate.endpoint.SanitizableData; import org.springframework.boot.test.context.SpringBootTest; import org.springframework.boot.test.web.server.LocalManagementPort; -import org.springframework.boot.webtestclient.AutoConfigureWebTestClient; +import org.springframework.boot.webtestclient.autoconfigure.AutoConfigureWebTestClient; import org.springframework.http.MediaType; import org.springframework.test.web.reactive.server.WebTestClient; diff --git a/spring-cloud-kubernetes-client-config/src/test/java/org/springframework/cloud/kubernetes/client/config/sanitize_secrets/ConfigDataKubernetesClientSanitizeEnvEndpointTests.java b/spring-cloud-kubernetes-client-config/src/test/java/org/springframework/cloud/kubernetes/client/config/sanitize_secrets/ConfigDataKubernetesClientSanitizeEnvEndpointTests.java index 4ab79bce3a..8e13e65e95 100644 --- a/spring-cloud-kubernetes-client-config/src/test/java/org/springframework/cloud/kubernetes/client/config/sanitize_secrets/ConfigDataKubernetesClientSanitizeEnvEndpointTests.java +++ b/spring-cloud-kubernetes-client-config/src/test/java/org/springframework/cloud/kubernetes/client/config/sanitize_secrets/ConfigDataKubernetesClientSanitizeEnvEndpointTests.java @@ -23,7 +23,7 @@ import org.springframework.boot.actuate.endpoint.SanitizableData; import org.springframework.boot.test.context.SpringBootTest; import org.springframework.boot.test.web.server.LocalManagementPort; -import org.springframework.boot.webtestclient.AutoConfigureWebTestClient; +import org.springframework.boot.webtestclient.autoconfigure.AutoConfigureWebTestClient; import org.springframework.http.MediaType; import org.springframework.test.web.reactive.server.WebTestClient; diff --git a/spring-cloud-kubernetes-controllers/spring-cloud-kubernetes-discoveryserver/src/test/java/org/springframework/cloud/kubernetes/discoveryserver/DiscoveryServerIntegrationAppsEndpointTests.java b/spring-cloud-kubernetes-controllers/spring-cloud-kubernetes-discoveryserver/src/test/java/org/springframework/cloud/kubernetes/discoveryserver/DiscoveryServerIntegrationAppsEndpointTests.java index 1479e5b94c..652686bb9d 100644 --- a/spring-cloud-kubernetes-controllers/spring-cloud-kubernetes-discoveryserver/src/test/java/org/springframework/cloud/kubernetes/discoveryserver/DiscoveryServerIntegrationAppsEndpointTests.java +++ b/spring-cloud-kubernetes-controllers/spring-cloud-kubernetes-discoveryserver/src/test/java/org/springframework/cloud/kubernetes/discoveryserver/DiscoveryServerIntegrationAppsEndpointTests.java @@ -38,7 +38,7 @@ import org.springframework.beans.factory.annotation.Autowired; import org.springframework.boot.test.context.SpringBootTest; import org.springframework.boot.test.context.TestConfiguration; -import org.springframework.boot.webtestclient.AutoConfigureWebTestClient; +import org.springframework.boot.webtestclient.autoconfigure.AutoConfigureWebTestClient; import org.springframework.cloud.kubernetes.client.discovery.KubernetesClientInformerReactiveDiscoveryClient; import org.springframework.cloud.kubernetes.client.discovery.VisibleKubernetesClientInformerDiscoveryClient; import org.springframework.cloud.kubernetes.commons.KubernetesNamespaceProvider; diff --git a/spring-cloud-kubernetes-controllers/spring-cloud-kubernetes-discoveryserver/src/test/java/org/springframework/cloud/kubernetes/discoveryserver/DiscoveryServerIntegrationAppsNameEndpointTests.java b/spring-cloud-kubernetes-controllers/spring-cloud-kubernetes-discoveryserver/src/test/java/org/springframework/cloud/kubernetes/discoveryserver/DiscoveryServerIntegrationAppsNameEndpointTests.java index 3ddab5ad9a..78b5dedd0b 100644 --- a/spring-cloud-kubernetes-controllers/spring-cloud-kubernetes-discoveryserver/src/test/java/org/springframework/cloud/kubernetes/discoveryserver/DiscoveryServerIntegrationAppsNameEndpointTests.java +++ b/spring-cloud-kubernetes-controllers/spring-cloud-kubernetes-discoveryserver/src/test/java/org/springframework/cloud/kubernetes/discoveryserver/DiscoveryServerIntegrationAppsNameEndpointTests.java @@ -38,7 +38,7 @@ import org.springframework.beans.factory.annotation.Autowired; import org.springframework.boot.test.context.SpringBootTest; import org.springframework.boot.test.context.TestConfiguration; -import org.springframework.boot.webtestclient.AutoConfigureWebTestClient; +import org.springframework.boot.webtestclient.autoconfigure.AutoConfigureWebTestClient; import org.springframework.cloud.kubernetes.client.discovery.KubernetesClientInformerReactiveDiscoveryClient; import org.springframework.cloud.kubernetes.client.discovery.VisibleKubernetesClientInformerDiscoveryClient; import org.springframework.cloud.kubernetes.commons.KubernetesNamespaceProvider; diff --git a/spring-cloud-kubernetes-controllers/spring-cloud-kubernetes-discoveryserver/src/test/java/org/springframework/cloud/kubernetes/discoveryserver/DiscoveryServerIntegrationInstanceEndpointTests.java b/spring-cloud-kubernetes-controllers/spring-cloud-kubernetes-discoveryserver/src/test/java/org/springframework/cloud/kubernetes/discoveryserver/DiscoveryServerIntegrationInstanceEndpointTests.java index 2ff4eaa4a9..0b3787810c 100644 --- a/spring-cloud-kubernetes-controllers/spring-cloud-kubernetes-discoveryserver/src/test/java/org/springframework/cloud/kubernetes/discoveryserver/DiscoveryServerIntegrationInstanceEndpointTests.java +++ b/spring-cloud-kubernetes-controllers/spring-cloud-kubernetes-discoveryserver/src/test/java/org/springframework/cloud/kubernetes/discoveryserver/DiscoveryServerIntegrationInstanceEndpointTests.java @@ -38,7 +38,7 @@ import org.springframework.beans.factory.annotation.Autowired; import org.springframework.boot.test.context.SpringBootTest; import org.springframework.boot.test.context.TestConfiguration; -import org.springframework.boot.webtestclient.AutoConfigureWebTestClient; +import org.springframework.boot.webtestclient.autoconfigure.AutoConfigureWebTestClient; import org.springframework.cloud.kubernetes.client.discovery.KubernetesClientInformerReactiveDiscoveryClient; import org.springframework.cloud.kubernetes.client.discovery.VisibleKubernetesClientInformerDiscoveryClient; import org.springframework.cloud.kubernetes.commons.KubernetesNamespaceProvider; diff --git a/spring-cloud-kubernetes-controllers/spring-cloud-kubernetes-discoveryserver/src/test/java/org/springframework/cloud/kubernetes/discoveryserver/HeartbeatTests.java b/spring-cloud-kubernetes-controllers/spring-cloud-kubernetes-discoveryserver/src/test/java/org/springframework/cloud/kubernetes/discoveryserver/HeartbeatTests.java index 50496d3ce2..ecbe4848a5 100644 --- a/spring-cloud-kubernetes-controllers/spring-cloud-kubernetes-discoveryserver/src/test/java/org/springframework/cloud/kubernetes/discoveryserver/HeartbeatTests.java +++ b/spring-cloud-kubernetes-controllers/spring-cloud-kubernetes-discoveryserver/src/test/java/org/springframework/cloud/kubernetes/discoveryserver/HeartbeatTests.java @@ -24,7 +24,7 @@ import org.springframework.beans.factory.annotation.Autowired; import org.springframework.boot.test.context.SpringBootTest; import org.springframework.boot.test.context.TestConfiguration; -import org.springframework.boot.webtestclient.AutoConfigureWebTestClient; +import org.springframework.boot.webtestclient.autoconfigure.AutoConfigureWebTestClient; import org.springframework.cloud.client.discovery.event.HeartbeatEvent; import org.springframework.cloud.kubernetes.client.discovery.KubernetesClientInformerReactiveDiscoveryClient; import org.springframework.cloud.kubernetes.commons.discovery.EndpointNameAndNamespace; diff --git a/spring-cloud-kubernetes-fabric8-autoconfig/src/test/java/org/springframework/cloud/kubernetes/fabric8/Fabric8HealthIndicatorDisabledTest.java b/spring-cloud-kubernetes-fabric8-autoconfig/src/test/java/org/springframework/cloud/kubernetes/fabric8/Fabric8HealthIndicatorDisabledTest.java index 11f093b140..c11cbc06ae 100644 --- a/spring-cloud-kubernetes-fabric8-autoconfig/src/test/java/org/springframework/cloud/kubernetes/fabric8/Fabric8HealthIndicatorDisabledTest.java +++ b/spring-cloud-kubernetes-fabric8-autoconfig/src/test/java/org/springframework/cloud/kubernetes/fabric8/Fabric8HealthIndicatorDisabledTest.java @@ -25,7 +25,7 @@ import org.springframework.beans.factory.annotation.Autowired; import org.springframework.boot.test.context.SpringBootTest; import org.springframework.boot.test.web.server.LocalManagementPort; -import org.springframework.boot.webtestclient.AutoConfigureWebTestClient; +import org.springframework.boot.webtestclient.autoconfigure.AutoConfigureWebTestClient; import org.springframework.http.MediaType; import org.springframework.test.web.reactive.server.WebTestClient; diff --git a/spring-cloud-kubernetes-fabric8-autoconfig/src/test/java/org/springframework/cloud/kubernetes/fabric8/Fabric8InsideHealthIndicatorTest.java b/spring-cloud-kubernetes-fabric8-autoconfig/src/test/java/org/springframework/cloud/kubernetes/fabric8/Fabric8InsideHealthIndicatorTest.java index ced4fc1297..88b1b1caaf 100644 --- a/spring-cloud-kubernetes-fabric8-autoconfig/src/test/java/org/springframework/cloud/kubernetes/fabric8/Fabric8InsideHealthIndicatorTest.java +++ b/spring-cloud-kubernetes-fabric8-autoconfig/src/test/java/org/springframework/cloud/kubernetes/fabric8/Fabric8InsideHealthIndicatorTest.java @@ -28,7 +28,7 @@ import org.springframework.beans.factory.annotation.Autowired; import org.springframework.boot.test.context.SpringBootTest; import org.springframework.boot.test.web.server.LocalManagementPort; -import org.springframework.boot.webtestclient.AutoConfigureWebTestClient; +import org.springframework.boot.webtestclient.autoconfigure.AutoConfigureWebTestClient; import org.springframework.cloud.kubernetes.commons.PodUtils; import org.springframework.context.annotation.Bean; import org.springframework.context.annotation.Configuration; diff --git a/spring-cloud-kubernetes-fabric8-autoconfig/src/test/java/org/springframework/cloud/kubernetes/fabric8/Fabric8InsideInfoContributorTest.java b/spring-cloud-kubernetes-fabric8-autoconfig/src/test/java/org/springframework/cloud/kubernetes/fabric8/Fabric8InsideInfoContributorTest.java index 1d473552d6..a1a5915f54 100644 --- a/spring-cloud-kubernetes-fabric8-autoconfig/src/test/java/org/springframework/cloud/kubernetes/fabric8/Fabric8InsideInfoContributorTest.java +++ b/spring-cloud-kubernetes-fabric8-autoconfig/src/test/java/org/springframework/cloud/kubernetes/fabric8/Fabric8InsideInfoContributorTest.java @@ -26,7 +26,7 @@ import org.springframework.beans.factory.annotation.Autowired; import org.springframework.boot.test.context.SpringBootTest; import org.springframework.boot.test.web.server.LocalManagementPort; -import org.springframework.boot.webtestclient.AutoConfigureWebTestClient; +import org.springframework.boot.webtestclient.autoconfigure.AutoConfigureWebTestClient; import org.springframework.context.annotation.Bean; import org.springframework.context.annotation.Configuration; import org.springframework.context.annotation.Import; diff --git a/spring-cloud-kubernetes-fabric8-autoconfig/src/test/java/org/springframework/cloud/kubernetes/fabric8/Fabric8NotInsideHealthIndicatorTest.java b/spring-cloud-kubernetes-fabric8-autoconfig/src/test/java/org/springframework/cloud/kubernetes/fabric8/Fabric8NotInsideHealthIndicatorTest.java index 1f271822f6..7e911d9162 100644 --- a/spring-cloud-kubernetes-fabric8-autoconfig/src/test/java/org/springframework/cloud/kubernetes/fabric8/Fabric8NotInsideHealthIndicatorTest.java +++ b/spring-cloud-kubernetes-fabric8-autoconfig/src/test/java/org/springframework/cloud/kubernetes/fabric8/Fabric8NotInsideHealthIndicatorTest.java @@ -26,7 +26,7 @@ import org.springframework.beans.factory.annotation.Autowired; import org.springframework.boot.test.context.SpringBootTest; import org.springframework.boot.test.web.server.LocalManagementPort; -import org.springframework.boot.webtestclient.AutoConfigureWebTestClient; +import org.springframework.boot.webtestclient.autoconfigure.AutoConfigureWebTestClient; import org.springframework.http.MediaType; import org.springframework.test.web.reactive.server.WebTestClient; diff --git a/spring-cloud-kubernetes-fabric8-autoconfig/src/test/java/org/springframework/cloud/kubernetes/fabric8/Fabric8NotInsideInfoContributorTest.java b/spring-cloud-kubernetes-fabric8-autoconfig/src/test/java/org/springframework/cloud/kubernetes/fabric8/Fabric8NotInsideInfoContributorTest.java index c5ef8b34f9..d713619134 100644 --- a/spring-cloud-kubernetes-fabric8-autoconfig/src/test/java/org/springframework/cloud/kubernetes/fabric8/Fabric8NotInsideInfoContributorTest.java +++ b/spring-cloud-kubernetes-fabric8-autoconfig/src/test/java/org/springframework/cloud/kubernetes/fabric8/Fabric8NotInsideInfoContributorTest.java @@ -23,7 +23,7 @@ import org.springframework.beans.factory.annotation.Autowired; import org.springframework.boot.test.context.SpringBootTest; import org.springframework.boot.test.web.server.LocalManagementPort; -import org.springframework.boot.webtestclient.AutoConfigureWebTestClient; +import org.springframework.boot.webtestclient.autoconfigure.AutoConfigureWebTestClient; import org.springframework.http.MediaType; import org.springframework.test.web.reactive.server.WebTestClient; diff --git a/spring-cloud-kubernetes-fabric8-config/src/test/java/org/springframework/cloud/kubernetes/fabric8/config/HealthIndicatorTest.java b/spring-cloud-kubernetes-fabric8-config/src/test/java/org/springframework/cloud/kubernetes/fabric8/config/HealthIndicatorTest.java index f818c5b727..93c9af962d 100644 --- a/spring-cloud-kubernetes-fabric8-config/src/test/java/org/springframework/cloud/kubernetes/fabric8/config/HealthIndicatorTest.java +++ b/spring-cloud-kubernetes-fabric8-config/src/test/java/org/springframework/cloud/kubernetes/fabric8/config/HealthIndicatorTest.java @@ -25,7 +25,7 @@ import org.springframework.beans.factory.annotation.Autowired; import org.springframework.boot.test.context.SpringBootTest; import org.springframework.boot.test.web.server.LocalManagementPort; -import org.springframework.boot.webtestclient.AutoConfigureWebTestClient; +import org.springframework.boot.webtestclient.autoconfigure.AutoConfigureWebTestClient; import org.springframework.http.MediaType; import org.springframework.test.web.reactive.server.WebTestClient; diff --git a/spring-cloud-kubernetes-fabric8-config/src/test/java/org/springframework/cloud/kubernetes/fabric8/config/actuator/DisabledHealthTests.java b/spring-cloud-kubernetes-fabric8-config/src/test/java/org/springframework/cloud/kubernetes/fabric8/config/actuator/DisabledHealthTests.java index be27248d7b..55ff6d6187 100644 --- a/spring-cloud-kubernetes-fabric8-config/src/test/java/org/springframework/cloud/kubernetes/fabric8/config/actuator/DisabledHealthTests.java +++ b/spring-cloud-kubernetes-fabric8-config/src/test/java/org/springframework/cloud/kubernetes/fabric8/config/actuator/DisabledHealthTests.java @@ -23,7 +23,7 @@ import org.springframework.boot.health.registry.ReactiveHealthContributorRegistry; import org.springframework.boot.test.context.SpringBootTest; import org.springframework.boot.test.web.server.LocalManagementPort; -import org.springframework.boot.webtestclient.AutoConfigureWebTestClient; +import org.springframework.boot.webtestclient.autoconfigure.AutoConfigureWebTestClient; import org.springframework.cloud.kubernetes.fabric8.config.TestApplication; import org.springframework.http.MediaType; import org.springframework.test.web.reactive.server.WebTestClient; diff --git a/spring-cloud-kubernetes-fabric8-config/src/test/java/org/springframework/cloud/kubernetes/fabric8/config/actuator/EnabledHealthTests.java b/spring-cloud-kubernetes-fabric8-config/src/test/java/org/springframework/cloud/kubernetes/fabric8/config/actuator/EnabledHealthTests.java index 207b1346e8..efe13af2be 100644 --- a/spring-cloud-kubernetes-fabric8-config/src/test/java/org/springframework/cloud/kubernetes/fabric8/config/actuator/EnabledHealthTests.java +++ b/spring-cloud-kubernetes-fabric8-config/src/test/java/org/springframework/cloud/kubernetes/fabric8/config/actuator/EnabledHealthTests.java @@ -23,7 +23,7 @@ import org.springframework.boot.health.registry.HealthContributorRegistry; import org.springframework.boot.test.context.SpringBootTest; import org.springframework.boot.test.web.server.LocalManagementPort; -import org.springframework.boot.webtestclient.AutoConfigureWebTestClient; +import org.springframework.boot.webtestclient.autoconfigure.AutoConfigureWebTestClient; import org.springframework.cloud.kubernetes.fabric8.config.TestApplication; import org.springframework.http.MediaType; import org.springframework.test.web.reactive.server.WebTestClient; diff --git a/spring-cloud-kubernetes-fabric8-config/src/test/java/org/springframework/cloud/kubernetes/fabric8/config/array_with_profiles/BootstrapMultipleYamlDocumentsDevActiveTests.java b/spring-cloud-kubernetes-fabric8-config/src/test/java/org/springframework/cloud/kubernetes/fabric8/config/array_with_profiles/BootstrapMultipleYamlDocumentsDevActiveTests.java index a6633a6598..1473f166e9 100644 --- a/spring-cloud-kubernetes-fabric8-config/src/test/java/org/springframework/cloud/kubernetes/fabric8/config/array_with_profiles/BootstrapMultipleYamlDocumentsDevActiveTests.java +++ b/spring-cloud-kubernetes-fabric8-config/src/test/java/org/springframework/cloud/kubernetes/fabric8/config/array_with_profiles/BootstrapMultipleYamlDocumentsDevActiveTests.java @@ -25,7 +25,7 @@ import org.junit.jupiter.api.Test; import org.springframework.beans.factory.annotation.Autowired; -import org.springframework.boot.webtestclient.AutoConfigureWebTestClient; +import org.springframework.boot.webtestclient.autoconfigure.AutoConfigureWebTestClient; import org.springframework.test.context.ActiveProfiles; import org.springframework.test.context.TestPropertySource; import org.springframework.test.web.reactive.server.WebTestClient; diff --git a/spring-cloud-kubernetes-fabric8-config/src/test/java/org/springframework/cloud/kubernetes/fabric8/config/array_with_profiles/BootstrapMultipleYamlDocumentsNoActiveProfileTests.java b/spring-cloud-kubernetes-fabric8-config/src/test/java/org/springframework/cloud/kubernetes/fabric8/config/array_with_profiles/BootstrapMultipleYamlDocumentsNoActiveProfileTests.java index 7cd417388e..b7e5a2f91d 100644 --- a/spring-cloud-kubernetes-fabric8-config/src/test/java/org/springframework/cloud/kubernetes/fabric8/config/array_with_profiles/BootstrapMultipleYamlDocumentsNoActiveProfileTests.java +++ b/spring-cloud-kubernetes-fabric8-config/src/test/java/org/springframework/cloud/kubernetes/fabric8/config/array_with_profiles/BootstrapMultipleYamlDocumentsNoActiveProfileTests.java @@ -25,7 +25,7 @@ import org.junit.jupiter.api.Test; import org.springframework.beans.factory.annotation.Autowired; -import org.springframework.boot.webtestclient.AutoConfigureWebTestClient; +import org.springframework.boot.webtestclient.autoconfigure.AutoConfigureWebTestClient; import org.springframework.test.context.TestPropertySource; import org.springframework.test.web.reactive.server.WebTestClient; diff --git a/spring-cloud-kubernetes-fabric8-config/src/test/java/org/springframework/cloud/kubernetes/fabric8/config/array_with_profiles/ConfigDataMultipleYamlDocumentsDevActiveTests.java b/spring-cloud-kubernetes-fabric8-config/src/test/java/org/springframework/cloud/kubernetes/fabric8/config/array_with_profiles/ConfigDataMultipleYamlDocumentsDevActiveTests.java index 9f72bac13f..a89ce42cd1 100644 --- a/spring-cloud-kubernetes-fabric8-config/src/test/java/org/springframework/cloud/kubernetes/fabric8/config/array_with_profiles/ConfigDataMultipleYamlDocumentsDevActiveTests.java +++ b/spring-cloud-kubernetes-fabric8-config/src/test/java/org/springframework/cloud/kubernetes/fabric8/config/array_with_profiles/ConfigDataMultipleYamlDocumentsDevActiveTests.java @@ -25,7 +25,7 @@ import org.junit.jupiter.api.Test; import org.springframework.beans.factory.annotation.Autowired; -import org.springframework.boot.webtestclient.AutoConfigureWebTestClient; +import org.springframework.boot.webtestclient.autoconfigure.AutoConfigureWebTestClient; import org.springframework.test.context.ActiveProfiles; import org.springframework.test.context.TestPropertySource; import org.springframework.test.web.reactive.server.WebTestClient; diff --git a/spring-cloud-kubernetes-fabric8-config/src/test/java/org/springframework/cloud/kubernetes/fabric8/config/array_with_profiles/ConfigDataMultipleYamlDocumentsNoActiveProfileTests.java b/spring-cloud-kubernetes-fabric8-config/src/test/java/org/springframework/cloud/kubernetes/fabric8/config/array_with_profiles/ConfigDataMultipleYamlDocumentsNoActiveProfileTests.java index b51c652a1b..0b2bd885b5 100644 --- a/spring-cloud-kubernetes-fabric8-config/src/test/java/org/springframework/cloud/kubernetes/fabric8/config/array_with_profiles/ConfigDataMultipleYamlDocumentsNoActiveProfileTests.java +++ b/spring-cloud-kubernetes-fabric8-config/src/test/java/org/springframework/cloud/kubernetes/fabric8/config/array_with_profiles/ConfigDataMultipleYamlDocumentsNoActiveProfileTests.java @@ -25,7 +25,7 @@ import org.junit.jupiter.api.Test; import org.springframework.beans.factory.annotation.Autowired; -import org.springframework.boot.webtestclient.AutoConfigureWebTestClient; +import org.springframework.boot.webtestclient.autoconfigure.AutoConfigureWebTestClient; import org.springframework.test.context.TestPropertySource; import org.springframework.test.web.reactive.server.WebTestClient; diff --git a/spring-cloud-kubernetes-fabric8-config/src/test/java/org/springframework/cloud/kubernetes/fabric8/config/config_map_source/ConfigMapSource.java b/spring-cloud-kubernetes-fabric8-config/src/test/java/org/springframework/cloud/kubernetes/fabric8/config/config_map_source/ConfigMapSource.java index df5353fec4..813a39617e 100644 --- a/spring-cloud-kubernetes-fabric8-config/src/test/java/org/springframework/cloud/kubernetes/fabric8/config/config_map_source/ConfigMapSource.java +++ b/spring-cloud-kubernetes-fabric8-config/src/test/java/org/springframework/cloud/kubernetes/fabric8/config/config_map_source/ConfigMapSource.java @@ -26,7 +26,7 @@ import org.springframework.beans.factory.annotation.Autowired; import org.springframework.boot.test.context.SpringBootTest; -import org.springframework.boot.webtestclient.AutoConfigureWebTestClient; +import org.springframework.boot.webtestclient.autoconfigure.AutoConfigureWebTestClient; import org.springframework.cloud.kubernetes.fabric8.config.TestApplication; import org.springframework.test.web.reactive.server.WebTestClient; diff --git a/spring-cloud-kubernetes-fabric8-config/src/test/java/org/springframework/cloud/kubernetes/fabric8/config/config_map_with_active_profile_no_profile/ConfigMapsWithProfilesNoActiveProfile.java b/spring-cloud-kubernetes-fabric8-config/src/test/java/org/springframework/cloud/kubernetes/fabric8/config/config_map_with_active_profile_no_profile/ConfigMapsWithProfilesNoActiveProfile.java index 8ee5efad52..5d8955f59b 100644 --- a/spring-cloud-kubernetes-fabric8-config/src/test/java/org/springframework/cloud/kubernetes/fabric8/config/config_map_with_active_profile_no_profile/ConfigMapsWithProfilesNoActiveProfile.java +++ b/spring-cloud-kubernetes-fabric8-config/src/test/java/org/springframework/cloud/kubernetes/fabric8/config/config_map_with_active_profile_no_profile/ConfigMapsWithProfilesNoActiveProfile.java @@ -25,7 +25,7 @@ import org.springframework.beans.factory.annotation.Autowired; import org.springframework.boot.test.context.SpringBootTest; -import org.springframework.boot.webtestclient.AutoConfigureWebTestClient; +import org.springframework.boot.webtestclient.autoconfigure.AutoConfigureWebTestClient; import org.springframework.cloud.kubernetes.commons.config.Constants; import org.springframework.cloud.kubernetes.fabric8.config.TestApplication; import org.springframework.test.web.reactive.server.WebTestClient; diff --git a/spring-cloud-kubernetes-fabric8-config/src/test/java/org/springframework/cloud/kubernetes/fabric8/config/config_map_with_profile_expression/ConfigMapsWithProfileExpression.java b/spring-cloud-kubernetes-fabric8-config/src/test/java/org/springframework/cloud/kubernetes/fabric8/config/config_map_with_profile_expression/ConfigMapsWithProfileExpression.java index 0529104430..0e9f81a7bd 100644 --- a/spring-cloud-kubernetes-fabric8-config/src/test/java/org/springframework/cloud/kubernetes/fabric8/config/config_map_with_profile_expression/ConfigMapsWithProfileExpression.java +++ b/spring-cloud-kubernetes-fabric8-config/src/test/java/org/springframework/cloud/kubernetes/fabric8/config/config_map_with_profile_expression/ConfigMapsWithProfileExpression.java @@ -25,7 +25,7 @@ import org.springframework.beans.factory.annotation.Autowired; import org.springframework.boot.test.context.SpringBootTest; -import org.springframework.boot.webtestclient.AutoConfigureWebTestClient; +import org.springframework.boot.webtestclient.autoconfigure.AutoConfigureWebTestClient; import org.springframework.cloud.kubernetes.commons.config.Constants; import org.springframework.cloud.kubernetes.fabric8.config.ConfigMapTestUtil; import org.springframework.cloud.kubernetes.fabric8.config.TestApplication; diff --git a/spring-cloud-kubernetes-fabric8-config/src/test/java/org/springframework/cloud/kubernetes/fabric8/config/config_map_with_profiles/ConfigMapsWithProfiles.java b/spring-cloud-kubernetes-fabric8-config/src/test/java/org/springframework/cloud/kubernetes/fabric8/config/config_map_with_profiles/ConfigMapsWithProfiles.java index dc75b9faf1..4828dc85aa 100644 --- a/spring-cloud-kubernetes-fabric8-config/src/test/java/org/springframework/cloud/kubernetes/fabric8/config/config_map_with_profiles/ConfigMapsWithProfiles.java +++ b/spring-cloud-kubernetes-fabric8-config/src/test/java/org/springframework/cloud/kubernetes/fabric8/config/config_map_with_profiles/ConfigMapsWithProfiles.java @@ -25,7 +25,7 @@ import org.springframework.beans.factory.annotation.Autowired; import org.springframework.boot.test.context.SpringBootTest; -import org.springframework.boot.webtestclient.AutoConfigureWebTestClient; +import org.springframework.boot.webtestclient.autoconfigure.AutoConfigureWebTestClient; import org.springframework.cloud.kubernetes.commons.config.Constants; import org.springframework.cloud.kubernetes.fabric8.config.ConfigMapTestUtil; import org.springframework.cloud.kubernetes.fabric8.config.TestApplication; diff --git a/spring-cloud-kubernetes-fabric8-config/src/test/java/org/springframework/cloud/kubernetes/fabric8/config/config_maps_mixed/ConfigMapsMixed.java b/spring-cloud-kubernetes-fabric8-config/src/test/java/org/springframework/cloud/kubernetes/fabric8/config/config_maps_mixed/ConfigMapsMixed.java index ec2720a704..33d19b0a2f 100644 --- a/spring-cloud-kubernetes-fabric8-config/src/test/java/org/springframework/cloud/kubernetes/fabric8/config/config_maps_mixed/ConfigMapsMixed.java +++ b/spring-cloud-kubernetes-fabric8-config/src/test/java/org/springframework/cloud/kubernetes/fabric8/config/config_maps_mixed/ConfigMapsMixed.java @@ -26,7 +26,7 @@ import org.springframework.beans.factory.annotation.Autowired; import org.springframework.boot.test.context.SpringBootTest; -import org.springframework.boot.webtestclient.AutoConfigureWebTestClient; +import org.springframework.boot.webtestclient.autoconfigure.AutoConfigureWebTestClient; import org.springframework.cloud.kubernetes.fabric8.config.TestApplication; import org.springframework.test.web.reactive.server.WebTestClient; diff --git a/spring-cloud-kubernetes-fabric8-config/src/test/java/org/springframework/cloud/kubernetes/fabric8/config/config_maps_with_active_profiles/ConfigMapsWithActiveProfilesName.java b/spring-cloud-kubernetes-fabric8-config/src/test/java/org/springframework/cloud/kubernetes/fabric8/config/config_maps_with_active_profiles/ConfigMapsWithActiveProfilesName.java index a5efc7005b..db9b6e3cf9 100644 --- a/spring-cloud-kubernetes-fabric8-config/src/test/java/org/springframework/cloud/kubernetes/fabric8/config/config_maps_with_active_profiles/ConfigMapsWithActiveProfilesName.java +++ b/spring-cloud-kubernetes-fabric8-config/src/test/java/org/springframework/cloud/kubernetes/fabric8/config/config_maps_with_active_profiles/ConfigMapsWithActiveProfilesName.java @@ -25,7 +25,7 @@ import org.springframework.beans.factory.annotation.Autowired; import org.springframework.boot.test.context.SpringBootTest; -import org.springframework.boot.webtestclient.AutoConfigureWebTestClient; +import org.springframework.boot.webtestclient.autoconfigure.AutoConfigureWebTestClient; import org.springframework.cloud.kubernetes.commons.config.Constants; import org.springframework.cloud.kubernetes.fabric8.config.TestApplication; import org.springframework.test.context.ActiveProfiles; diff --git a/spring-cloud-kubernetes-fabric8-config/src/test/java/org/springframework/cloud/kubernetes/fabric8/config/config_maps_without_profiles/ConfigMapsWithoutProfiles.java b/spring-cloud-kubernetes-fabric8-config/src/test/java/org/springframework/cloud/kubernetes/fabric8/config/config_maps_without_profiles/ConfigMapsWithoutProfiles.java index 05e418e174..b0078286e6 100644 --- a/spring-cloud-kubernetes-fabric8-config/src/test/java/org/springframework/cloud/kubernetes/fabric8/config/config_maps_without_profiles/ConfigMapsWithoutProfiles.java +++ b/spring-cloud-kubernetes-fabric8-config/src/test/java/org/springframework/cloud/kubernetes/fabric8/config/config_maps_without_profiles/ConfigMapsWithoutProfiles.java @@ -25,7 +25,7 @@ import org.springframework.beans.factory.annotation.Autowired; import org.springframework.boot.test.context.SpringBootTest; -import org.springframework.boot.webtestclient.AutoConfigureWebTestClient; +import org.springframework.boot.webtestclient.autoconfigure.AutoConfigureWebTestClient; import org.springframework.cloud.kubernetes.commons.config.Constants; import org.springframework.cloud.kubernetes.fabric8.config.ConfigMapTestUtil; import org.springframework.cloud.kubernetes.fabric8.config.TestApplication; diff --git a/spring-cloud-kubernetes-fabric8-config/src/test/java/org/springframework/cloud/kubernetes/fabric8/config/labeled_config_map_with_prefix/LabeledConfigMapWithPrefix.java b/spring-cloud-kubernetes-fabric8-config/src/test/java/org/springframework/cloud/kubernetes/fabric8/config/labeled_config_map_with_prefix/LabeledConfigMapWithPrefix.java index 634a55a630..bb53bbf680 100644 --- a/spring-cloud-kubernetes-fabric8-config/src/test/java/org/springframework/cloud/kubernetes/fabric8/config/labeled_config_map_with_prefix/LabeledConfigMapWithPrefix.java +++ b/spring-cloud-kubernetes-fabric8-config/src/test/java/org/springframework/cloud/kubernetes/fabric8/config/labeled_config_map_with_prefix/LabeledConfigMapWithPrefix.java @@ -27,7 +27,7 @@ import org.springframework.beans.factory.annotation.Autowired; import org.springframework.boot.test.context.SpringBootTest; -import org.springframework.boot.webtestclient.AutoConfigureWebTestClient; +import org.springframework.boot.webtestclient.autoconfigure.AutoConfigureWebTestClient; import org.springframework.test.web.reactive.server.WebTestClient; /** diff --git a/spring-cloud-kubernetes-fabric8-config/src/test/java/org/springframework/cloud/kubernetes/fabric8/config/labeled_secret_with_prefix/LabeledSecretWithPrefix.java b/spring-cloud-kubernetes-fabric8-config/src/test/java/org/springframework/cloud/kubernetes/fabric8/config/labeled_secret_with_prefix/LabeledSecretWithPrefix.java index f0f1c98bfc..d4aeb4085e 100644 --- a/spring-cloud-kubernetes-fabric8-config/src/test/java/org/springframework/cloud/kubernetes/fabric8/config/labeled_secret_with_prefix/LabeledSecretWithPrefix.java +++ b/spring-cloud-kubernetes-fabric8-config/src/test/java/org/springframework/cloud/kubernetes/fabric8/config/labeled_secret_with_prefix/LabeledSecretWithPrefix.java @@ -29,7 +29,7 @@ import org.springframework.beans.factory.annotation.Autowired; import org.springframework.boot.test.context.SpringBootTest; -import org.springframework.boot.webtestclient.AutoConfigureWebTestClient; +import org.springframework.boot.webtestclient.autoconfigure.AutoConfigureWebTestClient; import org.springframework.test.web.reactive.server.WebTestClient; /** diff --git a/spring-cloud-kubernetes-fabric8-config/src/test/java/org/springframework/cloud/kubernetes/fabric8/config/multiple_configmaps_sources/MultipleConfigMapsSources.java b/spring-cloud-kubernetes-fabric8-config/src/test/java/org/springframework/cloud/kubernetes/fabric8/config/multiple_configmaps_sources/MultipleConfigMapsSources.java index a9b5db1274..6878aa389c 100644 --- a/spring-cloud-kubernetes-fabric8-config/src/test/java/org/springframework/cloud/kubernetes/fabric8/config/multiple_configmaps_sources/MultipleConfigMapsSources.java +++ b/spring-cloud-kubernetes-fabric8-config/src/test/java/org/springframework/cloud/kubernetes/fabric8/config/multiple_configmaps_sources/MultipleConfigMapsSources.java @@ -26,7 +26,7 @@ import org.springframework.beans.factory.annotation.Autowired; import org.springframework.boot.test.context.SpringBootTest; -import org.springframework.boot.webtestclient.AutoConfigureWebTestClient; +import org.springframework.boot.webtestclient.autoconfigure.AutoConfigureWebTestClient; import org.springframework.cloud.kubernetes.fabric8.config.example2.ExampleApp; import org.springframework.test.web.reactive.server.WebTestClient; diff --git a/spring-cloud-kubernetes-fabric8-config/src/test/java/org/springframework/cloud/kubernetes/fabric8/config/multiple_secrets/MultipleSecrets.java b/spring-cloud-kubernetes-fabric8-config/src/test/java/org/springframework/cloud/kubernetes/fabric8/config/multiple_secrets/MultipleSecrets.java index 6292fbaeaf..8d7ef0f767 100644 --- a/spring-cloud-kubernetes-fabric8-config/src/test/java/org/springframework/cloud/kubernetes/fabric8/config/multiple_secrets/MultipleSecrets.java +++ b/spring-cloud-kubernetes-fabric8-config/src/test/java/org/springframework/cloud/kubernetes/fabric8/config/multiple_secrets/MultipleSecrets.java @@ -28,7 +28,7 @@ import org.springframework.beans.factory.annotation.Autowired; import org.springframework.boot.test.context.SpringBootTest; -import org.springframework.boot.webtestclient.AutoConfigureWebTestClient; +import org.springframework.boot.webtestclient.autoconfigure.AutoConfigureWebTestClient; import org.springframework.cloud.kubernetes.fabric8.config.example3.MultiSecretsApp; import org.springframework.test.web.reactive.server.WebTestClient; diff --git a/spring-cloud-kubernetes-fabric8-config/src/test/java/org/springframework/cloud/kubernetes/fabric8/config/named_config_map_with_prefix/NamedConfigMapWithPrefix.java b/spring-cloud-kubernetes-fabric8-config/src/test/java/org/springframework/cloud/kubernetes/fabric8/config/named_config_map_with_prefix/NamedConfigMapWithPrefix.java index 31bcb44b5e..e62b756e7a 100644 --- a/spring-cloud-kubernetes-fabric8-config/src/test/java/org/springframework/cloud/kubernetes/fabric8/config/named_config_map_with_prefix/NamedConfigMapWithPrefix.java +++ b/spring-cloud-kubernetes-fabric8-config/src/test/java/org/springframework/cloud/kubernetes/fabric8/config/named_config_map_with_prefix/NamedConfigMapWithPrefix.java @@ -27,7 +27,7 @@ import org.springframework.beans.factory.annotation.Autowired; import org.springframework.boot.test.context.SpringBootTest; -import org.springframework.boot.webtestclient.AutoConfigureWebTestClient; +import org.springframework.boot.webtestclient.autoconfigure.AutoConfigureWebTestClient; import org.springframework.test.web.reactive.server.WebTestClient; /** diff --git a/spring-cloud-kubernetes-fabric8-config/src/test/java/org/springframework/cloud/kubernetes/fabric8/config/named_config_map_with_profile/NamedConfigMapWithProfile.java b/spring-cloud-kubernetes-fabric8-config/src/test/java/org/springframework/cloud/kubernetes/fabric8/config/named_config_map_with_profile/NamedConfigMapWithProfile.java index f7f0a833ca..7c236e65ca 100644 --- a/spring-cloud-kubernetes-fabric8-config/src/test/java/org/springframework/cloud/kubernetes/fabric8/config/named_config_map_with_profile/NamedConfigMapWithProfile.java +++ b/spring-cloud-kubernetes-fabric8-config/src/test/java/org/springframework/cloud/kubernetes/fabric8/config/named_config_map_with_profile/NamedConfigMapWithProfile.java @@ -27,7 +27,7 @@ import org.springframework.beans.factory.annotation.Autowired; import org.springframework.boot.test.context.SpringBootTest; -import org.springframework.boot.webtestclient.AutoConfigureWebTestClient; +import org.springframework.boot.webtestclient.autoconfigure.AutoConfigureWebTestClient; import org.springframework.test.context.ActiveProfiles; import org.springframework.test.web.reactive.server.WebTestClient; diff --git a/spring-cloud-kubernetes-fabric8-config/src/test/java/org/springframework/cloud/kubernetes/fabric8/config/named_secret_with_prefix/NamedSecretWithPrefix.java b/spring-cloud-kubernetes-fabric8-config/src/test/java/org/springframework/cloud/kubernetes/fabric8/config/named_secret_with_prefix/NamedSecretWithPrefix.java index cebc3d4050..31b24ff40c 100644 --- a/spring-cloud-kubernetes-fabric8-config/src/test/java/org/springframework/cloud/kubernetes/fabric8/config/named_secret_with_prefix/NamedSecretWithPrefix.java +++ b/spring-cloud-kubernetes-fabric8-config/src/test/java/org/springframework/cloud/kubernetes/fabric8/config/named_secret_with_prefix/NamedSecretWithPrefix.java @@ -29,7 +29,7 @@ import org.springframework.beans.factory.annotation.Autowired; import org.springframework.boot.test.context.SpringBootTest; -import org.springframework.boot.webtestclient.AutoConfigureWebTestClient; +import org.springframework.boot.webtestclient.autoconfigure.AutoConfigureWebTestClient; import org.springframework.test.web.reactive.server.WebTestClient; /** diff --git a/spring-cloud-kubernetes-fabric8-config/src/test/java/org/springframework/cloud/kubernetes/fabric8/config/named_secret_with_profile/NamedSecretWithProfile.java b/spring-cloud-kubernetes-fabric8-config/src/test/java/org/springframework/cloud/kubernetes/fabric8/config/named_secret_with_profile/NamedSecretWithProfile.java index 253904498f..b7bfa00d81 100644 --- a/spring-cloud-kubernetes-fabric8-config/src/test/java/org/springframework/cloud/kubernetes/fabric8/config/named_secret_with_profile/NamedSecretWithProfile.java +++ b/spring-cloud-kubernetes-fabric8-config/src/test/java/org/springframework/cloud/kubernetes/fabric8/config/named_secret_with_profile/NamedSecretWithProfile.java @@ -29,7 +29,7 @@ import org.springframework.beans.factory.annotation.Autowired; import org.springframework.boot.test.context.SpringBootTest; -import org.springframework.boot.webtestclient.AutoConfigureWebTestClient; +import org.springframework.boot.webtestclient.autoconfigure.AutoConfigureWebTestClient; import org.springframework.test.context.ActiveProfiles; import org.springframework.test.web.reactive.server.WebTestClient; diff --git a/spring-cloud-kubernetes-fabric8-config/src/test/java/org/springframework/cloud/kubernetes/fabric8/config/retryable_sources_order/RetryableSourcesOrder.java b/spring-cloud-kubernetes-fabric8-config/src/test/java/org/springframework/cloud/kubernetes/fabric8/config/retryable_sources_order/RetryableSourcesOrder.java index d89102c94e..1f7d28307d 100644 --- a/spring-cloud-kubernetes-fabric8-config/src/test/java/org/springframework/cloud/kubernetes/fabric8/config/retryable_sources_order/RetryableSourcesOrder.java +++ b/spring-cloud-kubernetes-fabric8-config/src/test/java/org/springframework/cloud/kubernetes/fabric8/config/retryable_sources_order/RetryableSourcesOrder.java @@ -30,7 +30,7 @@ import org.springframework.beans.factory.annotation.Autowired; import org.springframework.boot.test.context.SpringBootTest; -import org.springframework.boot.webtestclient.AutoConfigureWebTestClient; +import org.springframework.boot.webtestclient.autoconfigure.AutoConfigureWebTestClient; import org.springframework.test.web.reactive.server.WebTestClient; @SpringBootTest(webEnvironment = SpringBootTest.WebEnvironment.RANDOM_PORT, classes = RetryableSourcesOrderApp.class, diff --git a/spring-cloud-kubernetes-fabric8-config/src/test/java/org/springframework/cloud/kubernetes/fabric8/config/sanitize_secrets/BootstrapFabric8SanitizeConfigpropsEndpointTests.java b/spring-cloud-kubernetes-fabric8-config/src/test/java/org/springframework/cloud/kubernetes/fabric8/config/sanitize_secrets/BootstrapFabric8SanitizeConfigpropsEndpointTests.java index 4e5013cc6e..5deabdec9d 100644 --- a/spring-cloud-kubernetes-fabric8-config/src/test/java/org/springframework/cloud/kubernetes/fabric8/config/sanitize_secrets/BootstrapFabric8SanitizeConfigpropsEndpointTests.java +++ b/spring-cloud-kubernetes-fabric8-config/src/test/java/org/springframework/cloud/kubernetes/fabric8/config/sanitize_secrets/BootstrapFabric8SanitizeConfigpropsEndpointTests.java @@ -26,7 +26,7 @@ import org.springframework.boot.actuate.endpoint.SanitizableData; import org.springframework.boot.test.context.SpringBootTest; import org.springframework.boot.test.web.server.LocalManagementPort; -import org.springframework.boot.webtestclient.AutoConfigureWebTestClient; +import org.springframework.boot.webtestclient.autoconfigure.AutoConfigureWebTestClient; import org.springframework.http.MediaType; import org.springframework.test.web.reactive.server.WebTestClient; diff --git a/spring-cloud-kubernetes-fabric8-config/src/test/java/org/springframework/cloud/kubernetes/fabric8/config/sanitize_secrets/BootstrapFabric8SanitizeEnvEndpointTests.java b/spring-cloud-kubernetes-fabric8-config/src/test/java/org/springframework/cloud/kubernetes/fabric8/config/sanitize_secrets/BootstrapFabric8SanitizeEnvEndpointTests.java index ff208533a0..f1439eac48 100644 --- a/spring-cloud-kubernetes-fabric8-config/src/test/java/org/springframework/cloud/kubernetes/fabric8/config/sanitize_secrets/BootstrapFabric8SanitizeEnvEndpointTests.java +++ b/spring-cloud-kubernetes-fabric8-config/src/test/java/org/springframework/cloud/kubernetes/fabric8/config/sanitize_secrets/BootstrapFabric8SanitizeEnvEndpointTests.java @@ -26,7 +26,7 @@ import org.springframework.boot.actuate.endpoint.SanitizableData; import org.springframework.boot.test.context.SpringBootTest; import org.springframework.boot.test.web.server.LocalManagementPort; -import org.springframework.boot.webtestclient.AutoConfigureWebTestClient; +import org.springframework.boot.webtestclient.autoconfigure.AutoConfigureWebTestClient; import org.springframework.http.MediaType; import org.springframework.test.web.reactive.server.WebTestClient; diff --git a/spring-cloud-kubernetes-fabric8-config/src/test/java/org/springframework/cloud/kubernetes/fabric8/config/sanitize_secrets/ConfigDataFabric8ConfigpropsEndpointTests.java b/spring-cloud-kubernetes-fabric8-config/src/test/java/org/springframework/cloud/kubernetes/fabric8/config/sanitize_secrets/ConfigDataFabric8ConfigpropsEndpointTests.java index 7a7ef20f53..4b6a2dc414 100644 --- a/spring-cloud-kubernetes-fabric8-config/src/test/java/org/springframework/cloud/kubernetes/fabric8/config/sanitize_secrets/ConfigDataFabric8ConfigpropsEndpointTests.java +++ b/spring-cloud-kubernetes-fabric8-config/src/test/java/org/springframework/cloud/kubernetes/fabric8/config/sanitize_secrets/ConfigDataFabric8ConfigpropsEndpointTests.java @@ -26,7 +26,7 @@ import org.springframework.boot.actuate.endpoint.SanitizableData; import org.springframework.boot.test.context.SpringBootTest; import org.springframework.boot.test.web.server.LocalManagementPort; -import org.springframework.boot.webtestclient.AutoConfigureWebTestClient; +import org.springframework.boot.webtestclient.autoconfigure.AutoConfigureWebTestClient; import org.springframework.http.MediaType; import org.springframework.test.web.reactive.server.WebTestClient; diff --git a/spring-cloud-kubernetes-fabric8-config/src/test/java/org/springframework/cloud/kubernetes/fabric8/config/sanitize_secrets/ConfigDataFabric8SanitizeEnvEndpointTests.java b/spring-cloud-kubernetes-fabric8-config/src/test/java/org/springframework/cloud/kubernetes/fabric8/config/sanitize_secrets/ConfigDataFabric8SanitizeEnvEndpointTests.java index 92d9a5dabf..817dbe4c8a 100644 --- a/spring-cloud-kubernetes-fabric8-config/src/test/java/org/springframework/cloud/kubernetes/fabric8/config/sanitize_secrets/ConfigDataFabric8SanitizeEnvEndpointTests.java +++ b/spring-cloud-kubernetes-fabric8-config/src/test/java/org/springframework/cloud/kubernetes/fabric8/config/sanitize_secrets/ConfigDataFabric8SanitizeEnvEndpointTests.java @@ -26,7 +26,7 @@ import org.springframework.boot.actuate.endpoint.SanitizableData; import org.springframework.boot.test.context.SpringBootTest; import org.springframework.boot.test.web.server.LocalManagementPort; -import org.springframework.boot.webtestclient.AutoConfigureWebTestClient; +import org.springframework.boot.webtestclient.autoconfigure.AutoConfigureWebTestClient; import org.springframework.http.MediaType; import org.springframework.test.web.reactive.server.WebTestClient; diff --git a/spring-cloud-kubernetes-fabric8-config/src/test/java/org/springframework/cloud/kubernetes/fabric8/config/secrets_with_labels/SecretsWithLabels.java b/spring-cloud-kubernetes-fabric8-config/src/test/java/org/springframework/cloud/kubernetes/fabric8/config/secrets_with_labels/SecretsWithLabels.java index 3b509f3713..7fea55777f 100644 --- a/spring-cloud-kubernetes-fabric8-config/src/test/java/org/springframework/cloud/kubernetes/fabric8/config/secrets_with_labels/SecretsWithLabels.java +++ b/spring-cloud-kubernetes-fabric8-config/src/test/java/org/springframework/cloud/kubernetes/fabric8/config/secrets_with_labels/SecretsWithLabels.java @@ -29,7 +29,7 @@ import org.springframework.beans.factory.annotation.Autowired; import org.springframework.boot.test.context.SpringBootTest; -import org.springframework.boot.webtestclient.AutoConfigureWebTestClient; +import org.springframework.boot.webtestclient.autoconfigure.AutoConfigureWebTestClient; import org.springframework.test.web.reactive.server.WebTestClient; /** diff --git a/spring-cloud-kubernetes-fabric8-config/src/test/java/org/springframework/cloud/kubernetes/fabric8/config/single_source_multiple_files/SingleSourceMultipleFiles.java b/spring-cloud-kubernetes-fabric8-config/src/test/java/org/springframework/cloud/kubernetes/fabric8/config/single_source_multiple_files/SingleSourceMultipleFiles.java index a350a33d0a..b23f4f7316 100644 --- a/spring-cloud-kubernetes-fabric8-config/src/test/java/org/springframework/cloud/kubernetes/fabric8/config/single_source_multiple_files/SingleSourceMultipleFiles.java +++ b/spring-cloud-kubernetes-fabric8-config/src/test/java/org/springframework/cloud/kubernetes/fabric8/config/single_source_multiple_files/SingleSourceMultipleFiles.java @@ -27,7 +27,7 @@ import org.springframework.beans.factory.annotation.Autowired; import org.springframework.boot.test.context.SpringBootTest; -import org.springframework.boot.webtestclient.AutoConfigureWebTestClient; +import org.springframework.boot.webtestclient.autoconfigure.AutoConfigureWebTestClient; import org.springframework.test.context.ActiveProfiles; import org.springframework.test.web.reactive.server.WebTestClient; diff --git a/spring-cloud-kubernetes-fabric8-config/src/test/java/org/springframework/cloud/kubernetes/fabric8/config/sources_order/SourcesOrder.java b/spring-cloud-kubernetes-fabric8-config/src/test/java/org/springframework/cloud/kubernetes/fabric8/config/sources_order/SourcesOrder.java index 7bc9498840..43aceb58f7 100644 --- a/spring-cloud-kubernetes-fabric8-config/src/test/java/org/springframework/cloud/kubernetes/fabric8/config/sources_order/SourcesOrder.java +++ b/spring-cloud-kubernetes-fabric8-config/src/test/java/org/springframework/cloud/kubernetes/fabric8/config/sources_order/SourcesOrder.java @@ -30,7 +30,7 @@ import org.springframework.beans.factory.annotation.Autowired; import org.springframework.boot.test.context.SpringBootTest; -import org.springframework.boot.webtestclient.AutoConfigureWebTestClient; +import org.springframework.boot.webtestclient.autoconfigure.AutoConfigureWebTestClient; import org.springframework.test.web.reactive.server.WebTestClient; /** diff --git a/spring-cloud-kubernetes-fabric8-leader/src/test/java/org/springframework/cloud/kubernetes/fabric8/leader/Fabric8LeaderAutoConfigurationTests.java b/spring-cloud-kubernetes-fabric8-leader/src/test/java/org/springframework/cloud/kubernetes/fabric8/leader/Fabric8LeaderAutoConfigurationTests.java index 15a7140b16..2c3f5bc02f 100644 --- a/spring-cloud-kubernetes-fabric8-leader/src/test/java/org/springframework/cloud/kubernetes/fabric8/leader/Fabric8LeaderAutoConfigurationTests.java +++ b/spring-cloud-kubernetes-fabric8-leader/src/test/java/org/springframework/cloud/kubernetes/fabric8/leader/Fabric8LeaderAutoConfigurationTests.java @@ -23,7 +23,7 @@ import org.springframework.boot.autoconfigure.EnableAutoConfiguration; import org.springframework.boot.test.context.SpringBootTest; import org.springframework.boot.test.web.server.LocalManagementPort; -import org.springframework.boot.webtestclient.AutoConfigureWebTestClient; +import org.springframework.boot.webtestclient.autoconfigure.AutoConfigureWebTestClient; import org.springframework.http.MediaType; import org.springframework.test.web.reactive.server.WebTestClient; From ed5321f9d2183787e73521e730a207a716070431 Mon Sep 17 00:00:00 2001 From: wind57 Date: Sat, 1 Nov 2025 23:16:49 +0200 Subject: [PATCH 12/16] fix tests Signed-off-by: wind57 --- .../Fabric8LeaderElectionInfoContributorIsLeaderTest.java | 2 +- .../Fabric8LeaderElectionInfoContributorIsNotLeaderTest.java | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/spring-cloud-kubernetes-fabric8-leader/src/test/java/org/springframework/cloud/kubernetes/fabric8/leader/election/Fabric8LeaderElectionInfoContributorIsLeaderTest.java b/spring-cloud-kubernetes-fabric8-leader/src/test/java/org/springframework/cloud/kubernetes/fabric8/leader/election/Fabric8LeaderElectionInfoContributorIsLeaderTest.java index 007e4f7c87..375628a527 100644 --- a/spring-cloud-kubernetes-fabric8-leader/src/test/java/org/springframework/cloud/kubernetes/fabric8/leader/election/Fabric8LeaderElectionInfoContributorIsLeaderTest.java +++ b/spring-cloud-kubernetes-fabric8-leader/src/test/java/org/springframework/cloud/kubernetes/fabric8/leader/election/Fabric8LeaderElectionInfoContributorIsLeaderTest.java @@ -40,7 +40,7 @@ import org.springframework.boot.test.context.SpringBootTest; import org.springframework.boot.test.context.TestConfiguration; import org.springframework.boot.test.web.server.LocalManagementPort; -import org.springframework.boot.webtestclient.AutoConfigureWebTestClient; +import org.springframework.boot.webtestclient.autoconfigure.AutoConfigureWebTestClient; import org.springframework.cloud.kubernetes.commons.leader.LeaderUtils; import org.springframework.context.annotation.Bean; import org.springframework.context.annotation.Primary; diff --git a/spring-cloud-kubernetes-fabric8-leader/src/test/java/org/springframework/cloud/kubernetes/fabric8/leader/election/Fabric8LeaderElectionInfoContributorIsNotLeaderTest.java b/spring-cloud-kubernetes-fabric8-leader/src/test/java/org/springframework/cloud/kubernetes/fabric8/leader/election/Fabric8LeaderElectionInfoContributorIsNotLeaderTest.java index 53d0dec9d8..9b4252a398 100644 --- a/spring-cloud-kubernetes-fabric8-leader/src/test/java/org/springframework/cloud/kubernetes/fabric8/leader/election/Fabric8LeaderElectionInfoContributorIsNotLeaderTest.java +++ b/spring-cloud-kubernetes-fabric8-leader/src/test/java/org/springframework/cloud/kubernetes/fabric8/leader/election/Fabric8LeaderElectionInfoContributorIsNotLeaderTest.java @@ -40,7 +40,7 @@ import org.springframework.boot.test.context.SpringBootTest; import org.springframework.boot.test.context.TestConfiguration; import org.springframework.boot.test.web.server.LocalManagementPort; -import org.springframework.boot.webtestclient.AutoConfigureWebTestClient; +import org.springframework.boot.webtestclient.autoconfigure.AutoConfigureWebTestClient; import org.springframework.cloud.kubernetes.commons.leader.LeaderUtils; import org.springframework.context.annotation.Bean; import org.springframework.context.annotation.Primary; From fdccea5815362ea8a9670853076326561fc73e57 Mon Sep 17 00:00:00 2001 From: wind57 Date: Sat, 1 Nov 2025 23:45:20 +0200 Subject: [PATCH 13/16] minor refactor Signed-off-by: wind57 --- .../leader/election/PodReadyRunner.java | 36 +++++++++++-------- 1 file changed, 22 insertions(+), 14 deletions(-) diff --git a/spring-cloud-kubernetes-commons/src/main/java/org/springframework/cloud/kubernetes/commons/leader/election/PodReadyRunner.java b/spring-cloud-kubernetes-commons/src/main/java/org/springframework/cloud/kubernetes/commons/leader/election/PodReadyRunner.java index a9051362b9..63d4f4411a 100644 --- a/spring-cloud-kubernetes-commons/src/main/java/org/springframework/cloud/kubernetes/commons/leader/election/PodReadyRunner.java +++ b/spring-cloud-kubernetes-commons/src/main/java/org/springframework/cloud/kubernetes/commons/leader/election/PodReadyRunner.java @@ -42,16 +42,16 @@ public PodReadyRunner(String candidateIdentity, String candidateNamespace) { private static final LogAccessor LOG = new LogAccessor(PodReadyRunner.class); - private final CachedSingleThreadScheduler podReadyScheduler = new CachedSingleThreadScheduler("podReadyExecutor", - TTL_MILLIS); + private final CachedSingleThreadScheduler podReadySelfShutDownScheduler = + new CachedSingleThreadScheduler("podReadyExecutor", TTL_MILLIS); public CompletableFuture podReady(BooleanSupplier podReadySupplier) { - CompletableFuture podReadyFuture = new CompletableFuture<>(); + CompletableFuture podReadyCompletableFuture = new CompletableFuture<>(); - ScheduledFuture future = podReadyScheduler.scheduleWithFixedDelay(() -> { + ScheduledFuture scheduledFuture = podReadySelfShutDownScheduler.scheduleWithFixedDelay(() -> { - if (podReadyFuture.isDone()) { + if (podReadyCompletableFuture.isDone()) { LOG.info(() -> "pod readiness is known, not running another cycle"); return; } @@ -60,7 +60,7 @@ public CompletableFuture podReady(BooleanSupplier podReadySupplier) { if (podReadySupplier.getAsBoolean()) { LOG.info( () -> "Pod : " + candidateIdentity + " in namespace : " + candidateNamespace + " is ready"); - podReadyFuture.complete(null); + podReadyCompletableFuture.complete(null); } else { LOG.debug(() -> "Pod : " + candidateIdentity + " in namespace : " + candidateNamespace @@ -70,15 +70,26 @@ public CompletableFuture podReady(BooleanSupplier podReadySupplier) { catch (Exception e) { LOG.error(() -> "exception waiting for pod : " + candidateIdentity); LOG.error(() -> "pod readiness for : " + candidateIdentity + " failed with : " + e.getMessage()); - podReadyFuture.completeExceptionally(e); + podReadyCompletableFuture.completeExceptionally(e); } }, 1, 1, TimeUnit.SECONDS); - // cancel the future, thus shutting down the executor - podReadyFuture.whenComplete((ok, nok) -> { + attachShutDownHook(podReadyCompletableFuture, scheduledFuture); + + return podReadyCompletableFuture; + + } + + /** + * call scheduledFuture::cancel, thus the podReadySelfShutDownScheduler will shutdown + */ + private void attachShutDownHook(CompletableFuture podReadyCompletableFuture, + ScheduledFuture scheduledFuture) { + + podReadyCompletableFuture.whenComplete((ok, nok) -> { if (nok != null) { - if (podReadyFuture.isCancelled()) { + if (podReadyCompletableFuture.isCancelled()) { // something triggered us externally by calling // CompletableFuture::cancel, // need to shut down the readiness check @@ -94,11 +105,8 @@ public CompletableFuture podReady(BooleanSupplier podReadySupplier) { // no matter the outcome, we cancel the future and thus shut down the // executor that runs it. - future.cancel(true); + scheduledFuture.cancel(true); }); - - return podReadyFuture; - } } From 597ac5ce66cd4c2075c3e2317b91f12b7455e262 Mon Sep 17 00:00:00 2001 From: wind57 Date: Sat, 1 Nov 2025 23:50:26 +0200 Subject: [PATCH 14/16] simplify Signed-off-by: wind57 --- .../leader/Fabric8LeaderAutoConfigurationTests.java | 8 -------- 1 file changed, 8 deletions(-) diff --git a/spring-cloud-kubernetes-fabric8-leader/src/test/java/org/springframework/cloud/kubernetes/fabric8/leader/Fabric8LeaderAutoConfigurationTests.java b/spring-cloud-kubernetes-fabric8-leader/src/test/java/org/springframework/cloud/kubernetes/fabric8/leader/Fabric8LeaderAutoConfigurationTests.java index 2c3f5bc02f..35373df243 100644 --- a/spring-cloud-kubernetes-fabric8-leader/src/test/java/org/springframework/cloud/kubernetes/fabric8/leader/Fabric8LeaderAutoConfigurationTests.java +++ b/spring-cloud-kubernetes-fabric8-leader/src/test/java/org/springframework/cloud/kubernetes/fabric8/leader/Fabric8LeaderAutoConfigurationTests.java @@ -19,8 +19,6 @@ import org.junit.jupiter.api.Test; import org.springframework.beans.factory.annotation.Autowired; -import org.springframework.boot.SpringBootConfiguration; -import org.springframework.boot.autoconfigure.EnableAutoConfiguration; import org.springframework.boot.test.context.SpringBootTest; import org.springframework.boot.test.web.server.LocalManagementPort; import org.springframework.boot.webtestclient.autoconfigure.AutoConfigureWebTestClient; @@ -61,10 +59,4 @@ void infoEndpointShouldContainLeaderElection() { .exists(); } - @SpringBootConfiguration - @EnableAutoConfiguration - protected static class TestConfig { - - } - } From 3d4066d1d4b91407e2a609551a58a0847bfd4e65 Mon Sep 17 00:00:00 2001 From: wind57 Date: Sun, 2 Nov 2025 09:35:07 +0200 Subject: [PATCH 15/16] checkstyle Signed-off-by: wind57 --- .../kubernetes/commons/leader/election/PodReadyRunner.java | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/spring-cloud-kubernetes-commons/src/main/java/org/springframework/cloud/kubernetes/commons/leader/election/PodReadyRunner.java b/spring-cloud-kubernetes-commons/src/main/java/org/springframework/cloud/kubernetes/commons/leader/election/PodReadyRunner.java index 63d4f4411a..32b15eab01 100644 --- a/spring-cloud-kubernetes-commons/src/main/java/org/springframework/cloud/kubernetes/commons/leader/election/PodReadyRunner.java +++ b/spring-cloud-kubernetes-commons/src/main/java/org/springframework/cloud/kubernetes/commons/leader/election/PodReadyRunner.java @@ -42,8 +42,8 @@ public PodReadyRunner(String candidateIdentity, String candidateNamespace) { private static final LogAccessor LOG = new LogAccessor(PodReadyRunner.class); - private final CachedSingleThreadScheduler podReadySelfShutDownScheduler = - new CachedSingleThreadScheduler("podReadyExecutor", TTL_MILLIS); + private final CachedSingleThreadScheduler podReadySelfShutDownScheduler = new CachedSingleThreadScheduler( + "podReadyExecutor", TTL_MILLIS); public CompletableFuture podReady(BooleanSupplier podReadySupplier) { @@ -82,7 +82,7 @@ public CompletableFuture podReady(BooleanSupplier podReadySupplier) { } /** - * call scheduledFuture::cancel, thus the podReadySelfShutDownScheduler will shutdown + * call scheduledFuture::cancel, thus the podReadySelfShutDownScheduler will shutdown. */ private void attachShutDownHook(CompletableFuture podReadyCompletableFuture, ScheduledFuture scheduledFuture) { From 4822f60b45111b90bdb4c7872c083d0d2b08b3a1 Mon Sep 17 00:00:00 2001 From: wind57 Date: Sun, 2 Nov 2025 15:20:56 +0200 Subject: [PATCH 16/16] fix tests Signed-off-by: wind57 --- .../leader/election/CachedSingleThreadScheduler.java | 2 +- .../commons/leader/election/PodReadyRunnerTests.java | 10 ++++++---- 2 files changed, 7 insertions(+), 5 deletions(-) diff --git a/spring-cloud-kubernetes-commons/src/main/java/org/springframework/cloud/kubernetes/commons/leader/election/CachedSingleThreadScheduler.java b/spring-cloud-kubernetes-commons/src/main/java/org/springframework/cloud/kubernetes/commons/leader/election/CachedSingleThreadScheduler.java index 32b141c6ef..80e4e4bb6f 100644 --- a/spring-cloud-kubernetes-commons/src/main/java/org/springframework/cloud/kubernetes/commons/leader/election/CachedSingleThreadScheduler.java +++ b/spring-cloud-kubernetes-commons/src/main/java/org/springframework/cloud/kubernetes/commons/leader/election/CachedSingleThreadScheduler.java @@ -109,7 +109,7 @@ private ThreadFactory threadFactory() { @Override public Thread newThread(@Nonnull Runnable runnable) { Thread thread = threadFactory.newThread(runnable); - thread.setName("fabric8-leader-election" + "-" + thread.getName()); + thread.setName("cached-single-thread-scheduler" + "-" + thread.getName()); thread.setDaemon(true); return thread; } diff --git a/spring-cloud-kubernetes-commons/src/test/java/org/springframework/cloud/kubernetes/commons/leader/election/PodReadyRunnerTests.java b/spring-cloud-kubernetes-commons/src/test/java/org/springframework/cloud/kubernetes/commons/leader/election/PodReadyRunnerTests.java index 61c8493a6f..d1e836d2ec 100644 --- a/spring-cloud-kubernetes-commons/src/test/java/org/springframework/cloud/kubernetes/commons/leader/election/PodReadyRunnerTests.java +++ b/spring-cloud-kubernetes-commons/src/test/java/org/springframework/cloud/kubernetes/commons/leader/election/PodReadyRunnerTests.java @@ -167,11 +167,11 @@ void readinessFailsOnTheSecondCycleAttachNewPipeline(CapturedOutput output) { assertThat(output.getOut()).contains("exception waiting for pod : identity"); assertThat(output.getOut()).contains("pod readiness for : identity failed with : fail on the second cycle"); assertThat(output.getOut()).contains("readiness failed and we caught that"); - assertThat(output.getOut()).contains("canceling scheduled future because readiness failed"); await().atMost(Duration.ofSeconds(3)) .pollInterval(Duration.ofMillis(200)) .until(() -> output.getOut().contains("Shutting down executor : podReadyExecutor")); + assertThat(output.getOut()).contains("canceling scheduled future because readiness failed"); } assertThat(caught).isTrue(); } @@ -222,12 +222,14 @@ void readinessCanceledOnTheSecondCycleAttachNewPipeline(CapturedOutput output) t assertThat(output.getOut()).doesNotContain("leader election for : identity was not successful"); assertThat(output.getOut()).contains("readiness failed and we caught that"); - assertThat(output.getOut()).contains("canceling scheduled future because completable future was cancelled"); - assertThat(output.getOut()).doesNotContain("canceling scheduled future because readiness failed"); - await().atMost(Duration.ofSeconds(3)) .pollInterval(Duration.ofMillis(200)) .until(() -> output.getOut().contains("Shutting down executor : podReadyExecutor")); + + assertThat(output.getOut()).contains("canceling scheduled future because completable future was cancelled"); + assertThat(output.getOut()).doesNotContain("canceling scheduled future because readiness failed"); + + } assertThat(caught).isTrue(); cancelScheduler.shutdownNow();