diff --git a/docs/confluent-cloud.md b/docs/confluent-cloud.md index dc6a397e..594d46d1 100644 --- a/docs/confluent-cloud.md +++ b/docs/confluent-cloud.md @@ -6,7 +6,7 @@ This tool was designed to work with Confluent Cloud. It can manage service accou Ensure you have installed `kafka-gitops` or are using the `kafka-gitops` docker image as described in the [installation][installation] instructions. -You must have the `ccloud` command line tools installed if you wish to auto-populate the `principal` fields on services. +You must have the `confluent` command line tool installed if you wish to auto-populate the `principal` fields on services. ## Desired State File @@ -14,7 +14,7 @@ Create a basic desired state file, `state.yaml`, such as: ```yaml settings: - ccloud: + confluent: enabled: true topics: @@ -46,18 +46,18 @@ To use `kafka-gitops` with Confluent Cloud, you'll need to set a few environment * `KAFKA_SASL_MECHANISM`: `PLAIN` * `KAFKA_SSL_ENDPOINT_IDENTIFICATION_ALGORITHM`: `HTTPS` -Additionally, you'll need to login to the `ccloud` tool. You can automate this by setting the following environment variables: +Additionally, you'll need to login to the `confluent` tool. You can automate this by setting the following environment variables: -* `XX_CCLOUD_EMAIL`: Your Confluent Cloud administrator email -* `XX_CCLOUD_PASSWORD`: Your Confluent Cloud administrator password +* `CONFLUENT_CLOUD_EMAIL`: Your Confluent Cloud administrator email +* `CONFLUENT_CLOUD_PASSWORD`: Your Confluent Cloud administrator password -Then, you can run `ccloud login` and it will run without a prompt. This is great for CI builds. +Then, you can run `confluent login` and it will run without a prompt. This is great for CI builds. -You can optionally specify a path to a `ccloud` executable: +You can optionally specify a path to a `confluent` executable: -* `CCLOUD_EXECUTABLE_PATH`: `/full/path/to/ccloud` +* `CONFLUENT_EXECUTABLE_PATH`: `/full/path/to/confluent` -Otherwise, `ccloud` must be on your path. +Otherwise, `confluent` must be on your path. ## Validate diff --git a/docs/quick-start.md b/docs/quick-start.md index aecee291..3d7058bb 100644 --- a/docs/quick-start.md +++ b/docs/quick-start.md @@ -5,7 +5,7 @@ Getting started with `kafka-gitops` is simple. For this tutorial, we will assume - You have installed the `kafka-gitops` command as [described here](/installation.md). - You have a kafka cluster running on `localhost:9092`. -!> **NOTE**: If you desire to use this with Confluent Cloud, read our [Confluent Cloud page][ccloud]. +!> **NOTE**: If you desire to use this with Confluent Cloud, read our [Confluent Cloud page][confluent]. ## Desired State File @@ -147,7 +147,7 @@ org.apache.kafka.common.errors.PolicyViolationException: Topic replication facto Congrats! You've successfully started using GitOps strategies to manage your cluster. If you have security on your cluster, read the [services][services] page to start defining services. -[ccloud]: /confluent-cloud.md +[confluent]: /confluent-cloud.md [permissions]: /permissions.md [specification]: /specification.md [services]: /services.md \ No newline at end of file diff --git a/docs/specification.md b/docs/specification.md index 3e25f247..7145b55f 100644 --- a/docs/specification.md +++ b/docs/specification.md @@ -19,7 +19,7 @@ The desired state file consists of: **Options**: -- **ccloud** [Optional]: An object which contains an `enabled` field. Set this to true if using a Confluent Cloud cluster. +- **confluent** [Optional]: An object which contains an `enabled` field. Set this to true if using a Confluent Cloud cluster. - **topics** [Optional]: - **defaults** [Optional]: Specify topic defaults so you don't need to specify them for every topic in the state file. Currently, only replication is supported. - **blacklist** [Optional]: Add a prefixed topic blacklist for ignoring specific topics when using `kafka-gitops`. This allows topics to be ignored from being deleted if they are not defined in the desired state file. @@ -27,7 +27,7 @@ The desired state file consists of: **Example**: ```yaml settings: - ccloud: + confluent: enabled: true topics: defaults: diff --git a/examples/confluent-cloud/README.md b/examples/confluent-cloud/README.md index 94603846..ad17caf9 100644 --- a/examples/confluent-cloud/README.md +++ b/examples/confluent-cloud/README.md @@ -13,8 +13,8 @@ The following environment variables need to be set locally or in the build job: ```bash # For service account creation / listing -export XX_CCLOUD_EMAIL="Your Confluent Cloud email address" -export XX_CCLOUD_PASSWORD="Your Confluent Cloud password" +export CONFLUENT_CLOUD_EMAIL="Your Confluent Cloud email address" +export CONFLUENT_CLOUD_PASSWORD="Your Confluent Cloud password" # For executing against the cluster export KAFKA_BOOTSTRAP_SERVERS="Your Confluent Cloud cluster URL" @@ -32,7 +32,7 @@ export CLICOLOR_FORCE="true" Once defining services and users, you can generate service accounts. -**NOTE**: Before running `accounts` or `plan`, ensure you are logged in to Confluent Cloud. Use `ccloud login`. +**NOTE**: Before running `accounts` or `plan`, ensure you are logged in to Confluent Cloud. Use `confluent login`. Create accounts: diff --git a/examples/confluent-cloud/state.yaml b/examples/confluent-cloud/state.yaml index b7cfb6ed..6dfba772 100644 --- a/examples/confluent-cloud/state.yaml +++ b/examples/confluent-cloud/state.yaml @@ -1,5 +1,5 @@ settings: - ccloud: + confluent: enabled: true files: topics: topics.yaml diff --git a/src/main/java/com/devshawn/kafka/gitops/StateManager.java b/src/main/java/com/devshawn/kafka/gitops/StateManager.java index be0b797b..7eea51a4 100644 --- a/src/main/java/com/devshawn/kafka/gitops/StateManager.java +++ b/src/main/java/com/devshawn/kafka/gitops/StateManager.java @@ -322,8 +322,8 @@ private void validateTopics(DesiredStateFile desiredStateFile) { } private boolean isConfluentCloudEnabled(DesiredStateFile desiredStateFile) { - if (desiredStateFile.getSettings().isPresent() && desiredStateFile.getSettings().get().getCcloud().isPresent()) { - return desiredStateFile.getSettings().get().getCcloud().get().isEnabled(); + if (desiredStateFile.getSettings().isPresent() && desiredStateFile.getSettings().get().getConfluent().isPresent()) { + return desiredStateFile.getSettings().get().getConfluent().get().isEnabled(); } return false; } diff --git a/src/main/java/com/devshawn/kafka/gitops/domain/state/settings/Settings.java b/src/main/java/com/devshawn/kafka/gitops/domain/state/settings/Settings.java index 0753769a..2b2c11d0 100644 --- a/src/main/java/com/devshawn/kafka/gitops/domain/state/settings/Settings.java +++ b/src/main/java/com/devshawn/kafka/gitops/domain/state/settings/Settings.java @@ -9,7 +9,7 @@ @JsonDeserialize(builder = Settings.Builder.class) public interface Settings { - Optional getCcloud(); + Optional getConfluent(); Optional getTopics(); diff --git a/src/main/java/com/devshawn/kafka/gitops/domain/state/settings/SettingsCCloud.java b/src/main/java/com/devshawn/kafka/gitops/domain/state/settings/SettingsConfluent.java similarity index 59% rename from src/main/java/com/devshawn/kafka/gitops/domain/state/settings/SettingsCCloud.java rename to src/main/java/com/devshawn/kafka/gitops/domain/state/settings/SettingsConfluent.java index 5428561b..76a09e18 100644 --- a/src/main/java/com/devshawn/kafka/gitops/domain/state/settings/SettingsCCloud.java +++ b/src/main/java/com/devshawn/kafka/gitops/domain/state/settings/SettingsConfluent.java @@ -4,11 +4,11 @@ import org.inferred.freebuilder.FreeBuilder; @FreeBuilder -@JsonDeserialize(builder = SettingsCCloud.Builder.class) -public interface SettingsCCloud { +@JsonDeserialize(builder = SettingsConfluent.Builder.class) +public interface SettingsConfluent { boolean isEnabled(); - class Builder extends SettingsCCloud_Builder { + class Builder extends SettingsConfluent_Builder { } } diff --git a/src/main/java/com/devshawn/kafka/gitops/manager/ApplyManager.java b/src/main/java/com/devshawn/kafka/gitops/manager/ApplyManager.java index f32160ee..a887bc7e 100644 --- a/src/main/java/com/devshawn/kafka/gitops/manager/ApplyManager.java +++ b/src/main/java/com/devshawn/kafka/gitops/manager/ApplyManager.java @@ -1,6 +1,7 @@ package com.devshawn.kafka.gitops.manager; import com.devshawn.kafka.gitops.config.ManagerConfig; +import com.devshawn.kafka.gitops.domain.plan.AclPlan; import com.devshawn.kafka.gitops.domain.plan.DesiredPlan; import com.devshawn.kafka.gitops.domain.plan.TopicConfigPlan; import com.devshawn.kafka.gitops.domain.plan.TopicPlan; @@ -63,7 +64,9 @@ private void applyTopicConfiguration(TopicPlan topicPlan, TopicConfigPlan topicC } public void applyAcls(DesiredPlan desiredPlan) { - desiredPlan.getAclPlans().forEach(aclPlan -> { + List modifiableAclPlan = new ArrayList<>(desiredPlan.getAclPlans()); + modifiableAclPlan.sort(Comparator.comparing(AclPlan::getAction)); + modifiableAclPlan.forEach(aclPlan -> { if (aclPlan.getAction() == PlanAction.ADD) { LogUtil.printAclPreApply(aclPlan); kafkaService.createAcl(aclPlan.getAclDetails().toAclBinding()); diff --git a/src/main/java/com/devshawn/kafka/gitops/service/ConfluentCloudService.java b/src/main/java/com/devshawn/kafka/gitops/service/ConfluentCloudService.java index 113c4302..d03777a9 100644 --- a/src/main/java/com/devshawn/kafka/gitops/service/ConfluentCloudService.java +++ b/src/main/java/com/devshawn/kafka/gitops/service/ConfluentCloudService.java @@ -14,6 +14,7 @@ public class ConfluentCloudService { private static org.slf4j.Logger log = LoggerFactory.getLogger(ConfluentCloudService.class); private final ObjectMapper objectMapper; + private static final String confluentExecutable; private static final String ccloudExecutable; public ConfluentCloudService(ObjectMapper objectMapper) { @@ -21,7 +22,7 @@ public ConfluentCloudService(ObjectMapper objectMapper) { } public List getServiceAccounts() { - log.info("Fetching service account list from Confluent Cloud via ccloud tool."); + log.info("Fetching service account list from Confluent Cloud via confluent tool."); try { String result = execCmd(new String[]{ccloudExecutable, "service-account", "list", "-o", "json"}); return objectMapper.readValue(result, new TypeReference>() { @@ -33,11 +34,11 @@ public List getServiceAccounts() { } public ServiceAccount createServiceAccount(String name, boolean isUser) { - log.info("Creating service account {} in Confluent Cloud via ccloud tool.", name); + log.info("Creating service account {} in Confluent Cloud via confluent tool.", name); try { String serviceName = isUser ? String.format("user-%s", name) : name; String description = isUser ? String.format("User: %s", name) : String.format("Service account: %s", name); - String result = execCmd(new String[]{ccloudExecutable, "service-account", "create", serviceName, "--description", description, "-o", "json"}); + String result = execCmd(new String[]{confluentExecutable, "iam", "service-account", "create", serviceName, "--description", description, "-o", "json"}); return objectMapper.readValue(result, ServiceAccount.class); } catch (IOException ex) { throw new ConfluentCloudException(String.format("There was an error creating Confluent Cloud service account: %s.", name)); @@ -50,7 +51,9 @@ public static String execCmd(String[] cmd) throws java.io.IOException { } static { + confluentExecutable = System.getenv("CONFLUENT_EXECUTABLE_PATH") != null ? System.getenv("CONFLUENT_EXECUTABLE_PATH") : "confluent"; + log.info("Using confluent executable at: {}", confluentExecutable); ccloudExecutable = System.getenv("CCLOUD_EXECUTABLE_PATH") != null ? System.getenv("CCLOUD_EXECUTABLE_PATH") : "ccloud"; - log.info("Using ccloud executable at: {}", ccloudExecutable); + log.info("Using ccloud executable at: {}", confluentExecutable); } }