[Bug]: Cannot install Strimzi on KinD

[QuinnBast]

Bug Description

Attempting to install the Strimzi kafka operator to a KinD cluster results in the operator going into a crash-loop backoff.
My KinD cluster is running on Almalinux 9 (though this shouldn't matter?)

exec /opt/strimzi/bin/cluster_operator_run.sh: exec format error
Stream closed EOF for kafka/strimzi-cluster-operator-77cc6fbbdb-hc4rq (strimzi-cluster-operator)

Steps to reproduce

1. Perform a helm install from https://strimzi.io/charts using version: 0.41.0 of strimzi-kafka-operator
2. Use the values file below.
3. Once the minfests get applied, the operator goes into a crashloop backoff.

Expected behavior

Would expect the operator to work. It works on other baremetal clusters so this might be something to do with KinD or running inside of a virtualized environment.

Strimzi version

0.41.0

Kubernetes version

v1.29.2

Installation method

Helm Chart

Infrastructure

Bare-Metal

Configuration files and logs

Chart values.yaml

defaultImageRegistry: quay.io
defaultImageRepository: strimzi
defaultImageTag: 0.41.0

priorityClassName: ""

dashboards:
  enabled: false

CRD:

apiVersion: kafka.strimzi.io/v1beta2
kind: KafkaNodePool
metadata:
  name: kafka-pool
  labels:
    # Must match the name of your cluster below
    strimzi.io/cluster: kafka-cluster
spec:
  replicas: 1
  roles:
    - controller
    - broker
  # Configure storage here.
  storage:
    type: persistent-claim
    size: 10Gi
    class: standard
    deleteClaim: false
    kraftMetadata: shared
---
apiVersion: kafka.strimzi.io/v1beta2
kind: Kafka
metadata:
  name: kafka-cluster
  annotations:
    strimzi.io/node-pools: enabled
    strimzi.io/kraft: enabled
spec:
  kafka:
    replicas: 1
    resources:
      requests:
          memory: 1Gi
          cpu: 500m
      limits:
          memory: 4Gi
          cpu: 1
    version: 3.7.0
    logging:
      type: inline
      loggers:
        kafka.root.logger.level: INFO
    jvmOptions:
      "-XX":
        UseG1GC: "true"
        MaxGCPauseMillis: "20"
    # You might need more than this if you plan on exposing kafka outside of the cluster
    listeners:
      - name: plain
        port: 9092
        type: internal
        tls: false
        configuration:
          useServiceDnsDomain: true
    config:
      offsets.topic.replication.factor: 1
      transaction.state.log.replication.factor: 1
      transaction.state.log.min.isr: 1
      default.replication.factor: 1
      min.insync.replicas: 1
      inter.broker.protocol.version: "3.7"
      delete.topic.enable: "true"
      auto.create.topics.enable: "true"
      log.retention.ms: 600000 # 10 minutes
      # The cleaner settings determine how long out-of-date records can exist on a compacted log before they
      # can be deleted. These settings should be high enough so that if a consumer starts at the beginning of
      # a compacted log topic, it should be able to read to the end before the retention time expires.
      log.cleaner.max.compaction.lag.ms: 3600000 # 1 hour
      log.cleaner.min.compaction.lag.ms: 600000 # 10 minutes
      log.cleaner.delete.retention.ms: 600000 # 10 minutes
      # Compaction is only run on older segments (not the active one), so also set a relatively small segment
      # size to trigger rollover more often.
      log.segment.bytes: 52428800 # 50 MiB
  entityOperator:
    topicOperator:
      # You need to change this to your kafka namespace if different
      watchedNamespace: kafka
      reconciliationIntervalSeconds: 30
    userOperator:
      watchedNamespace: kafka
      reconciliationIntervalSeconds: 30

Additional context

My KinD cluster is running through Podman if that means anything.

[scholzj]

the error might suggest some platform mix-up? E.g. amd64 versus arm64? But hard to say for sure without being able to reproduce it.

[QuinnBast]

Is there a different image I would need to use instead? I am just using quay.io/strimzi/operator:0.41.0

[scholzj]

If you check the image (https://quay.io/repository/strimzi/operator?tab=tags&tag=0.41.0), you will see that it is a manifest that links to 4 different images for amd64, arm64, ppc64le and s390x. Normally, your Kubernetes cluster / container engine should choose the right image from the manifest. You could also try to use the right image directly - e.g. quay.io/strimzi/operator:0.41.0-amd64 depending on your platform. But I'm not sure why would it not pull the right image in the first place.

It could of course be that you are on some other platform that we don't support. Btu there are not that many of them.

[QuinnBast]

I am using Almalinux9 in a QEMU VM. But my processor is an Intel i7 which would be ARM64.

Architecture: x86-64
Hardware Vendor: QEMU
Hardware Model: Standard PC _i440FX + PIIX, 1996_
Firmware Version: 1.13.0-1ubuntu1.1

Podman states: "If your host machine has an ARM64 CPU architecture, then Podman will run the container with an ARM64 architecture." So I'm not sure why it wouldn't be working... I will try pulling the amd64 container explicitly and see what happens.

[scholzj]

Intel Core i7 is an AMD64 platform. Not Arm64. But QEMU is as far as I know an emulator which can emulate different hardware including Arm64 (although it can be quite slow at doing that). I do not know what exactly are you trying to do with it, but maybe that is causing some confusion in the Kind cluster and makes it pull incorrect images?