Bug: karavan.git.repository doesn't work correct at 4.7.0 #1347

drblack666 · 2024-08-02T15:33:48Z

Describe the bug

I tried to upgrade karavan from 4.5.1 to 4.7.0 in my k8s cluster and got an error. For some reason it uses gitea instead of my gitlab url. In pod logs:

INFO  [org.apa.cam.kar.ser.GitService] (main) Error connecting git: http://gitea:3000/karavan/karavan.git: cannot open git-upload-pack

The environment variable karavan.git.repository set up correctly:

root@karavan-8d998b4b6-gmh7b:/# env | grep git
karavan.git.repository=https://gitlab.<my_domain>.com

Downgrade to 4.6.0 or 4.5.1 fixed this problem

Steps to reproduce the behavior

Use custom karavan.git.repository and create kubernetes deployment with karavan version 4.7.0

Variant

Web Application

Container Management (if applicable)

Kubernetes

Operating System (if applicable)

None

Version

4.7.0

Relevant log output

2024-08-02 13:11:35,095 INFO  [org.apa.cam.kar.kub.KubernetesStatusService] (main) Status Listeners: starting...
2024-08-02 13:11:35,096 INFO  [org.apa.cam.kar.kub.KubernetesStatusService] (main) Stop Kubernetes Informers
2024-08-02 13:11:35,097 INFO  [org.apa.cam.kar.kub.KubernetesStatusService] (main) Starting Kubernetes Informers
2024-08-02 13:11:36,184 INFO  [org.apa.cam.kar.kub.KubernetesStatusService] (main) Started Kubernetes Informers
2024-08-02 13:11:36,184 INFO  [org.apa.cam.kar.kub.KubernetesStatusService] (main) Status Listeners: started
2024-08-02 13:11:36,188 INFO  [org.apa.cam.kar.KaravanStartupLoader] (main) Starting karavan in dev env in Kubernetes
2024-08-02 13:11:36,188 INFO  [org.apa.cam.kar.KaravanStartupLoader] (main) Projects loading...
2024-08-02 13:11:36,189 INFO  [org.apa.cam.kar.ser.GitService] (main) Check git
2024-08-02 13:11:36,194 INFO  [org.apa.cam.kar.ser.GitService] (main) Set UsernamePasswordCredentialsProvider
2024-08-02 13:11:39,357 INFO  [org.apa.cam.kar.ser.GitService] (main) Error connecting git: http://gitea:3000/karavan/karavan.git: cannot open git-upload-pack
2024-08-02 13:11:39,357 INFO  [org.apa.cam.kar.KaravanStartupLoader] (main) Starting Project service: git is ready

The text was updated successfully, but these errors were encountered:

mgubaidullin · 2024-08-02T21:48:29Z

Could you please try with KARAVAN_GIT_REPOSITORY instead of karavan.git.repository
https://github.com/quarkusio/quarkus/wiki/Migration-Guide-3.5#kubernetes-environment-variables

drblack666 · 2024-08-05T06:26:47Z

Yeah, now it sees correct repository and according to the logs writes on the correct branch, but I can't see anything. The repository is empty. If I change image version to 4.6.0 it works fine

2024-08-05 06:16:46,136 INFO  [org.apa.cam.kar.lis.CommitListener] (executor-thread-14) Commit event: {
  "message" : "newproj",
  "projectId" : "newproj",
  "userId" : "fb1fd1b0-702d-4daa-a149-a0fea5ee54ad"
}
2024-08-05 06:16:46,136 INFO  [org.apa.cam.kar.ser.ProjectService] (executor-thread-14) Commit project: newproj
2024-08-05 06:16:46,137 INFO  [org.apa.cam.kar.ser.GitService] (executor-thread-14) Commit and push project newproj
2024-08-05 06:16:46,137 INFO  [org.apa.cam.kar.ser.GitService] (executor-thread-14) Temp folder created /tmp/bbdb8efb-93d9-44de-b698-6381014de26b13943963097886175367
2024-08-05 06:16:46,137 INFO  [org.apa.cam.kar.ser.GitService] (executor-thread-14) Git checkout https://<gitlab_domain>/karavan-configuration
2024-08-05 06:16:46,137 INFO  [org.apa.cam.kar.ser.GitService] (executor-thread-14) Temp folder created /tmp/bbdb8efb-93d9-44de-b698-6381014de26b13943963097886175367
2024-08-05 06:16:46,137 INFO  [org.apa.cam.kar.ser.GitService] (executor-thread-14) Set UsernamePasswordCredentialsProvider
2024-08-05 06:16:46,178 INFO  [org.ecl.jgi.tra.TransportHttp] (executor-thread-14) URI 'https://<gitlab_domain>/karavan-configuration': following HTTP redirect #0  https://<gitlab_domain>/karavan-configuration/ -> https://<gitlab_domain>/karavan-configuration.git/
2024-08-05 06:16:46,531 INFO  [org.apa.cam.kar.ser.GitService] (executor-thread-14) Write files for project newproj
2024-08-05 06:16:46,532 INFO  [org.apa.cam.kar.ser.GitService] (executor-thread-14) Add file deployment.jkube.yaml
2024-08-05 06:16:46,532 INFO  [org.apa.cam.kar.ser.GitService] (executor-thread-14) Add file application.properties
2024-08-05 06:16:46,532 INFO  [org.apa.cam.kar.ser.GitService] (executor-thread-14) Add deleted files to git index for project newproj
2024-08-05 06:16:46,532 INFO  [org.apa.cam.kar.ser.GitService] (executor-thread-14) Checking file application.properties
2024-08-05 06:16:46,532 INFO  [org.apa.cam.kar.ser.GitService] (executor-thread-14) Checking file deployment.jkube.yaml
2024-08-05 06:16:46,532 INFO  [org.apa.cam.kar.ser.GitService] (executor-thread-14) Commit and push changes to the branch master
2024-08-05 06:16:46,535 INFO  [org.apa.cam.kar.ser.GitService] (executor-thread-14) Git add: org.eclipse.jgit.dircache.DirCache@7fd1ed39
2024-08-05 06:16:46,539 INFO  [org.apa.cam.kar.ser.GitService] (executor-thread-14) Git commit: commit 3f078a3d6cecd883364979af159285556050eb61 1722838606 ------p
2024-08-05 06:16:46,539 INFO  [org.apa.cam.kar.ser.GitService] (executor-thread-14) Set UsernamePasswordCredentialsProvider
2024-08-05 06:16:46,579 INFO  [org.ecl.jgi.tra.TransportHttp] (executor-thread-14) URI 'https://<gitlab_domain>/karavan-configuration': following HTTP redirect #0  https://<gitlab_domain>/karavan-configuration/ -> https://<gitlab_domain>/karavan-configuration.git/
2024-08-05 06:16:46,759 INFO  [org.apa.cam.kar.ser.GitService] (executor-thread-14) Git push: [org.eclipse.jgit.transport.PushResult@51852f2c]

enter-marlah · 2024-08-19T08:15:32Z

We have a similar problem.

First we replaced the karavan.git.repository with KARAVAN_GIT_REPOSITORY in the deployment.yaml to get the git connection to working.

The problem now is with the git branch. Karavan always uses main branch and pushes to the branch are not working.

We replaced karavan.git.branch with KARAVAN_GIT_BRANCH with no luck. Karavan always reads from the main branch and when we do a push logs say its using the correct branch but nothing is coming to git. The pushes dont appear in main branch or in the custom branch.

mgubaidullin · 2024-08-21T01:30:56Z

Please share your deployment.yaml

enter-marlah · 2024-08-21T06:01:28Z

Here is the contents of the deployment.yaml

---
apiVersion: "apps/v1"
kind: "Deployment"
metadata:
  name: "_CI_BRANCH_NAME_"
spec:
  replicas: 1
  selector:
    matchLabels:
      app: "_CI_BRANCH_NAME_"
  template:
    metadata:
      labels:
        app: "_CI_BRANCH_NAME_"
    spec:
      containers:
      - env:
        - name: "KARAVAN_DEVMODE_SERVICE_ACCOUNT"
          value: "karavan"
        - name: "KARAVAN_BUILDER_SERVICE_ACCOUNT"
          value: "karavan"
        - name: "KARAVAN_ENVIRONMENT"
          value: "dev"          
        - name: "KARAVAN_CONTAINER_STATUS_INTERVAL"
          value: "disabled"
        - name: "KARAVAN_CONTAINER_STATISTICS_INTERVAL"
          value: "disabled"
        - name: "KARAVAN_CAMEL_STATUS_INTERVAL"
          value: "3s"
        - name: "KARAVAN_DEVMODE_IMAGE"
          value: "ghcr.io/apache/camel-karavan-devmode:4.7.0"
        - name: "KARAVAN_GIT_REPOSITORY"
          valueFrom:
            secretKeyRef:
              key: "karavan.git.repository"
              name: "karavan"
        - name: "KARAVAN_GIT_USERNAME"
          valueFrom:
            secretKeyRef:
              key: "karavan.git.username"
              name: "karavan"
        - name: "KARAVAN_GIT_PASSWORD"
          valueFrom:
            secretKeyRef:
              key: "karavan.git.password"
              name: "karavan"
        - name: "KARAVAN_GIT_BRANCH"
          valueFrom:
            secretKeyRef:
              key: "karavan.git.branch"
              name: "karavan"
        - name: "KARAVAN_KEYCLOAK_URL"
          valueFrom:
            secretKeyRef:
              key: "karavan.keycloak.url"
              name: "karavan"
        - name: "KARAVAN_KEYCLOAK_REALM"
          valueFrom:
            secretKeyRef:
              key: "karavan.keycloak.realm"
              name: "karavan"
        - name: "KARAVAN_KEYCLOAK_FRONTEND_CLIENTID"
          valueFrom:
            secretKeyRef:
              key: "karavan.keycloak.frontend.clientId"
              name: "karavan"
        - name: "KARAVAN_KEYCLOAK_BACKEND_CLIENTID"
          valueFrom:
            secretKeyRef:
              key: "karavan.keycloak.backend.clientId"
              name: "karavan"
        - name: "KARAVAN_KEYCLOAK_BACKEND_SECRET"
          valueFrom:
            secretKeyRef:
              key: "karavan.keycloak.backend.secret"
              name: "karavan"
        - name: "KUBERNETES_NAMESPACE"
          valueFrom:
            fieldRef:
              apiVersion: ""
              fieldPath: "metadata.namespace"
        image: "ghcr.io/apache/camel-karavan:4.7.0"
        imagePullPolicy: "Always"
        name: "_CI_BRANCH_NAME_"
        ports:
        - containerPort: 8080
          name: "_CI_BRANCH_NAME_"
        resources:
          requests:
            memory: "1024Mi"
      serviceAccount: "karavan"

thgruiz · 2024-08-29T19:54:20Z

I think this would be fixed with #1389

enter-marlah · 2024-08-30T08:53:38Z

I tried the new deployment.yaml from the PR and the Karavan still uses the main branch regardless of the value in KARAVAN_GIT_BRANCH secret.

thgruiz · 2024-08-30T09:16:28Z

I tried the new deployment.yaml from the PR and the Karavan still uses the main branch regardless of the value in KARAVAN_GIT_BRANCH secret.

Yeah, sorry I missed the main reason of this bug report.

The PR I said above only chances the env var to upper case, but I did not change anything related to java code (that does the poll/push from git).

Note: my push was not working too, but after some debugging it was because my git repo's default branch was "master" and karavan-app tried to check out the inexistent branch and gives an error. Just until this is fixed, change the default branch of your repo to "main" and the pushes may work

t33muki · 2024-09-02T09:09:17Z

change the default branch of your repo to "main" and the pushes may work

Thanks, our default is main. Problem is that our specific branches get ignored.
We deploy Karavans dynamically to different namespaces and use our 'main' branch as a base.
Pipeline creates - say- 'test1' branch and adjusts Karavan secrets accordingly and deploys it to cluster. The KARAVAN_GIT_BRANCH secret now correctly contains 'test1' on the deployed Karavan - but the UI shows projects from the 'main' branch.
On 4.6.0 the branches work as intended and ui shows contents of the correct branch.

When I compared the versions 4.6.0 and 4.7.0 of
karavan-app/src/main/java/org/apache/camel/karavan/service/GitService.java
I noticed the branch handling of the getGit-method had changed quite a bit - I unfortunately don't have the knowledge or the tools to debug it further.

…clone Closes: apache#1347

Closes: apache#1347

drblack666 added the bug Something isn't working label Aug 2, 2024

mgubaidullin self-assigned this Aug 2, 2024

thgruiz added a commit to thgruiz/camel-karavan that referenced this issue Sep 2, 2024

fix(app): apache#1347 Fix the usage of karavan.git.repository on git …

13e2f1e

…clone Closes: apache#1347

thgruiz mentioned this issue Sep 2, 2024

Fix the usage of karavan.git.branch on git clone (Closes issue #1347) #1393

Merged

thgruiz added a commit to thgruiz/camel-karavan that referenced this issue Sep 2, 2024

fix(app): apache#1347 Fix the usage of karavan.git.branch on git clone

6ce81dc

Closes: apache#1347

mgubaidullin closed this as completed in d319dfa Sep 4, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Bug: karavan.git.repository doesn't work correct at 4.7.0 #1347

Bug: karavan.git.repository doesn't work correct at 4.7.0 #1347

drblack666 commented Aug 2, 2024

mgubaidullin commented Aug 2, 2024

drblack666 commented Aug 5, 2024

enter-marlah commented Aug 19, 2024

mgubaidullin commented Aug 21, 2024

enter-marlah commented Aug 21, 2024

thgruiz commented Aug 29, 2024

enter-marlah commented Aug 30, 2024

thgruiz commented Aug 30, 2024

t33muki commented Sep 2, 2024

Bug: karavan.git.repository doesn't work correct at 4.7.0 #1347

Bug: karavan.git.repository doesn't work correct at 4.7.0 #1347

Comments

drblack666 commented Aug 2, 2024

Describe the bug

Steps to reproduce the behavior

Variant

Container Management (if applicable)

Operating System (if applicable)

Version

Relevant log output

mgubaidullin commented Aug 2, 2024

drblack666 commented Aug 5, 2024

enter-marlah commented Aug 19, 2024

mgubaidullin commented Aug 21, 2024

enter-marlah commented Aug 21, 2024

thgruiz commented Aug 29, 2024

enter-marlah commented Aug 30, 2024

thgruiz commented Aug 30, 2024

t33muki commented Sep 2, 2024