Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Integrating block-sync in an init-container #779

Open
wants to merge 7 commits into
base: master
Choose a base branch
from
Open
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Next Next commit
Added yaml file changes and prometheus builder changes that was removed
from block-sync.
Signed-off-by: Kushal Shukla <[email protected]>
kushalShukla-web committed Nov 15, 2024

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
commit e55d9dd632c229546d8b28675cf25b39f57820cb
15 changes: 14 additions & 1 deletion prombench/docs/kind.md
Original file line number Diff line number Diff line change
@@ -114,7 +114,20 @@ If used with the GitHub integration:
export PR_NUMBER=<PR to benchmark against the selected $RELEASE>
```

2. Deploy the Kubernetes objects:
2. Before applying benchmarking objects , You have two choices to make:
- **Option 1: Download data from object storage**
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How would someone coming to this for the first time understand this choice?
Can we give some context why data might be downloaded?


To download data from object storage, create a Kubernetes secret with exact named `bucket-config` and file name ```object-config.yml``` with the necessary credentials as per your object storage. This secret enables access to the stored data.
> Note: Make sure this secret applied before ```3b_prometheus-test_deployment.yaml```
- **Option 2: Skip downloading data**

If you don’t need to download data, edit the `3b_prometheus-test_deployment.yaml` file:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could we instead tell people to create an empty Secret?


- Remove the `bucket-config` volume section from.
- Remove the `volumeMount` section name `bucket-config` from `data-downloader`.
> Note: You have to remove these two sections from both prometheus-test-pr-{{ .PR_NUMBER }} and prometheus-test-{{ normalise .RELEASE }} deployments.
3. Deploy the Kubernetes objects:
> **_Note:_** If you encounter a `too many files open` error caused by promtail, increase the default value of `/proc/sys/fs/inotify/max_user_instances` from 128 to 512:
> ```bash
> sudo sysctl fs.inotify.max_user_instances=512
Original file line number Diff line number Diff line change
@@ -34,7 +34,7 @@ spec:
runAsUser: 0
initContainers:
- name: prometheus-builder
image: docker.io/prominfra/prometheus-builder:master
image: kushalshukla/builder
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fix up before merge.

imagePullPolicy: Always
env:
- name: PR_NUMBER
@@ -48,6 +48,18 @@ spec:
volumeMounts:
- name: prometheus-executable
mountPath: /prometheus-builder
- name: key
mountPath: /config
- name: data-downloader
image: kushalshukla/writer
imagePullPolicy: Always
volumeMounts:
- name: instance-ssd
mountPath: /data
- name: bucket-config
mountPath: /config
- name: key
mountPath: /key
containers:
- name: prometheus
image: quay.io/prometheus/busybox:latest
@@ -88,6 +100,11 @@ spec:
path: /mnt/disks/ssd0 #gke ssds
- name: prometheus-executable
emptyDir: {}
- name: bucket-config # Define the Secret volume
secret:
secretName: bucket-secret
- name: key
emptyDir: {}
terminationGracePeriodSeconds: 300
nodeSelector:
node-name: prometheus-{{ .PR_NUMBER }}
@@ -144,6 +161,31 @@ spec:
- prometheus
securityContext:
runAsUser: 0
initContainers:
- name: download-key
image: kushalshukla/builder
imagePullPolicy: Always
command: [ "/download-key/key.sh" ]
env:
- name: PR_NUMBER
value: "{{ .PR_NUMBER }}"
- name: GITHUB_ORG
value: "{{ .GITHUB_ORG }}"
- name: GITHUB_REPO
value: "{{ .GITHUB_REPO }}"
volumeMounts:
- name: key
mountPath: /config
- name: data-downloader
image: kushalshukla/writer
imagePullPolicy: Always
volumeMounts:
- name: instance-ssd
mountPath: /data
- name: bucket-config
mountPath: /config
- name: key
mountPath: /key
containers:
- name: prometheus
image: quay.io/prometheus/prometheus:{{ .RELEASE }}
@@ -172,6 +214,14 @@ spec:
# /mnt is where GKE keeps it's SSD
# don't change this if you want Prometheus to take advantage of these local SSDs
path: /mnt/disks/ssd0
- name: config
hostPath:
path: /object-config
- name: bucket-config # Define the Secret volume
secret:
secretName: bucket-secret
- name: key
emptyDir: {}
terminationGracePeriodSeconds: 300
nodeSelector:
node-name: prometheus-{{ .PR_NUMBER }}
2 changes: 2 additions & 0 deletions tools/prometheus-builder/Dockerfile
Original file line number Diff line number Diff line change
@@ -4,6 +4,8 @@ RUN mkdir -p /go/src/github.com

COPY ./build.sh /go/src/github.com/build.sh

COPY ./key.sh /download-key/key.sh
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is this in such a different place from the other script?


RUN chmod +x /go/src/github.com/build.sh

ENTRYPOINT ["/go/src/github.com/build.sh"]
11 changes: 11 additions & 0 deletions tools/prometheus-builder/build.sh
Original file line number Diff line number Diff line change
@@ -24,6 +24,17 @@ fi

git checkout pr-branch

# Here, MKDIR is specified in the volumeMount section of the prometheus-builder init container,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It would be better to have this in another script.

# where it will copy the key.yml file from the Prometheus directory to the volume section of the
# emptyDir. This file will later be used by the data-downloader init container.
MKDIR="/config"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think you need to change the name.

if [ -f "$DIR/key.yml" ]; then
echo "File exists."
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Who is expected to read this message, and what are they expected to do about it?

cp "$DIR/key.yml" "$MKDIR/key.yml"
else
echo "File does not exist."
fi

echo ">> Creating prometheus binaries"
if ! make build PROMU_BINARIES="prometheus"; then
echo "ERROR:: Building of binaries failed"
36 changes: 36 additions & 0 deletions tools/prometheus-builder/key.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
#!/bin/bash
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's be more specific concerning the filename. key.sh is too generic.


DIR="/go/src/github.com/prometheus/prometheus"

if [[ -z $PR_NUMBER || -z $VOLUME_DIR || -z $GITHUB_ORG || -z $GITHUB_REPO ]]; then
echo "ERROR:: environment variables not set correctly"
exit 1;
fi

# Clone the repository with a shallow clone
echo ">> Cloning repository $GITHUB_ORG/$GITHUB_REPO (shallow clone)"
if ! git clone --depth 1 https://github.com/$GITHUB_ORG/$GITHUB_REPO.git $DIR; then
echo "ERROR:: Cloning of repo $GITHUB_ORG/$GITHUB_REPO failed"
exit 1;
fi

cd $DIR || exit 1

echo ">> Fetching Pull Request $GITHUB_ORG/$GITHUB_REPO/pull/$PR_NUMBER"
if ! git fetch origin pull/$PR_NUMBER/head:pr-branch; then
echo "ERROR:: Fetching of PR $PR_NUMBER failed"
exit 1;
fi

git checkout pr-branch

# Here, MKDIR is specified in the volumeMount section of the prometheus-builder init container,
# where it will copy the key.yml file from the Prometheus directory to the volume section of the
# emptyDir. This file will later be used by the data-downloader init container.
MKDIR="/config"
if [ -f "$DIR/key.yml" ]; then
echo "File exists."
cp "$DIR/key.yml" "$MKDIR/key.yml"
else
echo "File does not exist."
fi