Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

K8SPG-708 replace ready/live probe http check with custom command, change pg entrypoint #1099

Merged
merged 47 commits into from
Apr 9, 2025
Merged

K8SPG-708 replace ready/live probe http check with custom command, change pg entrypoint #1099

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
47 commits
Select commit Hold shift + click to select a range
7f5625f
K8SPG-708 replace ready/live probe http check with custom command
gkech Mar 20, 2025
9be2645
use bash -c for the probes to work properly
gkech Mar 24, 2025
e618bd7
fix imports
gkech Mar 24, 2025
85aada5
cr check and improve unit test
gkech Mar 24, 2025
27d6db0
fix linter
gkech Mar 24, 2025
7eaae86
e2e test use the K8SPG-708-ppg image for now
gkech Mar 24, 2025
b759611
revert experiment
gkech Mar 24, 2025
4c6b426
cr: add comments on the upstream code
gkech Mar 26, 2025
84350e5
add new entrypoint script and fix unit tests
gkech Mar 28, 2025
41132e6
utilize init container for the new .sh
gkech Apr 3, 2025
cdd7e32
fix references to op paths
gkech Apr 3, 2025
8e5f5c0
create /tmp/postgres
gkech Apr 3, 2025
e410638
fix paths again
gkech Apr 3, 2025
dcb5531
add init container to db pod
gkech Apr 3, 2025
c1f48de
fix imports
gkech Apr 3, 2025
38369c3
add crunchy bin volume to init container
gkech Apr 3, 2025
6c02b69
use the correct init image
gkech Apr 3, 2025
0a6cfff
always enable cr init image
gkech Apr 3, 2025
96ce741
fix mounts
gkech Apr 3, 2025
41afaf7
fix imports ordering
gkech Apr 3, 2025
df5ca7f
revert library check
gkech Apr 3, 2025
9300084
introduce func for instance init
gkech Apr 3, 2025
523038d
temp commit for e2e tests
gkech Apr 4, 2025
dd584de
set ppg image to main
gkech Apr 4, 2025
3a53a41
Merge branch 'main' into K8SPG-708
gkech Apr 4, 2025
61d225b
append init container
gkech Apr 4, 2025
dd07386
use K8SPG-708-12
gkech Apr 4, 2025
75a381d
fix imports
gkech Apr 4, 2025
2258101
K8SPG-613: replace initImage with initContainer
pooknull Apr 3, 2025
04b4eec
fix unit-tests
pooknull Apr 3, 2025
867e656
use inticontainer
gkech Apr 6, 2025
9291383
generate main
gkech Apr 6, 2025
041febb
remove unused ctx
gkech Apr 6, 2025
3843fd2
generate main
gkech Apr 6, 2025
3510a25
fix imports
gkech Apr 6, 2025
62ce7e2
remove ctx from caller
gkech Apr 7, 2025
61f19f6
controller test init image details
gkech Apr 7, 2025
dbe5f09
add image to missing controller test
gkech Apr 7, 2025
bcc09c7
add init container to readTestCR
gkech Apr 7, 2025
a96b148
add init image
gkech Apr 7, 2025
2ea0968
fix TestCustomLabels and TestCustomAnnotations tests
gkech Apr 7, 2025
4286621
init container image on e2e functions
gkech Apr 7, 2025
ebf5d76
add missing instance set init container
gkech Apr 7, 2025
8481dfc
missing instance init
gkech Apr 7, 2025
bf76488
Merge branch 'main' into K8SPG-708
hors Apr 8, 2025
e068324
fix custom-extensions test
hors Apr 8, 2025
41fa93d
Merge branch 'main' into K8SPG-708
hors Apr 8, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
264 changes: 264 additions & 0 deletions build/crd/crunchy/generated/postgres-operator.crunchydata.com_postgresclusters.yaml
View file
Open in desktop

Large diffs are not rendered by default.

264 changes: 264 additions & 0 deletions build/crd/percona/generated/pgv2.percona.com_perconapgclusters.yaml
View file
Open in desktop

Large diffs are not rendered by default.

3 changes: 3 additions & 0 deletions build/postgres-operator/Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -63,6 +63,9 @@ COPY --from=go_builder /usr/local/bin/pgbackrest /usr/local/bin/
COPY --from=go_builder /licenses /licenses
COPY build/postgres-operator/install-extensions.sh /usr/local/bin
COPY build/postgres-operator/init-entrypoint.sh /usr/local/bin
COPY build/postgres-operator/postgres-entrypoint.sh /usr/local/bin
COPY build/postgres-operator/postgres-liveness-check.sh /usr/local/bin
COPY build/postgres-operator/postgres-readiness-check.sh /usr/local/bin
COPY hack/tools/queries /opt/crunchy/conf

RUN chgrp -R 0 /opt/crunchy/conf && chmod -R g=u opt/crunchy/conf
Expand Down
3 changes: 3 additions & 0 deletions build/postgres-operator/init-entrypoint.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,3 +6,6 @@ set -o xtrace
CRUNCHY_BINDIR="/opt/crunchy"

install -o "$(id -u)" -g "$(id -g)" -m 0755 -D "/usr/local/bin/pgbackrest" "${CRUNCHY_BINDIR}/bin/pgbackrest"
install -o "$(id -u)" -g "$(id -g)" -m 0755 -D "/usr/local/bin/postgres-entrypoint.sh" "${CRUNCHY_BINDIR}/bin/postgres-entrypoint.sh"
install -o "$(id -u)" -g "$(id -g)" -m 0755 -D "/usr/local/bin/postgres-liveness-check.sh" "${CRUNCHY_BINDIR}/bin/postgres-liveness-check.sh"
install -o "$(id -u)" -g "$(id -g)" -m 0755 -D "/usr/local/bin/postgres-readiness-check.sh" "${CRUNCHY_BINDIR}/bin/postgres-readiness-check.sh"
20 changes: 20 additions & 0 deletions build/postgres-operator/postgres-entrypoint.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
#!/bin/bash

# /pgdata/ is already mounted to the pg database container with rw permissions
recovery_file='/pgdata/sleep-forever'
if [[ -f ${recovery_file} ]]; then
set +o xtrace
echo "The $recovery_file file is detected, node entered an infinite sleep"
echo "If you want to exit from the infinite sleep, remove the $recovery_file file"

if [[ ! -d /tmp/postgres ]]; then
mkdir -p /tmp/postgres
fi

while [ -f "${recovery_file}" ]; do
sleep 3
done
exit 0
fi

exec "$@"
20 changes: 20 additions & 0 deletions build/postgres-operator/postgres-liveness-check.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
#!/bin/bash

PATRONI_PORT=8008
PATRONI_HOST=localhost

# /pgdata/ is already mounted to the pg database container with rw permissions
recovery_file='/pgdata/sleep-forever'
if [ -f "${recovery_file}" ]; then
set +o xtrace
echo "The $recovery_file file is detected, node entered an infinite sleep"
echo "If you want to exit from the infinite sleep, remove the $recovery_file file"
exit 0
fi

response=$(curl -s -o /dev/null -w "%{http_code}" -k "https://${PATRONI_HOST}:${PATRONI_PORT}/liveness")

if [[ $response -eq 200 ]]; then
exit 0
fi
exit 1
11 changes: 11 additions & 0 deletions build/postgres-operator/postgres-readiness-check.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
#!/bin/bash

PATRONI_PORT=8008
PATRONI_HOST=localhost

response=$(curl -s -o /dev/null -w "%{http_code}" -k "https://${PATRONI_HOST}:${PATRONI_PORT}/readiness")

if [[ $response -eq 200 ]]; then
exit 0
fi
exit 1
264 changes: 264 additions & 0 deletions config/crd/bases/pgv2.percona.com_perconapgclusters.yaml
View file
Open in desktop

Large diffs are not rendered by default.

264 changes: 264 additions & 0 deletions config/crd/bases/postgres-operator.crunchydata.com_postgresclusters.yaml
View file
Open in desktop

Large diffs are not rendered by default.

528 changes: 528 additions & 0 deletions deploy/bundle.yaml
View file
Open in desktop

Large diffs are not rendered by default.

25 changes: 25 additions & 0 deletions deploy/cr.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -174,6 +174,31 @@ spec:
instances:
- name: instance1
replicas: 3
# initContainer:
# image: perconalab/percona-postgresql-operator:main
# resources:
# limits:
# cpu: 2.0
# memory: 4Gi
# containerSecurityContext:
# fsGroup: 1001
# runAsUser: 1001
# runAsNonRoot: true
# fsGroupChangePolicy: "OnRootMismatch"
# runAsGroup: 1001
# seLinuxOptions:
# type: spc_t
# level: s0:c123,c456
# seccompProfile:
# type: Localhost
# localhostProfile: localhost/profile.json
# supplementalGroups:
# - 1001
# sysctls:
# - name: net.ipv4.tcp_keepalive_time
# value: "600"
# - name: net.ipv4.tcp_keepalive_intvl
# value: "60"

affinity:
podAntiAffinity:
Expand Down
528 changes: 528 additions & 0 deletions deploy/crd.yaml
View file
Open in desktop

Large diffs are not rendered by default.

528 changes: 528 additions & 0 deletions deploy/cw-bundle.yaml
View file
Open in desktop

Large diffs are not rendered by default.

1 change: 1 addition & 0 deletions e2e-tests/functions
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -178,6 +178,7 @@ get_cr() {
.spec.users += [{"name":"postgres","password":{"type":"AlphaNumeric"}}] |
.spec.users += [{"name":"'${cr_name}'","password":{"type":"AlphaNumeric"}}] |
.spec.image = "'$IMAGE_POSTGRESQL'" |
.spec.initContainer.image = "perconalab/percona-postgresql-operator:K8SPG-708-12" |
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should we fix this image with main tag?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it is just for tests. As soon as we see that all tests are ok we need to remove it at all

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes correct, we have it only for these tests to work and to confirm that the PR is alright. We will change it on a followup PR after we merge.

.spec.backups.pgbackrest.image = "'$IMAGE_BACKREST'" |
.spec.proxy.pgBouncer.image = "'$IMAGE_PGBOUNCER'" |
.spec.pmm.image = "'$IMAGE_PMM_CLIENT'" |
Expand Down
2 changes: 2 additions & 0 deletions e2e-tests/tests/custom-extensions/02-assert.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -52,6 +52,8 @@ spec:
- name: postgres-data
- mountPath: /tmp
name: tmp
- command:
- /usr/local/bin/init-entrypoint.sh
- name: nss-wrapper-init
status:
observedGeneration: 2
Expand Down
2 changes: 2 additions & 0 deletions e2e-tests/tests/custom-extensions/05-assert.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -52,6 +52,8 @@ spec:
- name: postgres-data
- mountPath: /tmp
name: tmp
- command:
- /usr/local/bin/init-entrypoint.sh
- name: nss-wrapper-init
status:
observedGeneration: 2
Expand Down
2 changes: 2 additions & 0 deletions e2e-tests/tests/custom-extensions/06-assert.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -54,6 +54,8 @@ spec:
- name: postgres-data
- mountPath: /tmp
name: tmp
- command:
- /usr/local/bin/init-entrypoint.sh
- name: nss-wrapper-init
status:
observedGeneration: 3
Expand Down
4 changes: 3 additions & 1 deletion e2e-tests/tests/custom-extensions/07-assert.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,8 @@ spec:
- /usr/local/bin/relocate-extensions.sh
- command:
- /usr/local/bin/install-extensions.sh
- command:
- /usr/local/bin/init-entrypoint.sh
- name: nss-wrapper-init
status:
observedGeneration: 3
Expand All @@ -47,4 +49,4 @@ status:
size: 3
ready: 3
size: 3
state: ready
state: ready
40 changes: 32 additions & 8 deletions internal/controller/postgrescluster/cluster_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -144,11 +144,17 @@ func TestCustomLabels(t *testing.T) {
cluster.ObjectMeta.Name = "global-cluster"
cluster.ObjectMeta.Namespace = ns.Name
cluster.Spec.InstanceSets = []v1beta1.PostgresInstanceSetSpec{{
Name: "daisy-instance1",
Name: "daisy-instance1",
InitContainer: &v1beta1.InitContainerSpec{
Image: "some-image",
},
Replicas: initialize.Int32(1),
DataVolumeClaimSpec: testVolumeClaimSpec(),
}, {
Name: "daisy-instance2",
Name: "daisy-instance2",
InitContainer: &v1beta1.InitContainerSpec{
Image: "some-image",
},
Replicas: initialize.Int32(1),
DataVolumeClaimSpec: testVolumeClaimSpec(),
}}
Expand Down Expand Up @@ -194,14 +200,20 @@ func TestCustomLabels(t *testing.T) {
cluster.ObjectMeta.Name = "instance-cluster"
cluster.ObjectMeta.Namespace = ns.Name
cluster.Spec.InstanceSets = []v1beta1.PostgresInstanceSetSpec{{
Name: "max-instance",
Name: "max-instance",
InitContainer: &v1beta1.InitContainerSpec{
Image: "some-image",
},
Replicas: initialize.Int32(1),
DataVolumeClaimSpec: testVolumeClaimSpec(),
Metadata: &v1beta1.Metadata{
Labels: map[string]string{"my.instance.label": "max"},
},
}, {
Name: "lucy-instance",
Name: "lucy-instance",
InitContainer: &v1beta1.InitContainerSpec{
Image: "some-image",
},
Replicas: initialize.Int32(1),
DataVolumeClaimSpec: testVolumeClaimSpec(),
Metadata: &v1beta1.Metadata{
Expand Down Expand Up @@ -396,11 +408,17 @@ func TestCustomAnnotations(t *testing.T) {
cluster.ObjectMeta.Name = "global-cluster"
cluster.ObjectMeta.Namespace = ns.Name
cluster.Spec.InstanceSets = []v1beta1.PostgresInstanceSetSpec{{
Name: "daisy-instance1",
Name: "daisy-instance1",
InitContainer: &v1beta1.InitContainerSpec{
Image: "some-image",
},
Replicas: initialize.Int32(1),
DataVolumeClaimSpec: testVolumeClaimSpec(),
}, {
Name: "daisy-instance2",
Name: "daisy-instance2",
InitContainer: &v1beta1.InitContainerSpec{
Image: "some-image",
},
Replicas: initialize.Int32(1),
DataVolumeClaimSpec: testVolumeClaimSpec(),
}}
Expand Down Expand Up @@ -447,14 +465,20 @@ func TestCustomAnnotations(t *testing.T) {
cluster.ObjectMeta.Name = "instance-cluster"
cluster.ObjectMeta.Namespace = ns.Name
cluster.Spec.InstanceSets = []v1beta1.PostgresInstanceSetSpec{{
Name: "max-instance",
Name: "max-instance",
InitContainer: &v1beta1.InitContainerSpec{
Image: "some-image",
},
Replicas: initialize.Int32(1),
DataVolumeClaimSpec: testVolumeClaimSpec(),
Metadata: &v1beta1.Metadata{
Annotations: map[string]string{"my.instance.annotation": "max"},
},
}, {
Name: "lucy-instance",
Name: "lucy-instance",
InitContainer: &v1beta1.InitContainerSpec{
Image: "some-image",
},
Replicas: initialize.Int32(1),
DataVolumeClaimSpec: testVolumeClaimSpec(),
Metadata: &v1beta1.Metadata{
Expand Down
6 changes: 6 additions & 0 deletions internal/controller/postgrescluster/controller_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -97,6 +97,8 @@ spec:
image: postgres
instances:
- name: register-now
initContainer:
image: postgres
dataVolumeClaimSpec:
accessModes:
- "ReadWriteMany"
Expand Down Expand Up @@ -233,6 +235,8 @@ spec:
image: postgres
instances:
- name: samba
initContainer:
image: postgres
dataVolumeClaimSpec:
accessModes:
- "ReadWriteMany"
Expand Down Expand Up @@ -438,6 +442,8 @@ spec:
image: postgres
instances:
- name: samba
initContainer:
image: postgres
dataVolumeClaimSpec:
accessModes:
- "ReadWriteMany"
Expand Down
5 changes: 4 additions & 1 deletion internal/controller/postgrescluster/helpers_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -116,7 +116,10 @@ func testCluster() *v1beta1.PostgresCluster {
Name: "myImagePullSecret"},
},
InstanceSets: []v1beta1.PostgresInstanceSetSpec{{
Name: "instance1",
Name: "instance1",
InitContainer: &v1beta1.InitContainerSpec{
Image: "some-image",
},
Replicas: initialize.Int32(1),
DataVolumeClaimSpec: testVolumeClaimSpec(),
}},
Expand Down
9 changes: 8 additions & 1 deletion internal/controller/postgrescluster/instance.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,7 @@ import (
"github.com/percona/percona-postgresql-operator/internal/pgbackrest"
"github.com/percona/percona-postgresql-operator/internal/pki"
"github.com/percona/percona-postgresql-operator/internal/postgres"
"github.com/percona/percona-postgresql-operator/percona/k8s"
"github.com/percona/percona-postgresql-operator/pkg/apis/postgres-operator.crunchydata.com/v1beta1"
)

Expand Down Expand Up @@ -1206,9 +1207,15 @@ func (r *Reconciler) reconcileInstance(
ctx, cluster, instanceCertificates, &instance.Spec.Template.Spec)
}

// K8SPG-708
initImage, err := k8s.InitImage(ctx, r.Client, cluster, spec)
if err != nil {
return errors.Wrap(err, "failed to determine initial init image")
}

err = patroni.InstancePod(
ctx, cluster, clusterConfigMap, clusterPodService, patroniLeaderService,
spec, instanceCertificates, instanceConfigMap, &instance.Spec.Template)
spec, instanceCertificates, instanceConfigMap, &instance.Spec.Template, initImage) // K8SPG-708
}

// Add pgMonitor resources to the instance Pod spec
Expand Down
Loading
Loading