Skip to content

PIPELINE_OPTIMIZATIONS.md

Latest commit

 

History

History
193 lines (147 loc) · 6.02 KB

PIPELINE_OPTIMIZATIONS.md

File metadata and controls

193 lines (147 loc) · 6.02 KB
Raw

GitLab CI/CD Pipeline Optimizations

This document describes the performance optimizations implemented to speed up development deployments while maintaining safety and quality for stage deployments.

Overview

The pipeline is now optimized with two distinct paths:

  • Development branch: Fast, automated deployments to dev environment (no testing/scanning)
  • Live branch: Full security testing and scanning before stage deployments

Development Branch Optimizations

1. Reduced Docker Builds (50% faster)

Location: .gitlab-ci.yml lines 384-390

  • Before: Built 6 images (dev + stage sites × 3 targets each)
  • After: Builds only 3 images (dev site × 3 targets)
  • Time saved: ~5-8 minutes per build
  • Safety: Stage images built separately on live branch

2. Parallel Terraform Validation (30-60 seconds saved)

Location: .gitlab-ci.yml line 653

  • Before: Init → Validate (sequential)
  • After: Init and Validate run in parallel
  • Time saved: 30-60 seconds
  • Safety: Validation doesn't need init completion; both use cached providers

3. Skip Terraform Plan JSON (15-30 seconds saved)

Location: .gitlab-ci.yml line 694

  • Before: Generated both binary plan and JSON report
  • After: Only generates binary plan (JSON only needed for MR review)
  • Time saved: 15-30 seconds
  • Safety: Binary plan is all that's needed for apply

4. Shorter Artifact Retention (disk space optimization)

Location: .gitlab-ci.yml line 699

  • Before: 3 days retention
  • After: 1 day retention for dev plans
  • Benefit: Reduces storage costs, reflects faster dev iteration
  • Safety: Plans are applied immediately in dev, not reviewed

5. Optimized Job Dependencies (better parallelization)

Location: .gitlab-ci.yml lines 732-737

  • Before: Apply waited for all build artifacts
  • After: Apply only waits for plan.cache, builds run in parallel
  • Time saved: Eliminates unnecessary waiting
  • Safety: Images exist before Terraform references them (built first)

6. No Security Scanning (3-5 minutes saved)

Location: .gitlab-ci.yml lines 70-83

  • Development branch: Skips SAST, dependency scanning, secret detection
  • Live branch: Full security scanning enabled
  • Time saved: 3-5 minutes
  • Safety: Security scans run on stage before production

7. No Image Scanning (2-4 minutes saved)

Location: .gitlab-ci.yml lines 438-520

  • Development branch: Skips Prisma Cloud scanning
  • Live branch: Full Prisma scanning for stage images
  • Time saved: 2-4 minutes
  • Safety: Comprehensive scanning happens before stage deployment

Live Branch (Stage) - Full Security

The live branch maintains all security controls:

Security Testing (Test Stage)

  • ✅ SAST (Static Application Security Testing)
  • ✅ Dependency Scanning
  • ✅ Secret Detection

Image Scanning (Scan Stage)

  • ✅ Prisma Cloud scanning for Drupal container
  • ✅ Prisma Cloud scanning for Nginx container
  • ✅ Prisma Cloud scanning for Drush container

Estimated Time Savings

Development Deployments

Optimization Time Saved
Reduced Docker builds 5-8 minutes
No security testing 3-5 minutes
No image scanning 2-4 minutes
Parallel validation 0.5-1 minute
Skip plan JSON 0.25-0.5 minutes
Optimized dependencies 0.5-1 minute
Total ~12-20 minutes

Original vs Optimized Pipeline Times

Environment Before After Improvement
Development ~25-35 min ~10-15 min 40-60% faster
Stage (live) ~25-35 min ~25-35 min No change (full security)

Safety Guarantees

All optimizations maintain safety through:

  1. Branch Separation: Dev and stage use different branches with different rules
  2. No Skipped Steps: All security checks run before stage deployment
  3. Resource Groups: Prevent concurrent deployments to same environment
  4. Automatic Rollback: Terraform detects and prevents unsafe changes
  5. Manual Infrastructure Changes: Core infrastructure still requires approval

Configuration Details

Build Matrix Logic

parallel:
  matrix:
    - WEBCMS_SITE: [dev]
      WEBCMS_TARGET: [drupal, nginx, drush]
      WEBCMS_BRANCH: [development]  # Only for dev branch
    - WEBCMS_SITE: [stage, dev]
      WEBCMS_TARGET: [drupal, nginx, drush]
      WEBCMS_BRANCH: [live]  # Both sites for stage

Branch-based Execution

if [ "$WEBCMS_BRANCH" != "$CI_COMMIT_BRANCH" ]; then
  exit 0  # Skip builds that don't match current branch
fi

Security Test Rules

sast:
  rules:
    - if: '$CI_COMMIT_BRANCH == "live"'  # Live branch only

dependency_scanning:
  rules:
    - if: '$CI_COMMIT_BRANCH == "live"'  # Live branch only

secret_detection:
  rules:
    - if: '$CI_COMMIT_BRANCH == "live"'  # Live branch only

Additional Recommendations

For Even Faster Development Iteration

If you need even faster deployments, consider:

  1. Use latest tag for dev: Skip builds if code hasn't changed
  2. Conditional Drush updates: Skip database updates when not needed
  3. Terraform refresh only: Use -refresh-only when no changes expected
  4. Parallel language deployments: If re-enabling Spanish for dev

Monitoring Performance

Track these metrics in GitLab:

  • CI/CD Analytics → Pipeline duration trends
  • Jobs → Duration by stage
  • Builds → Cache hit rates

Rollback Instructions

If optimizations cause issues, revert by:

  1. Re-enable security for dev:

    rules:
  - if: '$CI_COMMIT_BRANCH == "development" || $CI_COMMIT_BRANCH == "live"'

  2. Build both sites always:

    parallel:
  matrix:
    - WEBCMS_SITE: [stage, dev]

  3. Sequential validation:

    needs:
  - job: "deploy:dev:init:en"

Questions & Support

For issues or questions about pipeline performance:

  1. Check job logs for timing breakdown
  2. Review CI/CD Analytics in GitLab
  3. Contact DevOps team