attend to comments

Author: george-kalisse-sada

perfkitbenchmarker/data/k8s_agents/README.md

@@ -23,10 +23,10 @@ k8s_agents/
 |   |-- snapshot-crds.yaml.j2          # PodSnapshot StorageConfig + Policy
 |   +-- snapshot-sandbox-template.yaml.j2  # Snapshot-specific SandboxTemplate
 +-- workloads/                       # Benchmark workload scripts
-    +-- vibe_coding/                    # Startup scripts for snapshot benchmarks
+    +-- startup_scripts/                 # Startup scripts for snapshot benchmarks
         |-- README.md
-        |-- startup_pip_fastapi.sh
-        +-- startup_npm_vite.sh
+        |-- pip_fastapi.sh
+        +-- npm_vite.sh
 ```
 
 ## Manifests
@@ -42,3 +42,7 @@ Each variant YAML contains all 7 benchmark keys with cluster and
 node pool specifications. See `variants/variant_reference_guide.md`
 for detailed descriptions of what each variant tests and its expected
 impact on each benchmark.
+
+## Startup Scripts
+
+The `workloads/startup_scripts/` directory contains pluggable bash scripts used exclusively by the Pod Snapshot benchmark. They simulate an AI agent initializing a fresh environment (installing dependencies, starting servers) before a snapshot is taken. Users select which script to run via the `--k8s_snapshot_preload_mode` flag to test different memory and I/O footprints (e.g., lightweight Python vs. heavyweight Node.js).

perfkitbenchmarker/data/k8s_agents/workloads/startup_scripts/README.md

@@ -0,0 +1,56 @@
+# Agent Startup Scripts
+
+Pluggable startup scripts for the snapshot saturation harness. Each script simulates a realistic sandbox cold-start — the kind of environment setup that happens when an AI coding agent provisions a new sandbox for a user.
+
+## How It Works
+
+These scripts are exclusively used by the Pod Snapshot benchmark. They are not run simultaneously; rather, the user selects *one* of these scripts via the `--k8s_snapshot_preload_mode=script:<path>` flag. The benchmark uses the selected script to prepare the environment before taking a snapshot.
+
+1. The script is read from disk and embedded into the pod's container entrypoint.
+2. The pod runs the script to completion (installs packages, starts services, etc.).
+3. After the script exits 0, the harness prints `SCRIPT_READY` and starts a counter loop.
+4. **TTFE** (Time To First Execution) is measured as the total time from SandboxClaim creation to `SCRIPT_READY`.
+
+This allows you to compare cold-start TTFE (full script execution) against snapshot/restore TTFE (resuming from a pre-snapshotted state where the script already ran).
+
+## Scripts
+
+These scripts are provided as distinct options to test different workload characteristics (lightweight vs. heavyweight initialization).
+
+### pip_fastapi.sh
+
+**Lightweight Python variant.** Runs natively in the `python:3.11-slim` base image.
+Simulates an agent setting up a Python API. Steps: `pip install fastapi uvicorn` → create app → start uvicorn → wait for first HTTP response.
+
+Typical cold-start: ~5–8s on GKE with fast network.
+
+```bash
+# Cold-start only
+python sweeps/snapshot_saturation_search.py \
+    --skip_snapshot \
+    --k8s_snapshot_preload_mode=script:workloads/startup_scripts/pip_fastapi.sh \
+    --burst_size=3
+```
+
+### npm_vite.sh
+
+**Heavier Node.js variant.**
+Simulates an agent setting up a frontend web project. Installs Node.js + npm from apt, then npm-installs Vite and starts a dev server.
+
+Typical cold-start: ~30–60s (apt + npm on cold cache).
+
+```bash
+python sweeps/snapshot_saturation_search.py \
+    --k8s_snapshot_preload_mode=script:workloads/startup_scripts/npm_vite.sh \
+    --burst_size=3
+```
+
+## Writing Your Own Script
+
+Requirements:
+- Must be a bash script (runs via `bash -c` in a `python:3.11-slim` container)
+- Must exit 0 on success (use `set -e` for fail-fast)
+- Should print progress to stdout (visible in pod logs for debugging)
+- The harness appends `SCRIPT_READY` + counter loop after your script — don't add your own
+
+The `PRELOAD_MB` env var is available but unused by these scripts. The sweep varies it to test different memory request levels on the pod.

…orkloads/vibe_coding/startup_npm_vite.sh …ts/workloads/startup_scripts/npm_vite.shperfkitbenchmarker/data/k8s_agents/workloads/vibe_coding/startup_npm_vite.sh renamed to perfkitbenchmarker/data/k8s_agents/workloads/startup_scripts/npm_vite.sh perfkitbenchmarker/data/k8s_agents/workloads/vibe_coding/startup_npm_vite.sh renamed to perfkitbenchmarker/data/k8s_agents/workloads/startup_scripts/npm_vite.sh

@@ -1,5 +1,5 @@
 #!/bin/bash
-# Vibe Coding Startup Script — npm + Vite dev server
+# Agent Startup Script — npm + Vite dev server
 #
 # Simulates a typical agentic sandbox "vibe coding" cold-start:
 #   1. Install Node.js dependencies (bun/npm)
@@ -13,14 +13,14 @@
 # Usage (cold-start only):
 #   python sweeps/snapshot_saturation_search.py \
 #     --skip_snapshot \
-#     --preload_mode=script:workloads/vibe_coding/startup_npm_vite.sh \
+#     --preload_mode=script:workloads/startup_scripts/npm_vite.sh \
 #     --burst_size=3 \
 #     --search_mode=binary --search_min=10 --search_max=30 \
 #     --ttfe_threshold_s=120
 #
 # Usage (with snapshot/restore):
 #   python sweeps/snapshot_saturation_search.py \
-#     --preload_mode=script:workloads/vibe_coding/startup_npm_vite.sh \
+#     --preload_mode=script:workloads/startup_scripts/npm_vite.sh \
 #     --burst_size=3 \
 #     --search_mode=binary --search_min=10 --search_max=30 \
 #     --ttfe_threshold_s=120 --restore_threshold_s=10
@@ -31,10 +31,10 @@
 
 set -e
 
-echo "[vibe-coding] Installing Node.js..."
+echo "[agent-startup] Installing Node.js..."
 apt-get update -qq && apt-get install -y -qq nodejs npm > /dev/null 2>&1
 
-echo "[vibe-coding] Creating project scaffold..."
+echo "[agent-startup] Creating project scaffold..."
 mkdir -p /tmp/vibe-project && cd /tmp/vibe-project
 
 # Create a minimal package.json with Vite
@@ -59,26 +59,26 @@ cat > index.html << 'EOF'
 </html>
 EOF
 
-echo "[vibe-coding] Installing npm dependencies..."
+echo "[agent-startup] Installing npm dependencies..."
 npm install --prefer-offline 2>&1 | tail -5
 
-echo "[vibe-coding] Starting Vite dev server..."
+echo "[agent-startup] Starting Vite dev server..."
 npx vite --host 0.0.0.0 --port 5173 &
 VITE_PID=$!
 
-echo "[vibe-coding] Waiting for server to be ready..."
+echo "[agent-startup] Waiting for server to be ready..."
 MAX_WAIT=60
 ELAPSED=0
 while ! curl -s http://localhost:5173 > /dev/null 2>&1; do
     sleep 1
     ELAPSED=$((ELAPSED + 1))
     if [ $ELAPSED -ge $MAX_WAIT ]; then
-        echo "[vibe-coding] ERROR: Server did not start within ${MAX_WAIT}s"
+        echo "[agent-startup] ERROR: Server did not start within ${MAX_WAIT}s"
         exit 1
     fi
 done
 
-echo "[vibe-coding] First page served successfully (${ELAPSED}s)"
+echo "[agent-startup] First page served successfully (${ELAPSED}s)"
 
 # Kill the vite server — we only needed to measure startup time
 kill $VITE_PID 2>/dev/null || true

…loads/vibe_coding/startup_pip_fastapi.sh …workloads/startup_scripts/pip_fastapi.shperfkitbenchmarker/data/k8s_agents/workloads/vibe_coding/startup_pip_fastapi.sh renamed to perfkitbenchmarker/data/k8s_agents/workloads/startup_scripts/pip_fastapi.sh perfkitbenchmarker/data/k8s_agents/workloads/vibe_coding/startup_pip_fastapi.sh renamed to perfkitbenchmarker/data/k8s_agents/workloads/startup_scripts/pip_fastapi.sh

@@ -1,5 +1,5 @@
 #!/bin/bash
-# Lightweight Vibe Coding Startup Script — pip install + FastAPI
+# Lightweight Agent Startup Script — pip install + FastAPI
 #
 # Simulates a Python-based agentic sandbox cold-start:
 #   1. Install Python packages (FastAPI + uvicorn)
@@ -12,14 +12,14 @@
 # Usage (cold-start only):
 #   python sweeps/snapshot_saturation_search.py \
 #     --skip_snapshot \
-#     --preload_mode=script:workloads/vibe_coding/startup_pip_fastapi.sh \
+#     --preload_mode=script:workloads/startup_scripts/pip_fastapi.sh \
 #     --burst_size=3 \
 #     --search_mode=binary --search_min=10 --search_max=30 \
 #     --ttfe_threshold_s=20
 #
 # Usage (with snapshot/restore):
 #   python sweeps/snapshot_saturation_search.py \
-#     --preload_mode=script:workloads/vibe_coding/startup_pip_fastapi.sh \
+#     --preload_mode=script:workloads/startup_scripts/pip_fastapi.sh \
 #     --burst_size=3 \
 #     --search_mode=binary --search_min=10 --search_max=30 \
 #     --ttfe_threshold_s=20 --restore_threshold_s=10
@@ -30,10 +30,10 @@
 
 set -e
 
-echo "[vibe-coding] Installing Python packages..."
+echo "[agent-startup] Installing Python packages..."
 pip install --quiet fastapi uvicorn 2>&1 | tail -3
 
-echo "[vibe-coding] Creating app..."
+echo "[agent-startup] Creating app..."
 cat > /tmp/app.py << 'EOF'
 from fastapi import FastAPI
 app = FastAPI()
@@ -43,23 +43,23 @@ def root():
     return {"status": "ready"}
 EOF
 
-echo "[vibe-coding] Starting uvicorn server..."
+echo "[agent-startup] Starting uvicorn server..."
 python -m uvicorn app:app --host 0.0.0.0 --port 8000 --app-dir /tmp &
 SERVER_PID=$!
 
-echo "[vibe-coding] Waiting for server to be ready..."
+echo "[agent-startup] Waiting for server to be ready..."
 MAX_WAIT=30
 ELAPSED=0
 while ! python -c "import urllib.request; urllib.request.urlopen('http://localhost:8000/')" 2>/dev/null; do
     sleep 1
     ELAPSED=$((ELAPSED + 1))
     if [ $ELAPSED -ge $MAX_WAIT ]; then
-        echo "[vibe-coding] ERROR: Server did not start within ${MAX_WAIT}s"
+        echo "[agent-startup] ERROR: Server did not start within ${MAX_WAIT}s"
         exit 1
     fi
 done
 
-echo "[vibe-coding] First request served successfully (${ELAPSED}s)"
+echo "[agent-startup] First request served successfully (${ELAPSED}s)"
 
 # Kill the server — we only needed to measure startup time
 kill $SERVER_PID 2>/dev/null || true