From a6d8fbc42a5ab2bf190c2d43dd3a184172a3eb9e Mon Sep 17 00:00:00 2001 From: Benjamin Wheeler Date: Tue, 10 Dec 2024 13:50:20 -0500 Subject: [PATCH 01/30] cranky-docs: Initial cranky documentation commit. WIP. Signed-off-by: Benjamin Wheeler --- docs/tutorial/cranking-the-kernel.md | 239 +++++++++++++++++++++++++++ 1 file changed, 239 insertions(+) create mode 100644 docs/tutorial/cranking-the-kernel.md diff --git a/docs/tutorial/cranking-the-kernel.md b/docs/tutorial/cranking-the-kernel.md new file mode 100644 index 0000000..f142141 --- /dev/null +++ b/docs/tutorial/cranking-the-kernel.md @@ -0,0 +1,239 @@ +# Turning the Crank + +## Introduction + +In this tutorial, we will crank an Ubuntu kernel, which means to apply updates to an existing derivative Ubuntu kernel from its parent kernel, package it, and prepare it for testing. + +[cranky](https://kernel.ubuntu.com/gitea/kernel/kteam-tools/src/branch/master/cranky) is a toolchain which assists in all of these steps. + +### Table of Contents/Steps in this Guide + +The steps of cranking a kernel are relatively straightforward: + +1. Set Up/Update Environment +2. Download Current Version of the Kernel +3. Apply Updates from the Upstream Kernel +4. Verify the Kernel Builds Successfully +5. Package the kernel for Release +6. Upload kernel for release/review + +--- + + +In this tutorial, we will crank a 24.04 LTS (Noble Numbat) (codename `noble`) Google cloud kernel (codename `linux-gke`). Keep these codenames in mind for future commands. + +Learn more about Ubuntu releases and derivative kernels [here](TODo). + + +## 1. Set Up/Update Environment + +First, follow the [Cranky Environment Setup](TODO) tutorial. + +### chroot environment +We use [chroot](https://en.wikipedia.org/wiki/Chroot) environments to isolate different sets of tools when kernel cranking. cranky helps us set up and manage these chroot jails. + +If it's our first time creating a chroot for this release (Noble Numbat), we must first create the chroot base: +```bash +cranky chroot create-base noble:linux-gke +``` +This will output a lot of text as it installs various packages needed for the chroot to work properly. (It took ~20 minutes to complete.) + +Next, create the chroot session: +```bash +cranky chroot create-session noble:linux-gke +``` + +This will also output a lot of text as it uses apt to install various packages needed for the crank. (It took ~2 minutes to complete.) + + + +### Update kteam-tools +Finally, update your local clone of kteam-tools to the latest commit on the main branch: + +```bash +cd ~/canonical/kteam-tools/ +git pull +``` + +## 2. Download Current Version of the Kernel +Next, clone the chosen kernel in its current state. + + + +```bash +cd ~/canonical/kernel/ubuntu/ +cranky checkout noble:linux-gke +``` + +Once this command is finished (It took ~20 minutes to complete), you should see the following directories inside the newly-created `./noble/linux-gke/` directory: +- `linux-lrm/` +- `linux-main/`: The actual Linux kernel source +- `linux-meta/` +- `linux-signed/`: Canonical's kernel signing stuff + + +## 3. Apply Updates from the Upstream Kernel +The upstream kernel will have updates that should be propagated down to this kernel. + +### 3.1. Fix helper scripts +Update the local (in-tree) helper scripts cranky uses to the latest version: +```bash +cd ~/canonical/kernel/ubuntu/noble/linux-gke/linux-main/ +cranky fix +``` +You should see some output showing that cranky executed several scripts. + + +### 3.2. Rebase on top of updated parent +Because this is a derivative kernel, we need to apply updates from its parent, the generic kernel: + +```bash +cranky rebase +``` + +:::{tip} +For non-derivative kernels (e.g., `noble:linux`), this step is not required, and doesn't do anything. +::: + +### 3.3. Fix helper scripts (again) +Sometimes after a `cranky rebase`, the helper scripts get updated. It's good practice to always re-run: + +```bash +cranky fix +``` + + +## TODO new section +### Starting commit + + + +Run the following command: + +```bash +cranky open +``` + +This command downloads ABI files from the previous release. + +This also creates a new commit. Run `git show` and see that this new commit starts with "UBUNTU: Start new release" and shows an update to `./debian.gke/changelog`. + + +### Review +Sometimes the rebase doesn't get all the changes. Run this command to manually review any outstanding changes: +```bash +cranky review-master-changes +``` + + +In particular, check if any changes were made to `debian.master/rules.d/` that +should be reflected in `debian.aws/rules.d/`. Those commits may not be +obvious in the master changelog. + + +In the event of a config change in the derivative, this commit title is +used: "UBUNTU: \[Config\] : Whatever title", +i.e "UBUNTU: \[Config\] aws: Whatever title". +This is useful in the changelog to make the distiction between a config +change in the parent vs a change in the derivative. + +### Link to the Launchpad Bug Tracker + +Run the following command to link this kernel to its corresponding Launchpad bug tracker: + +:::{caution} +Use `--dry-run` unless you are actually cranking a kernel. Otherwise, this will overwrite Launchpad and might make destructive changes! +::: + +```bash +cranky link-tb --dry-run +``` + + + +### Update DKMS Packages +```bash +cranky update-dkms-versions +``` + + + +### Closing commit +This step creates one final commit before a release is prepared. Run: +```bash +cranky close +``` + + +## 4. Verify the Kernel Builds Successfully +### Cloud Builder +Connect to the canonical VPN. Then, run: + + + + + +```bash +git push cbd +``` + + + + +## 5. Package the kernel for Release +Run the following command: + +```bash +cranky update-dependents +``` + +### Tag commit +```bash +cranky tags +``` + + +### Verify Preparation +```bash +cranky verify-release-ready +``` + +The `tag pushed: warning` is an expected warning if you do not have commit rights. + +### Pull sources + +```bash +cd .. +# CWD == ~/canonical/kernel/noble/linux-gke/ + +cranky pull-sources noble:linux-gke --latest +``` + +### Build Sources +```bash +cd linux-main/ +cranky build-sources +``` + +## Review +```bash +cd .. +# CWD == ~/canonical/kernel/noble/linux-gke/ +cranky review *.changes +``` + +## Upload + + + +Normally, at this point a cranker with upload rights would publish the newly-cranked kernel to its PPA, which automatically kicks off boot testing. +However, without upload rights, we can request someone with rights to review our work, and publish the kernel on our behalf. + +We can push the created files to a server that is accessible to the reviewer, like `kathleen`. + + +```bash +cd linux-main/ +cranky push-review -s 2024.10.08 kathleen +``` + From b4e92fc314c19e83eaf36fe7dd20e630cf4e7c61 Mon Sep 17 00:00:00 2001 From: Benjamin Wheeler Date: Wed, 11 Dec 2024 16:09:33 -0500 Subject: [PATCH 02/30] (WIP) Add more info to cranky tutorial. Also, switch to noble gke Signed-off-by: Benjamin Wheeler --- docs/tutorial/cranking-the-kernel.md | 49 +++++++++++++++++++--------- 1 file changed, 33 insertions(+), 16 deletions(-) diff --git a/docs/tutorial/cranking-the-kernel.md b/docs/tutorial/cranking-the-kernel.md index f142141..38c9048 100644 --- a/docs/tutorial/cranking-the-kernel.md +++ b/docs/tutorial/cranking-the-kernel.md @@ -66,7 +66,6 @@ cranky checkout noble:linux-gke ``` Once this command is finished (It took ~20 minutes to complete), you should see the following directories inside the newly-created `./noble/linux-gke/` directory: -- `linux-lrm/` - `linux-main/`: The actual Linux kernel source - `linux-meta/` - `linux-signed/`: Canonical's kernel signing stuff @@ -114,9 +113,13 @@ Run the following command: cranky open ``` -This command downloads ABI files from the previous release. +This command creates a commit which signifies the start of a new release. It updates `debian.gke/changelog` to reflect this. +Optionally, it may also update ABI versioning info. -This also creates a new commit. Run `git show` and see that this new commit starts with "UBUNTU: Start new release" and shows an update to `./debian.gke/changelog`. + + + +Run `git show` and see that this new commit starts with "UBUNTU: Start new release" and shows an update to `./debian.gke/changelog`. ### Review @@ -124,22 +127,20 @@ Sometimes the rebase doesn't get all the changes. Run this command to manually r ```bash cranky review-master-changes ``` - +This command will output any outstanding changes in a list with their commit hashes and descriptions. Use `git show ` to view the changes. -In particular, check if any changes were made to `debian.master/rules.d/` that -should be reflected in `debian.aws/rules.d/`. Those commits may not be -obvious in the master changelog. +Usually these can be ignored, but there are a few instances where further investigation is necessary: - -In the event of a config change in the derivative, this commit title is -used: "UBUNTU: \[Config\] : Whatever title", -i.e "UBUNTU: \[Config\] aws: Whatever title". -This is useful in the changelog to make the distiction between a config -change in the parent vs a change in the derivative. +- Commits with changes to `debian.master/rules.d/` + - Choose if these changes should be reflected in `debian.gke/rules.d/` +- Commits with descriptions starting with "UBUNTU: \[Config\]: ..." + - These indicate a change in the parent kernel's configuration. + - You'll need to compare this change with what appears in the derivative config (`debian.gke/`) to decide if it should be applied to this crank. ### Link to the Launchpad Bug Tracker Run the following command to link this kernel to its corresponding Launchpad bug tracker: + :::{caution} Use `--dry-run` unless you are actually cranking a kernel. Otherwise, this will overwrite Launchpad and might make destructive changes! @@ -149,29 +150,45 @@ Use `--dry-run` unless you are actually cranking a kernel. Otherwise, this will cranky link-tb --dry-run ``` - +This updates the Launchpad tracking bug, which, among other things, will be used as an input for subsequent steps. + + ### Update DKMS Packages + +`debian.master/dkms-versions` specifies dkms modules to be packaged with its kernel. + +This command updates the package versions in `debian.gke/dkms-versions` to match the ones expected for the SRU cycle + ```bash cranky update-dkms-versions ``` +In most cases there would be no change committed as the up-to-date versions should have been committed on the master kernel and picked-up by the derivative or backport on rebase. However, if there is any change, check that the version numbers only become higher and nothing gets dropped completely. In case anything looks suspicious, don’t hesitate to ask the team if the changes are expected. + ### Closing commit This step creates one final commit before a release is prepared. Run: ```bash cranky close ``` - +This command is a shortcut for several steps: + +1. Verifies there are no changes left. +2. Inserts changes from the parent kernel into the changelog. +3. Inserts git changes into the changelog. +4. Updates the release series, author and date on the changelog, thus closing the changelog. +5. Creates a commit signifying the finished crank. ## 4. Verify the Kernel Builds Successfully ### Cloud Builder + + Connect to the canonical VPN. Then, run: - ```bash git push cbd From 442a961b19c1e590995de8f2d6a4c85cbecb77af Mon Sep 17 00:00:00 2001 From: Benjamin Wheeler Date: Thu, 12 Dec 2024 15:22:29 -0500 Subject: [PATCH 03/30] Remove placeholder links that prevent sphinx build. Signed-off-by: Benjamin Wheeler --- docs/tutorial/cranking-the-kernel.md | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/docs/tutorial/cranking-the-kernel.md b/docs/tutorial/cranking-the-kernel.md index 38c9048..e3ec040 100644 --- a/docs/tutorial/cranking-the-kernel.md +++ b/docs/tutorial/cranking-the-kernel.md @@ -22,12 +22,12 @@ The steps of cranking a kernel are relatively straightforward: In this tutorial, we will crank a 24.04 LTS (Noble Numbat) (codename `noble`) Google cloud kernel (codename `linux-gke`). Keep these codenames in mind for future commands. -Learn more about Ubuntu releases and derivative kernels [here](TODo). +Learn more about Ubuntu releases and derivative kernels [here]. ## 1. Set Up/Update Environment -First, follow the [Cranky Environment Setup](TODO) tutorial. +First, follow the [Cranky Environment Setup] tutorial. ### chroot environment We use [chroot](https://en.wikipedia.org/wiki/Chroot) environments to isolate different sets of tools when kernel cranking. cranky helps us set up and manage these chroot jails. @@ -152,6 +152,8 @@ cranky link-tb --dry-run This updates the Launchpad tracking bug, which, among other things, will be used as an input for subsequent steps. + + ### Update DKMS Packages From 5d58b2ade1068d5606d84f9567b6af4d83638393 Mon Sep 17 00:00:00 2001 From: Benjamin Wheeler Date: Thu, 12 Dec 2024 15:23:37 -0500 Subject: [PATCH 04/30] Add MyST colon fence extension. Signed-off-by: Benjamin Wheeler --- docs/conf.py | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/docs/conf.py b/docs/conf.py index aac9423..d5f786d 100644 --- a/docs/conf.py +++ b/docs/conf.py @@ -214,7 +214,9 @@ # NOTE: By default, the following MyST extensions are enabled: # substitution, deflist, linkify -# myst_enable_extensions = set() +myst_enable_extensions = { + "colon_fence", +} # Custom Sphinx extensions; see # https://www.sphinx-doc.org/en/master/usage/extensions/index.html From c4c6f61195d1e53c946c80b9bc6048650a83d006 Mon Sep 17 00:00:00 2001 From: Benjamin Wheeler Date: Thu, 12 Dec 2024 15:25:14 -0500 Subject: [PATCH 05/30] Add `cranking-the-kernel.md` to the landing page. Now, Sphinx will build and we can see the rendered page. Signed-off-by: Benjamin Wheeler --- docs/index.md | 1 + 1 file changed, 1 insertion(+) diff --git a/docs/index.md b/docs/index.md index 672507a..a2cb1b4 100644 --- a/docs/index.md +++ b/docs/index.md @@ -76,4 +76,5 @@ fixes and constructive feedback. /how-to/index /reference/index /explanation/index +/tutorial/cranking-the-kernel.md ``` From 6635aded8a67d6ab67dcfe0aae7dc5a044d8dd46 Mon Sep 17 00:00:00 2001 From: Benjamin Wheeler Date: Wed, 18 Dec 2024 09:54:04 -0500 Subject: [PATCH 06/30] (WIP) add comments Signed-off-by: Benjamin Wheeler --- docs/tutorial/cranking-the-kernel.md | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/docs/tutorial/cranking-the-kernel.md b/docs/tutorial/cranking-the-kernel.md index e3ec040..d36f973 100644 --- a/docs/tutorial/cranking-the-kernel.md +++ b/docs/tutorial/cranking-the-kernel.md @@ -54,6 +54,7 @@ Finally, update your local clone of kteam-tools to the latest commit on the main cd ~/canonical/kteam-tools/ git pull ``` + ## 2. Download Current Version of the Kernel Next, clone the chosen kernel in its current state. @@ -188,6 +189,7 @@ This command is a shortcut for several steps: Connect to the canonical VPN. Then, run: + @@ -199,6 +201,8 @@ git push cbd + + ## 5. Package the kernel for Release Run the following command: From 0b008e5b2fbf3e671f2f9fbf8709621874b9baca Mon Sep 17 00:00:00 2001 From: Benjamin Wheeler Date: Wed, 18 Dec 2024 12:45:37 -0500 Subject: [PATCH 07/30] Improve title Signed-off-by: Benjamin Wheeler --- docs/tutorial/cranking-the-kernel.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/tutorial/cranking-the-kernel.md b/docs/tutorial/cranking-the-kernel.md index d36f973..e0f4707 100644 --- a/docs/tutorial/cranking-the-kernel.md +++ b/docs/tutorial/cranking-the-kernel.md @@ -1,4 +1,4 @@ -# Turning the Crank +# Tutorial: Crank your first kernel ## Introduction From 4a280f559e7baee253cfdbdec7e4b8151830c0e2 Mon Sep 17 00:00:00 2001 From: Benjamin Wheeler Date: Wed, 18 Dec 2024 12:47:58 -0500 Subject: [PATCH 08/30] Change filename to match title Signed-off-by: Benjamin Wheeler --- docs/index.md | 2 +- .../{cranking-the-kernel.md => crank-your-first-kernel.md} | 0 2 files changed, 1 insertion(+), 1 deletion(-) rename docs/tutorial/{cranking-the-kernel.md => crank-your-first-kernel.md} (100%) diff --git a/docs/index.md b/docs/index.md index a2cb1b4..8d4cbb5 100644 --- a/docs/index.md +++ b/docs/index.md @@ -76,5 +76,5 @@ fixes and constructive feedback. /how-to/index /reference/index /explanation/index -/tutorial/cranking-the-kernel.md +/tutorial/crank-your-first-kernel.md ``` diff --git a/docs/tutorial/cranking-the-kernel.md b/docs/tutorial/crank-your-first-kernel.md similarity index 100% rename from docs/tutorial/cranking-the-kernel.md rename to docs/tutorial/crank-your-first-kernel.md From 0b55aa357247b3e2d170c66f20e44737809409e5 Mon Sep 17 00:00:00 2001 From: Benjamin Wheeler Date: Wed, 18 Dec 2024 13:02:41 -0500 Subject: [PATCH 09/30] Improve introduction. * Remove table of contents * Move content to Related Topics * Reword content Signed-off-by: Benjamin Wheeler --- docs/tutorial/crank-your-first-kernel.md | 22 +++------------------- 1 file changed, 3 insertions(+), 19 deletions(-) diff --git a/docs/tutorial/crank-your-first-kernel.md b/docs/tutorial/crank-your-first-kernel.md index e0f4707..56d7ea5 100644 --- a/docs/tutorial/crank-your-first-kernel.md +++ b/docs/tutorial/crank-your-first-kernel.md @@ -2,29 +2,11 @@ ## Introduction -In this tutorial, we will crank an Ubuntu kernel, which means to apply updates to an existing derivative Ubuntu kernel from its parent kernel, package it, and prepare it for testing. - -[cranky](https://kernel.ubuntu.com/gitea/kernel/kteam-tools/src/branch/master/cranky) is a toolchain which assists in all of these steps. - -### Table of Contents/Steps in this Guide - -The steps of cranking a kernel are relatively straightforward: - -1. Set Up/Update Environment -2. Download Current Version of the Kernel -3. Apply Updates from the Upstream Kernel -4. Verify the Kernel Builds Successfully -5. Package the kernel for Release -6. Upload kernel for release/review - ---- +Cranking an Ubuntu kernel is the process of applying updates to an existing Ubuntu kernel, packaging it, and preparing it for testing. [cranky](https://kernel.ubuntu.com/gitea/kernel/kteam-tools/src/branch/master/cranky) is a toolchain which assists in all of these steps. In this tutorial, we will crank a 24.04 LTS (Noble Numbat) (codename `noble`) Google cloud kernel (codename `linux-gke`). Keep these codenames in mind for future commands. -Learn more about Ubuntu releases and derivative kernels [here]. - - ## 1. Set Up/Update Environment First, follow the [Cranky Environment Setup] tutorial. @@ -260,3 +242,5 @@ cd linux-main/ cranky push-review -s 2024.10.08 kathleen ``` +## Related topics +Learn more about Ubuntu releases and derivative kernels [here]. From 257c271cb4c2a36022aa373353828d2b96e0f610 Mon Sep 17 00:00:00 2001 From: Benjamin Wheeler Date: Wed, 18 Dec 2024 14:55:16 -0500 Subject: [PATCH 10/30] Number sections. Output of `git checkout cranky-tutorial`, other misc. Signed-off-by: Benjamin Wheeler --- docs/tutorial/crank-your-first-kernel.md | 58 +++++++++++++++++------- 1 file changed, 41 insertions(+), 17 deletions(-) diff --git a/docs/tutorial/crank-your-first-kernel.md b/docs/tutorial/crank-your-first-kernel.md index 56d7ea5..a24ad3a 100644 --- a/docs/tutorial/crank-your-first-kernel.md +++ b/docs/tutorial/crank-your-first-kernel.md @@ -11,7 +11,7 @@ In this tutorial, we will crank a 24.04 LTS (Noble Numbat) (codename `noble`) Go First, follow the [Cranky Environment Setup] tutorial. -### chroot environment +### 1.1. Update chroot environment We use [chroot](https://en.wikipedia.org/wiki/Chroot) environments to isolate different sets of tools when kernel cranking. cranky helps us set up and manage these chroot jails. If it's our first time creating a chroot for this release (Noble Numbat), we must first create the chroot base: @@ -29,15 +29,37 @@ This will also output a lot of text as it uses apt to install various packages n -### Update kteam-tools +### 1.2. Update kteam-tools Finally, update your local clone of kteam-tools to the latest commit on the main branch: +For the scope of this tutorial, we will use a particular version of cranky. Run the following commands to get it: + +:::{WARNING} +If your kteam-tools tree isn't clean, be sure to save your work before running the commands. +They will modify the repository! +Git should warn you if any destructive actions might occur. +::: ```bash cd ~/canonical/kteam-tools/ git pull +git checkout cranky-tutorial ``` +After `checkout`, you should see output similar similar to this: + +:::{terminal} + :input: git checkout cranky-tutorial + :dir: ~/canonical/kteam-tools/ + :user: user + :host: host +Note: switching to 'cranky-tutorial'. + +... + +HEAD is now at de4674e4 mainline-build/cod-update-virgin: Split script and update freedesktop repos +::: + ## 2. Download Current Version of the Kernel Next, clone the chosen kernel in its current state. @@ -55,7 +77,7 @@ Once this command is finished (It took ~20 minutes to complete), you should see ## 3. Apply Updates from the Upstream Kernel -The upstream kernel will have updates that should be propagated down to this kernel. +The upstream kernel will have changes that should be propagated down to this kernel. ### 3.1. Fix helper scripts Update the local (in-tree) helper scripts cranky uses to the latest version: @@ -85,8 +107,8 @@ cranky fix ``` -## TODO new section -### Starting commit +## 4. TODO new section +### 4.1. Starting commit @@ -105,7 +127,7 @@ Optionally, it may also update ABI versioning info. @@ -139,7 +161,7 @@ This updates the Launchpad tracking bug, which, among other things, will be used -### Update DKMS Packages +### 4.4. Update DKMS Packages `debian.master/dkms-versions` specifies dkms modules to be packaged with its kernel. @@ -153,7 +175,7 @@ cranky update-dkms-versions In most cases there would be no change committed as the up-to-date versions should have been committed on the master kernel and picked-up by the derivative or backport on rebase. However, if there is any change, check that the version numbers only become higher and nothing gets dropped completely. In case anything looks suspicious, don’t hesitate to ask the team if the changes are expected. -### Closing commit +### 4.5. Closing commit This step creates one final commit before a release is prepared. Run: ```bash cranky close @@ -166,7 +188,9 @@ This command is a shortcut for several steps: 4. Updates the release series, author and date on the changelog, thus closing the changelog. 5. Creates a commit signifying the finished crank. -## 4. Verify the Kernel Builds Successfully +## 5. Verify the Kernel Builds Successfully +At this point, the kernel is built and packaged. We should test that it builds successfully. + ### Cloud Builder @@ -185,27 +209,27 @@ git push cbd -## 5. Package the kernel for Release +## 6. Package the kernel for Release Run the following command: ```bash cranky update-dependents ``` -### Tag commit +### 6.1. Tag commit ```bash cranky tags ``` -### Verify Preparation +### 6.2. Verify Preparation ```bash cranky verify-release-ready ``` The `tag pushed: warning` is an expected warning if you do not have commit rights. -### Pull sources +### 6.3. Pull sources ```bash cd .. @@ -214,20 +238,20 @@ cd .. cranky pull-sources noble:linux-gke --latest ``` -### Build Sources +### 6.4. Build Sources ```bash cd linux-main/ cranky build-sources ``` -## Review +## 7. Review ```bash cd .. # CWD == ~/canonical/kernel/noble/linux-gke/ cranky review *.changes ``` -## Upload +## 8. Upload From 94547117d0cee586673cfc57d9e6298d3f004f99 Mon Sep 17 00:00:00 2001 From: Benjamin Wheeler Date: Tue, 14 Jan 2025 11:49:13 -0500 Subject: [PATCH 11/30] Set up tutorial parent page. Set up the Tutorials page and properly nest the Cranky tutorial underneath it, rather than at the top level. Signed-off-by: Benjamin Wheeler --- docs/index.md | 4 ++-- docs/tutorial/index.md | 12 ++++++++++++ 2 files changed, 14 insertions(+), 2 deletions(-) create mode 100644 docs/tutorial/index.md diff --git a/docs/index.md b/docs/index.md index 8d4cbb5..c73593b 100644 --- a/docs/index.md +++ b/docs/index.md @@ -25,7 +25,7 @@ processes for customization and maintenance. ````{grid} 1 1 2 2 -```{grid-item-card} [Tutorials](index) +```{grid-item-card} [Tutorials](/tutorial/index) **Start here**: a hands-on introduction to Ubuntu Linux kernel development for new users @@ -76,5 +76,5 @@ fixes and constructive feedback. /how-to/index /reference/index /explanation/index -/tutorial/crank-your-first-kernel.md +/tutorial/index ``` diff --git a/docs/tutorial/index.md b/docs/tutorial/index.md new file mode 100644 index 0000000..f609f7d --- /dev/null +++ b/docs/tutorial/index.md @@ -0,0 +1,12 @@ +# Tutorials + +The following tutorials will present you with different tutorials that address +key concepts, tools, processes and operations. + + +```{toctree} +:titlesonly: +:maxdepth: 1 + +Crank your first kernel +``` From c6cbabb5925a7103445e0d3e83f7ff7dd457224b Mon Sep 17 00:00:00 2001 From: Benjamin Wheeler Date: Wed, 15 Jan 2025 12:21:44 -0500 Subject: [PATCH 12/30] Add output of link-tb Signed-off-by: Benjamin Wheeler --- docs/tutorial/crank-your-first-kernel.md | 31 +++++++++++++++++++++++- 1 file changed, 30 insertions(+), 1 deletion(-) diff --git a/docs/tutorial/crank-your-first-kernel.md b/docs/tutorial/crank-your-first-kernel.md index a24ad3a..0ede6bd 100644 --- a/docs/tutorial/crank-your-first-kernel.md +++ b/docs/tutorial/crank-your-first-kernel.md @@ -159,7 +159,36 @@ This updates the Launchpad tracking bug, which, among other things, will be used - +This will create a git commit. Run `git show` to see it. It should look something like this: + +```gitdiff +commit 4345a7fc255b03ff9072cdcec1779a9b39d7519b (HEAD -> cranky/master-next) +Author: Benjamin Wheeler +Date: Wed Jan 15 12:14:59 2025 -0500 + + UBUNTU: link-to-tracker: update tracking bug + + BugLink: https://bugs.launchpad.net/bugs/2093494 + Properties: no-test-build + Signed-off-by: Benjamin Wheeler + +diff --git a/debian.gke/tracking-bug b/debian.gke/tracking-bug +index 6f5c6b4a700c..b3436ee66bdd 100644 +--- a/debian.gke/tracking-bug ++++ b/debian.gke/tracking-bug +@@ -1 +1 @@ +-2090338 s2024.10.28-1 ++2093494 s2024.12.02-1 +``` + +Then, click the `BugLink` URL to see the relevant Launchpad tracking bug. +A comment should appear on there that says something like: +``` + summary: - noble/linux-gke: -proposed tracker + + noble/linux-gke: 6.8.0-1017.21 -proposed tracker +``` + +If you see similar results, you've successfully linked to the tracking bug. ### 4.4. Update DKMS Packages From 2bb7e7c4a9c2243c90875a3fc30e31990f2fd48d Mon Sep 17 00:00:00 2001 From: Benjamin Wheeler Date: Wed, 15 Jan 2025 12:53:43 -0500 Subject: [PATCH 13/30] Add git commit output of `cranky close` Signed-off-by: Benjamin Wheeler --- docs/tutorial/crank-your-first-kernel.md | 45 +++++++++++++++++++++++- 1 file changed, 44 insertions(+), 1 deletion(-) diff --git a/docs/tutorial/crank-your-first-kernel.md b/docs/tutorial/crank-your-first-kernel.md index 0ede6bd..302c4bd 100644 --- a/docs/tutorial/crank-your-first-kernel.md +++ b/docs/tutorial/crank-your-first-kernel.md @@ -200,7 +200,11 @@ This command updates the package versions in `debian.gke/dkms-versions` to match cranky update-dkms-versions ``` - +No changes are needed this time, so you'll see this output: +``` +debian.gke/dkms-versions: No changes from kernel-versions +``` + In most cases there would be no change committed as the up-to-date versions should have been committed on the master kernel and picked-up by the derivative or backport on rebase. However, if there is any change, check that the version numbers only become higher and nothing gets dropped completely. In case anything looks suspicious, don’t hesitate to ask the team if the changes are expected. @@ -217,6 +221,45 @@ This command is a shortcut for several steps: 4. Updates the release series, author and date on the changelog, thus closing the changelog. 5. Creates a commit signifying the finished crank. +If the output went well, you should see a new commit when you run `git show`: +```gitdiff +commit 6c9a5055b22f4c30aa3ba0c9df306762edb29197 (HEAD -> cranky/master-next) +Author: Benjamin Wheeler +Date: Wed Jan 15 12:42:31 2025 -0500 + + UBUNTU: Ubuntu-gke-6.8.0-1017.21 + + Signed-off-by: Benjamin Wheeler + +diff --git a/debian.gke/changelog b/debian.gke/changelog +index ba1191ce3bc2..a9ccd0b375a6 100644 +--- a/debian.gke/changelog ++++ b/debian.gke/changelog +@@ -1,10 +1,18 @@ +-linux-gke (6.8.0-1017.21) UNRELEASED; urgency=medium ++linux-gke (6.8.0-1017.21) noble; urgency=medium + +- CHANGELOG: Do not edit directly. Autogenerated at release. +- CHANGELOG: Use the printchanges target to see the curent changes. +- CHANGELOG: Use the insertchanges target to create the final log. ++ * noble/linux-gke: 6.8.0-1017.21 -proposed tracker (LP: #2093494) + +- -- Benjamin Wheeler Wed, 15 Jan 2025 12:04:33 -0500 ++ [ Ubuntu: 6.8.0-52.53 ] ++ ++ * noble/linux: 6.8.0-52.53 -proposed tracker (LP: #2093521) ++ * CVE-2024-53164 ++ - net: sched: fix ordering of qlen adjustment ++ * CVE-2024-53141 ++ - netfilter: ipset: add missing range check in bitmap_ip_uadt ++ * CVE-2024-53103 ++ - hv_sock: Initializing vsk->trans to NULL to prevent a dangling pointer ++ ++ -- Benjamin Wheeler Wed, 15 Jan 2025 12:42:30 -0500 + + linux-gke (6.8.0-1016.20) noble; urgency=medium +``` + ## 5. Verify the Kernel Builds Successfully At this point, the kernel is built and packaged. We should test that it builds successfully. From bf0d4a188367113f805ca62ace776ff1d9640dd7 Mon Sep 17 00:00:00 2001 From: Benjamin Wheeler Date: Wed, 15 Jan 2025 13:04:06 -0500 Subject: [PATCH 14/30] Fix pygment lexer gitdiff -> diff Signed-off-by: Benjamin Wheeler --- docs/tutorial/crank-your-first-kernel.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/tutorial/crank-your-first-kernel.md b/docs/tutorial/crank-your-first-kernel.md index 302c4bd..6a12087 100644 --- a/docs/tutorial/crank-your-first-kernel.md +++ b/docs/tutorial/crank-your-first-kernel.md @@ -161,7 +161,7 @@ This updates the Launchpad tracking bug, which, among other things, will be used This will create a git commit. Run `git show` to see it. It should look something like this: -```gitdiff +```diff commit 4345a7fc255b03ff9072cdcec1779a9b39d7519b (HEAD -> cranky/master-next) Author: Benjamin Wheeler Date: Wed Jan 15 12:14:59 2025 -0500 @@ -222,7 +222,7 @@ This command is a shortcut for several steps: 5. Creates a commit signifying the finished crank. If the output went well, you should see a new commit when you run `git show`: -```gitdiff +```diff commit 6c9a5055b22f4c30aa3ba0c9df306762edb29197 (HEAD -> cranky/master-next) Author: Benjamin Wheeler Date: Wed Jan 15 12:42:31 2025 -0500 From c4cba458f37a3705f6728dadd66888295698a3cc Mon Sep 17 00:00:00 2001 From: Benjamin Wheeler Date: Wed, 15 Jan 2025 13:26:46 -0500 Subject: [PATCH 15/30] Add output of cbd Signed-off-by: Benjamin Wheeler --- docs/tutorial/crank-your-first-kernel.md | 15 +++++++++++++++ 1 file changed, 15 insertions(+) diff --git a/docs/tutorial/crank-your-first-kernel.md b/docs/tutorial/crank-your-first-kernel.md index 6a12087..38ae354 100644 --- a/docs/tutorial/crank-your-first-kernel.md +++ b/docs/tutorial/crank-your-first-kernel.md @@ -279,6 +279,21 @@ git push cbd +The output will initially look like this: +``` +Total 0 (delta 0), reused 0 (delta 0), pack-reused 0 +remote: *** kernel-cbd ********************************************************* +remote: * Queueing builds (your 'cranky/master-next'); ok to interrupt +remote: * For results: ssh cbd ls benjaminwheeler-noble-6c9a5055b22f-szkb +``` + +At this point you can interrupt the command with Ctrl-C and periodically check the build status with the ssh command it printed. + +When running it, you'll see a few files either called `BUILDING` or `BUILD-OK`. + +Once all the arches have `BUILD-OK`, we know the kernel built successfully. + + ## 6. Package the kernel for Release From feb09462c0912d8872df248d34f7f767b0e7f364 Mon Sep 17 00:00:00 2001 From: Benjamin Wheeler <34549440+benjamin051000@users.noreply.github.com> Date: Tue, 21 Jan 2025 13:28:44 -0500 Subject: [PATCH 16/30] Annecyh/cranky tutorial suggested edits (#52) * WIP: suggested edits for tutorial Signed-off-by: annecyh * WIP: suggested edits for tutorial Signed-off-by: annecyh * WIP Signed-off-by: annecyh * WIP Signed-off-by: annecyh --------- Signed-off-by: annecyh Co-authored-by: annecyh Signed-off-by: Benjamin Wheeler --- docs/conf.py | 1 + .../cranking/set-up-cranky-environment.md | 69 +++++ docs/how-to/index.md | 9 + docs/index.md | 4 +- docs/reference/glossary.md | 2 +- docs/tutorial/crank-your-first-kernel.md | 241 ++++++++++++------ docs/tutorial/index.md | 4 +- 7 files changed, 245 insertions(+), 85 deletions(-) create mode 100644 docs/how-to/cranking/set-up-cranky-environment.md diff --git a/docs/conf.py b/docs/conf.py index d5f786d..0e2f35a 100644 --- a/docs/conf.py +++ b/docs/conf.py @@ -216,6 +216,7 @@ myst_enable_extensions = { "colon_fence", + "substitution", } # Custom Sphinx extensions; see diff --git a/docs/how-to/cranking/set-up-cranky-environment.md b/docs/how-to/cranking/set-up-cranky-environment.md new file mode 100644 index 0000000..82560fb --- /dev/null +++ b/docs/how-to/cranking/set-up-cranky-environment.md @@ -0,0 +1,69 @@ +# How to set up cranky environment + +This guide describes the one-off process to configure your build environment to +crank Ubuntu kernels using `cranky`. + +## Install dependencies + +First, you will need to install the required packages. + +### Debian packages + +Use apt to install the following debian packages: + +```{code-block} text +sudo apt install \ + bash-completion \ + build-essential \ + ccache \ + debhelper \ + devscripts \ + docbook-utils \ + fakeroot \ + gawk \ + git \ + git-email \ + kernel-wedge \ + libncurses5-dev \ + makedumpfile \ + python3-launchpadlib \ + python3-ruamel.yaml \ + schroot \ + sharutils \ + transfig \ + ubuntu-dev-tools \ + wget \ + xmlto +``` + +### Snapcraft package + +Install snapcraft according to the Ubuntu release running on your build machine. + +```{tip} +You can check your release of Ubuntu by running `lsb_release -a`. +``` + +`````{tab-set} +````{tab-item} 24.04 LTS (Noble Numbat) or later + +Install `snapcraft` using snap: + +```{code-block} text +snap install snapcraft --classic +``` +```` + +````{tab-item} 22.04 LTS (Jammy Jellyfish) or earlier + +Install `snapcraft` using apt: + +```{code-block} text +sudo apt install snapcraft +``` +```` + +````` + + + diff --git a/docs/how-to/index.md b/docs/how-to/index.md index ba070ef..bf5c944 100644 --- a/docs/how-to/index.md +++ b/docs/how-to/index.md @@ -10,6 +10,7 @@ kernel packages and components. Source code access and management Development and customization +Set up cranky environment Test kernels in -proposed Contribute to kernel docs ``` @@ -22,6 +23,14 @@ tree to build custom kernel modules, and more. - [Obtain kernel source for an Ubuntu release using Git](/how-to/source-code/obtain-kernel-source-git) +## Cranking kernels + +In Canonical, "kernel cranking" refers to the process of applying patches and +updates to an Ubuntu kernel, packaging it, and getting it ready for testing -- +all done using the `cranky` toolchain. + +- [Set up cranky environment](/how-to/cranking/set-up-cranky-environment) + ## Development and customization The steps to build a kernel is similar but may have slightly difference diff --git a/docs/index.md b/docs/index.md index c73593b..4f8c8c8 100644 --- a/docs/index.md +++ b/docs/index.md @@ -28,7 +28,7 @@ processes for customization and maintenance. ```{grid-item-card} [Tutorials](/tutorial/index) **Start here**: a hands-on introduction to Ubuntu Linux kernel development for -new users +new users. ``` ```{grid-item-card} [How-to guides](/how-to/index) @@ -73,8 +73,8 @@ fixes and constructive feedback. :hidden: :maxdepth: 2 +/tutorial/index /how-to/index /reference/index /explanation/index -/tutorial/index ``` diff --git a/docs/reference/glossary.md b/docs/reference/glossary.md index 8d06c18..52a6f9b 100644 --- a/docs/reference/glossary.md +++ b/docs/reference/glossary.md @@ -47,7 +47,7 @@ linux-meta When a new kernel version is released and marked stable, the linux-meta package is updated to reference the new version, allowing automatic upgrades. -linux-signed +linux-signed Refers to kernel packages that are cryptographically signed to ensure their integrity and authenticity. These signatures are crucial for secure boot environments, as they enable the diff --git a/docs/tutorial/crank-your-first-kernel.md b/docs/tutorial/crank-your-first-kernel.md index 38ae354..fa917bf 100644 --- a/docs/tutorial/crank-your-first-kernel.md +++ b/docs/tutorial/crank-your-first-kernel.md @@ -1,43 +1,50 @@ # Tutorial: Crank your first kernel -## Introduction - -Cranking an Ubuntu kernel is the process of applying updates to an existing Ubuntu kernel, packaging it, and preparing it for testing. [cranky](https://kernel.ubuntu.com/gitea/kernel/kteam-tools/src/branch/master/cranky) is a toolchain which assists in all of these steps. +Cranking an Ubuntu kernel is the process of applying patches and updates to an existing Ubuntu kernel, packaging it, and preparing it for testing. All this done using the [cranky](https://kernel.ubuntu.com/gitea/kernel/kteam-tools/src/branch/master/cranky) toolchain. In this tutorial, we will crank a 24.04 LTS (Noble Numbat) (codename `noble`) Google cloud kernel (codename `linux-gke`). Keep these codenames in mind for future commands. -## 1. Set Up/Update Environment +## Set up and update build environment + + +You will need to complete the setup according to the {doc}`cranky environment setup ` guide before continuing. -First, follow the [Cranky Environment Setup] tutorial. +### Update chroot environment -### 1.1. Update chroot environment -We use [chroot](https://en.wikipedia.org/wiki/Chroot) environments to isolate different sets of tools when kernel cranking. cranky helps us set up and manage these chroot jails. +We use [chroot](https://en.wikipedia.org/wiki/Chroot) environments to isolate different sets of tools when doing kernel cranking. `cranky` helps us set up and manage these chroot jails. -If it's our first time creating a chroot for this release (Noble Numbat), we must first create the chroot base: + +If this is your first time creating a chroot for the Noble Numbat release, you must first create the chroot base: + + ```bash cranky chroot create-base noble:linux-gke ``` -This will output a lot of text as it installs various packages needed for the chroot to work properly. (It took ~20 minutes to complete.) + +This can take up to 20 minutes to complete. +If successful, you should observe the various packages being installed and set up in the terminal output. Next, create the chroot session: + ```bash cranky chroot create-session noble:linux-gke ``` -This will also output a lot of text as it uses apt to install various packages needed for the crank. (It took ~2 minutes to complete.) +This step uses APT to install various packages needed for the crank and takes about two minutes to complete. - +### Update kteam-tools repository -### 1.2. Update kteam-tools -Finally, update your local clone of kteam-tools to the latest commit on the main branch: +Update your local clone of `kteam-tools` to the latest commit on the `master` branch: For the scope of this tutorial, we will use a particular version of cranky. Run the following commands to get it: -:::{WARNING} + + +```{warning} If your kteam-tools tree isn't clean, be sure to save your work before running the commands. They will modify the repository! Git should warn you if any destructive actions might occur. -::: +``` ```bash cd ~/canonical/kteam-tools/ @@ -60,46 +67,60 @@ Note: switching to 'cranky-tutorial'. HEAD is now at de4674e4 mainline-build/cod-update-virgin: Split script and update freedesktop repos ::: -## 2. Download Current Version of the Kernel -Next, clone the chosen kernel in its current state. +## Download current version of kernel + +You're now ready to clone the `linux-gke` kernel in its current state. + + + ```bash cd ~/canonical/kernel/ubuntu/ cranky checkout noble:linux-gke ``` + + Once this command is finished (It took ~20 minutes to complete), you should see the following directories inside the newly-created `./noble/linux-gke/` directory: -- `linux-main/`: The actual Linux kernel source -- `linux-meta/` -- `linux-signed/`: Canonical's kernel signing stuff +- `linux-main`: The actual Linux kernel source. +- `linux-meta`: Stores a set of meta-packages for the kernel. See {term}`linux-meta` for more information. +- `linux-signed`: Kernel packages that are cryptographically signed to ensure their integrity and authenticity. See {term}`linux-signed` for more information. + -## 3. Apply Updates from the Upstream Kernel -The upstream kernel will have changes that should be propagated down to this kernel. +## Apply updates from upstream kernel + +The upstream kernel will possibly have changes that should be propagated down to this kernel. + +### Fix helper scripts -### 3.1. Fix helper scripts Update the local (in-tree) helper scripts cranky uses to the latest version: + ```bash cd ~/canonical/kernel/ubuntu/noble/linux-gke/linux-main/ cranky fix ``` You should see some output showing that cranky executed several scripts. +You see observe cranky going through the updating process for various scripts in the output terminal. +### Rebase on top of updated parent -### 3.2. Rebase on top of updated parent -Because this is a derivative kernel, we need to apply updates from its parent, the generic kernel: +As `linux-gke` is a derivative kernel, we need to apply updates from its parent, the generic kernel: ```bash cranky rebase ``` -:::{tip} -For non-derivative kernels (e.g., `noble:linux`), this step is not required, and doesn't do anything. -::: + + +```{tip} +For non-derivative kernels (e.g., `noble:linux`), this step is not required. +``` + +### Fix helper scripts (again) -### 3.3. Fix helper scripts (again) Sometimes after a `cranky rebase`, the helper scripts get updated. It's good practice to always re-run: ```bash @@ -107,33 +128,91 @@ cranky fix ``` -## 4. TODO new section -### 4.1. Starting commit +## Commit and review updates + +Now that we've pulled in all the upstream changes, we are ready to review and apply the commits to the `linux-gke` kernel. + +### Add starting commit + -Run the following command: +Create a commit that signifies the start of a new release. This new commit will contain {term}`ABI` changes and any customization required by backport kernels. ```bash cranky open ``` -This command creates a commit which signifies the start of a new release. It updates `debian.gke/changelog` to reflect this. -Optionally, it may also update ABI versioning info. +You should observe something similar in the output terminal: - +```{terminal} +:input: cranky chroot create-base noble:linux-gke +:user: kernel-engineer +:host: ubuntu-machine +:dir: ~/canonical/kernel/ubuntu/noble/linux-gke/linux-main +[...] +/home/kernel-engineer/canonical/kteam-tools/cranky/cranky startnewrelease --commit +Creating new changelog set for 6.8.0-1018.22... +[cranky/master-next 7c8a6e1e1f8b] UBUNTU: Start new release + 1 file changed, 8 insertions(+) -Run `git show` and see that this new commit starts with "UBUNTU: Start new release" and shows an update to `./debian.gke/changelog`. +***** Now please inspect the commit before pushing ***** +``` + +Run `git show` to verify that this new commit starts with "UBUNTU: Start new release" and shows an update to {file}`./debian.gke/changelog`. + +```{terminal} +:input: git show +:user: kernel-engineer +:host: ubuntu-machine +:dir: ~/canonical/kernel/ubuntu/noble/linux-gke/linux-main + +Author: kernel-engineer +Date: Mon Jan 20 16:55:52 2025 +0800 + + UBUNTU: Start new release + + Ignore: yes + Signed-off-by: annecyh + +diff --git a/debian.gke/changelog b/debian.gke/changelog +[...] +``` + +### Review rebased changes + +Sometimes the rebase doesn't get all the changes. So we need to run this command to manually review any outstanding changes: -### 4.2. Review -Sometimes the rebase doesn't get all the changes. Run this command to manually review any outstanding changes: ```bash cranky review-master-changes ``` This command will output any outstanding changes in a list with their commit hashes and descriptions. Use `git show ` to view the changes. +```{terminal} +:input: cranky review-master-changes +:user: kernel-engineer +:host: ubuntu-machine +:dir: ~/canonical/kernel/ubuntu/noble/linux-gke/linux-main + +Listing changes in "debian.master/" since 9f8080a647a9e2c8c9a52b3e471b3f22d4dc0c67... + +851f47333771 UBUNTU: [Packaging] debian.master/dkms-versions -- update from kernel-versions (main/2025.01.13) +41b4e628ce36 UBUNTU: Upstream stable to v6.6.55, v6.10.14 +1de0f53d2545 UBUNTU: [Config] updateconfigs for MICROSOFT_MANA +547230d18843 UBUNTU: [Config] updateconfigs for deprecated CONFIG_Z3FOLD +1b64b00b69b7 UBUNTU: [Config] updateconfigs to set ILLEGAL_POINTER_VALUE for riscv64 +14549d19d4b5 UBUNTU: [Config] updateconfigs to select PROC_MEM_ALWAYS_FORCE +3eb67a5e5da8 UBUNTU: [Packaging] Add list of used source files to buildinfo package +3b9e978a6cb4 UBUNTU: [Packaging] Sort build dependencies alphabetically +18d768dbc001 UBUNTU: Upstream stable to v6.6.54, v6.10.13 +0861dae772cb UBUNTU: [Config] update configs for CONFIG_CRYPTO_AES_GCM_P10 +82fbe5ae5484 UBUNTU: Upstream stable to v6.6.53, v6.10.12 +``` + + + Usually these can be ignored, but there are a few instances where further investigation is necessary: - Commits with changes to `debian.master/rules.d/` @@ -142,86 +221,88 @@ Usually these can be ignored, but there are a few instances where further invest - These indicate a change in the parent kernel's configuration. - You'll need to compare this change with what appears in the derivative config (`debian.gke/`) to decide if it should be applied to this crank. -### 4.3. Link to the Launchpad Bug Tracker +### Link to Launchpad bug tracker Run the following command to link this kernel to its corresponding Launchpad bug tracker: + -:::{caution} +```{warning} Use `--dry-run` unless you are actually cranking a kernel. Otherwise, this will overwrite Launchpad and might make destructive changes! -::: +``` ```bash cranky link-tb --dry-run ``` -This updates the Launchpad tracking bug, which, among other things, will be used as an input for subsequent steps. - - - -This will create a git commit. Run `git show` to see it. It should look something like this: +```{tip} +If you are running `cranky link-tb` for the first time, you will be directed to the "Authorize application to access Launchpad on your behalf" page. Choose your preferred option before continuing. +``` -```diff -commit 4345a7fc255b03ff9072cdcec1779a9b39d7519b (HEAD -> cranky/master-next) -Author: Benjamin Wheeler -Date: Wed Jan 15 12:14:59 2025 -0500 +This step should update the Launchpad tracking bug -- which, among other things, will be used as an input for subsequent steps -- and create a git commit. - UBUNTU: link-to-tracker: update tracking bug - - BugLink: https://bugs.launchpad.net/bugs/2093494 - Properties: no-test-build - Signed-off-by: Benjamin Wheeler +But since we used the `--dry-run` option for this tutorial, no changes are made to the local tree and no commit is created. -diff --git a/debian.gke/tracking-bug b/debian.gke/tracking-bug -index 6f5c6b4a700c..b3436ee66bdd 100644 ---- a/debian.gke/tracking-bug -+++ b/debian.gke/tracking-bug -@@ -1 +1 @@ --2090338 s2024.10.28-1 -+2093494 s2024.12.02-1 -``` +```{terminal} +:input: cranky link-tb --dry-run +:user: kernel-engineer +:host: ubuntu-machine +:dir: ~/canonical/kernel/ubuntu/noble/linux-gke/linux-main -Then, click the `BugLink` URL to see the relevant Launchpad tracking bug. -A comment should appear on there that says something like: +(This is a dry-run) +LP: #2093652 (noble/linux-gke: -proposed tracker) 2025.01.13-1 +Dry Run -- no changes made ``` - summary: - noble/linux-gke: -proposed tracker - + noble/linux-gke: 6.8.0-1017.21 -proposed tracker -``` - -If you see similar results, you've successfully linked to the tracking bug. -### 4.4. Update DKMS Packages + + -`debian.master/dkms-versions` specifies dkms modules to be packaged with its kernel. +### Update DKMS packages -This command updates the package versions in `debian.gke/dkms-versions` to match the ones expected for the SRU cycle +The `debian.master/dkms-versions` file specifies dkms modules to be packaged with its kernel. This command updates the package versions in `debian.gke/dkms-versions` to match the ones expected for the SRU cycle. + ```bash cranky update-dkms-versions ``` -No changes are needed this time, so you'll see this output: -``` +Since no changes are needed at this time, you should observe the following output: + +```{terminal} +:input: cranky update-dkms-versions +:user: kernel-engineer +:host: ubuntu-machine +:dir: ~/canonical/kernel/ubuntu/noble/linux-gke/linux-main + +[...] + debian.gke/dkms-versions: No changes from kernel-versions ``` -In most cases there would be no change committed as the up-to-date versions should have been committed on the master kernel and picked-up by the derivative or backport on rebase. However, if there is any change, check that the version numbers only become higher and nothing gets dropped completely. In case anything looks suspicious, don’t hesitate to ask the team if the changes are expected. + +```{tip} +In most cases, no changes are expected as the up-to-date DKMS versions should have been committed on the generic kernel and picked up by the derivative or backport on rebase. +``` + +### Add closing commit +We will now create one final commit before preparing a release by running: -### 4.5. Closing commit -This step creates one final commit before a release is prepared. Run: ```bash cranky close ``` -This command is a shortcut for several steps: +This command is a shortcut that does the following: + + + 1. Verifies there are no changes left. 2. Inserts changes from the parent kernel into the changelog. 3. Inserts git changes into the changelog. 4. Updates the release series, author and date on the changelog, thus closing the changelog. 5. Creates a commit signifying the finished crank. -If the output went well, you should see a new commit when you run `git show`: +If successful, you should see a new commit when you run `git show`: ```diff commit 6c9a5055b22f4c30aa3ba0c9df306762edb29197 (HEAD -> cranky/master-next) Author: Benjamin Wheeler diff --git a/docs/tutorial/index.md b/docs/tutorial/index.md index f609f7d..ab9ded5 100644 --- a/docs/tutorial/index.md +++ b/docs/tutorial/index.md @@ -1,7 +1,7 @@ # Tutorials -The following tutorials will present you with different tutorials that address -key concepts, tools, processes and operations. +The following section will present you with different tutorials that address +key concepts, tools, processes, and operations relevant to the kernel team. ```{toctree} From 2ae65e6f3ef3d822561512381231ea878058af26 Mon Sep 17 00:00:00 2001 From: Benjamin Wheeler <34549440+benjamin051000@users.noreply.github.com> Date: Wed, 22 Jan 2025 07:48:07 -0500 Subject: [PATCH 17/30] Annecyh/cranky tutorial suggested edits (#53) * WIP: suggested edits for tutorial Signed-off-by: annecyh * WIP: suggested edits for tutorial Signed-off-by: annecyh * WIP Signed-off-by: annecyh * WIP Signed-off-by: annecyh * WIP Signed-off-by: annecyh --------- Signed-off-by: annecyh Signed-off-by: Benjamin Wheeler Co-authored-by: annecyh Signed-off-by: Benjamin Wheeler --- docs/tutorial/crank-your-first-kernel.md | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/docs/tutorial/crank-your-first-kernel.md b/docs/tutorial/crank-your-first-kernel.md index fa917bf..3a7df12 100644 --- a/docs/tutorial/crank-your-first-kernel.md +++ b/docs/tutorial/crank-your-first-kernel.md @@ -175,7 +175,7 @@ Date: Mon Jan 20 16:55:52 2025 +0800 UBUNTU: Start new release Ignore: yes - Signed-off-by: annecyh + Signed-off-by: kernel-engineer diff --git a/debian.gke/changelog b/debian.gke/changelog [...] @@ -302,6 +302,9 @@ This command is a shortcut that does the following: 4. Updates the release series, author and date on the changelog, thus closing the changelog. 5. Creates a commit signifying the finished crank. + + + If successful, you should see a new commit when you run `git show`: ```diff commit 6c9a5055b22f4c30aa3ba0c9df306762edb29197 (HEAD -> cranky/master-next) From 670ae36c95232801f5dbe3e7462b42d91d0425c5 Mon Sep 17 00:00:00 2001 From: Benjamin Wheeler Date: Wed, 22 Jan 2025 08:08:44 -0500 Subject: [PATCH 18/30] Remove numbered list headings Signed-off-by: Benjamin Wheeler --- docs/tutorial/crank-your-first-kernel.md | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-) diff --git a/docs/tutorial/crank-your-first-kernel.md b/docs/tutorial/crank-your-first-kernel.md index 3a7df12..cd203b3 100644 --- a/docs/tutorial/crank-your-first-kernel.md +++ b/docs/tutorial/crank-your-first-kernel.md @@ -344,7 +344,7 @@ index ba1191ce3bc2..a9ccd0b375a6 100644 linux-gke (6.8.0-1016.20) noble; urgency=medium ``` -## 5. Verify the Kernel Builds Successfully +## Verify the Kernel Builds Successfully At this point, the kernel is built and packaged. We should test that it builds successfully. ### Cloud Builder @@ -380,27 +380,27 @@ Once all the arches have `BUILD-OK`, we know the kernel built successfully. -## 6. Package the kernel for Release +## Package the kernel for Release Run the following command: ```bash cranky update-dependents ``` -### 6.1. Tag commit +### Tag commit ```bash cranky tags ``` -### 6.2. Verify Preparation +### Verify Preparation ```bash cranky verify-release-ready ``` The `tag pushed: warning` is an expected warning if you do not have commit rights. -### 6.3. Pull sources +### Pull sources ```bash cd .. @@ -409,20 +409,20 @@ cd .. cranky pull-sources noble:linux-gke --latest ``` -### 6.4. Build Sources +### Build Sources ```bash cd linux-main/ cranky build-sources ``` -## 7. Review +## Review ```bash cd .. # CWD == ~/canonical/kernel/noble/linux-gke/ cranky review *.changes ``` -## 8. Upload +## Upload From e250009e92e0fd4f9be703ed8d1b3de0a71052a8 Mon Sep 17 00:00:00 2001 From: Benjamin Wheeler Date: Wed, 22 Jan 2025 10:11:32 -0500 Subject: [PATCH 19/30] Revert to pulling master for cranky. Be more vague with elapsed time for chroot Signed-off-by: Benjamin Wheeler --- docs/tutorial/crank-your-first-kernel.md | 26 +++++------------------- 1 file changed, 5 insertions(+), 21 deletions(-) diff --git a/docs/tutorial/crank-your-first-kernel.md b/docs/tutorial/crank-your-first-kernel.md index cd203b3..64c7c6f 100644 --- a/docs/tutorial/crank-your-first-kernel.md +++ b/docs/tutorial/crank-your-first-kernel.md @@ -15,14 +15,14 @@ You will need to complete the setup according to the {doc}`cranky environment se We use [chroot](https://en.wikipedia.org/wiki/Chroot) environments to isolate different sets of tools when doing kernel cranking. `cranky` helps us set up and manage these chroot jails. -If this is your first time creating a chroot for the Noble Numbat release, you must first create the chroot base: +Creating the chroot jail is done in two steps. First, create the chroot base: ```bash cranky chroot create-base noble:linux-gke ``` -This can take up to 20 minutes to complete. +This can take several minutes to complete. If successful, you should observe the various packages being installed and set up in the terminal output. Next, create the chroot session: @@ -31,14 +31,11 @@ Next, create the chroot session: cranky chroot create-session noble:linux-gke ``` -This step uses APT to install various packages needed for the crank and takes about two minutes to complete. +This step uses APT to install various packages needed for the crank and may take several minutes to complete. ### Update kteam-tools repository Update your local clone of `kteam-tools` to the latest commit on the `master` branch: -For the scope of this tutorial, we will use a particular version of cranky. Run the following commands to get it: - - ```{warning} If your kteam-tools tree isn't clean, be sure to save your work before running the commands. @@ -48,24 +45,11 @@ Git should warn you if any destructive actions might occur. ```bash cd ~/canonical/kteam-tools/ +git switch master git pull -git checkout cranky-tutorial ``` - - -After `checkout`, you should see output similar similar to this: - -:::{terminal} - :input: git checkout cranky-tutorial - :dir: ~/canonical/kteam-tools/ - :user: user - :host: host -Note: switching to 'cranky-tutorial'. - -... -HEAD is now at de4674e4 mainline-build/cod-update-virgin: Split script and update freedesktop repos -::: +If the command completes successfully, you've got the latest version of cranky. ## Download current version of kernel From 9e92fa2e8491478cc26de97bdeed0fd2a07b404b Mon Sep 17 00:00:00 2001 From: Benjamin Wheeler Date: Wed, 22 Jan 2025 15:13:10 -0500 Subject: [PATCH 20/30] WIP More updates Signed-off-by: Benjamin Wheeler --- docs/reference/glossary.md | 3 +- docs/tutorial/crank-your-first-kernel.md | 51 ++++++------------------ 2 files changed, 13 insertions(+), 41 deletions(-) diff --git a/docs/reference/glossary.md b/docs/reference/glossary.md index 52a6f9b..86ed161 100644 --- a/docs/reference/glossary.md +++ b/docs/reference/glossary.md @@ -38,8 +38,7 @@ HWE linux-meta Refers to a set of meta-packages in Linux distributions like Ubuntu. These meta-packages do not contain the kernel binaries or source code themselves but - instead define dependencies that point - to the latest kernel packages. + instead define dependencies that point to the latest kernel packages. By installing a linux-meta package (e.g. linux-generic), users can ensure they always receive the latest version of a specific kernel series through updates. In the kernel development and {term}`SRU` life cycle, linux-meta acts as a diff --git a/docs/tutorial/crank-your-first-kernel.md b/docs/tutorial/crank-your-first-kernel.md index 64c7c6f..4679db7 100644 --- a/docs/tutorial/crank-your-first-kernel.md +++ b/docs/tutorial/crank-your-first-kernel.md @@ -17,7 +17,6 @@ We use [chroot](https://en.wikipedia.org/wiki/Chroot) environments to isolate di Creating the chroot jail is done in two steps. First, create the chroot base: - ```bash cranky chroot create-base noble:linux-gke ``` @@ -55,24 +54,16 @@ If the command completes successfully, you've got the latest version of cranky. You're now ready to clone the `linux-gke` kernel in its current state. - - ```bash -cd ~/canonical/kernel/ubuntu/ cranky checkout noble:linux-gke ``` - +If the command completes successfully, you should see the following new directory: `~/canonical/kernel/ubuntu/noble/linux-gke/`. Inside, there should be git repositories cloned: `linux-main/`, `linux-meta`, and `linux-signed`. This command can take several minutes to complete. -Once this command is finished (It took ~20 minutes to complete), you should see the following directories inside the newly-created `./noble/linux-gke/` directory: -- `linux-main`: The actual Linux kernel source. -- `linux-meta`: Stores a set of meta-packages for the kernel. See {term}`linux-meta` for more information. -- `linux-signed`: Kernel packages that are cryptographically signed to ensure their integrity and authenticity. See {term}`linux-signed` for more information. - - +For more information about these, see {term}`linux-meta`, {term}`linux-signed` in the glossary. ## Apply updates from upstream kernel @@ -97,8 +88,6 @@ As `linux-gke` is a derivative kernel, we need to apply updates from its parent, cranky rebase ``` - - ```{tip} For non-derivative kernels (e.g., `noble:linux`), this step is not required. ``` @@ -118,9 +107,6 @@ Now that we've pulled in all the upstream changes, we are ready to review and ap ### Add starting commit - - - Create a commit that signifies the start of a new release. This new commit will contain {term}`ABI` changes and any customization required by backport kernels. ```bash @@ -194,22 +180,14 @@ Listing changes in "debian.master/" since 9f8080a647a9e2c8c9a52b3e471b3f22d4dc0c 0861dae772cb UBUNTU: [Config] update configs for CONFIG_CRYPTO_AES_GCM_P10 82fbe5ae5484 UBUNTU: Upstream stable to v6.6.53, v6.10.12 ``` + - - -Usually these can be ignored, but there are a few instances where further investigation is necessary: - -- Commits with changes to `debian.master/rules.d/` - - Choose if these changes should be reflected in `debian.gke/rules.d/` -- Commits with descriptions starting with "UBUNTU: \[Config\]: ..." - - These indicate a change in the parent kernel's configuration. - - You'll need to compare this change with what appears in the derivative config (`debian.gke/`) to decide if it should be applied to this crank. +For the purposes of this tutorial, no additional commits need to be added. ### Link to Launchpad bug tracker -Run the following command to link this kernel to its corresponding Launchpad bug tracker: - - +Run the following command to update the corresponding Launchpad tracking bug to this new kernel +version being created. ```{warning} Use `--dry-run` unless you are actually cranking a kernel. Otherwise, this will overwrite Launchpad and might make destructive changes! @@ -238,13 +216,9 @@ LP: #2093652 (noble/linux-gke: -proposed tracker) 2025.01 Dry Run -- no changes made ``` - - - ### Update DKMS packages -The `debian.master/dkms-versions` file specifies dkms modules to be packaged with its kernel. This command updates the package versions in `debian.gke/dkms-versions` to match the ones expected for the SRU cycle. - +The `debian.master/dkms-versions` file specifies dkms modules to be packaged with its kernel. This command updates the package versions in `debian.gke/dkms-versions` to match the ones expected for the SRU cycle. ```bash cranky update-dkms-versions @@ -278,10 +252,9 @@ cranky close This command is a shortcut that does the following: - 1. Verifies there are no changes left. -2. Inserts changes from the parent kernel into the changelog. +2. Inserts changes from the parent kernel (`noble:linux`) into the debian changelog. 3. Inserts git changes into the changelog. 4. Updates the release series, author and date on the changelog, thus closing the changelog. 5. Creates a commit signifying the finished crank. @@ -328,7 +301,7 @@ index ba1191ce3bc2..a9ccd0b375a6 100644 linux-gke (6.8.0-1016.20) noble; urgency=medium ``` -## Verify the Kernel Builds Successfully +## Verify the kernel builds successfully At this point, the kernel is built and packaged. We should test that it builds successfully. ### Cloud Builder @@ -364,7 +337,7 @@ Once all the arches have `BUILD-OK`, we know the kernel built successfully. -## Package the kernel for Release +## Package the kernel for release Run the following command: ```bash @@ -377,7 +350,7 @@ cranky tags ``` -### Verify Preparation +### Verify preparation ```bash cranky verify-release-ready ``` @@ -393,7 +366,7 @@ cd .. cranky pull-sources noble:linux-gke --latest ``` -### Build Sources +### Build sources ```bash cd linux-main/ cranky build-sources From 821290ceb4da3002d5eaf2fd7df03067797b5ce3 Mon Sep 17 00:00:00 2001 From: Benjamin Wheeler <34549440+benjamin051000@users.noreply.github.com> Date: Thu, 23 Jan 2025 09:39:14 -0500 Subject: [PATCH 21/30] Annecyh/cranky tutorial suggested edits (#55) * WIP: suggested edits for tutorial Signed-off-by: annecyh * WIP: suggested edits for tutorial Signed-off-by: annecyh * WIP Signed-off-by: annecyh * WIP Signed-off-by: annecyh * WIP Signed-off-by: annecyh * WIP Signed-off-by: annecyh --------- Signed-off-by: annecyh Signed-off-by: Benjamin Wheeler Co-authored-by: annecyh Signed-off-by: Benjamin Wheeler --- docs/tutorial/crank-your-first-kernel.md | 79 ++++++++++++++++-------- 1 file changed, 54 insertions(+), 25 deletions(-) diff --git a/docs/tutorial/crank-your-first-kernel.md b/docs/tutorial/crank-your-first-kernel.md index 4679db7..4d69f49 100644 --- a/docs/tutorial/crank-your-first-kernel.md +++ b/docs/tutorial/crank-your-first-kernel.md @@ -1,20 +1,32 @@ # Tutorial: Crank your first kernel -Cranking an Ubuntu kernel is the process of applying patches and updates to an existing Ubuntu kernel, packaging it, and preparing it for testing. All this done using the [cranky](https://kernel.ubuntu.com/gitea/kernel/kteam-tools/src/branch/master/cranky) toolchain. +Cranking an Ubuntu kernel is the process of applying patches and updates to an existing Ubuntu kernel, packaging it, and preparing it for testing. +All this is done using the [cranky](https://kernel.ubuntu.com/gitea/kernel/kteam-tools/src/branch/master/cranky) toolchain. -In this tutorial, we will crank a 24.04 LTS (Noble Numbat) (codename `noble`) Google cloud kernel (codename `linux-gke`). Keep these codenames in mind for future commands. +In this tutorial, we will crank a 24.04 LTS (Noble Numbat) (codename `noble`) Google cloud kernel (codename `linux-gke`). + +```{tip} + Keep these codenames handy for future commands. +``` + +## Prerequisites + +You will need to complete the prerequisite setup below before proceeding with the tutorial: + +- {doc}`Set up your cranky environment ` +- Set up your [VPN connection] +- Get access to [Kernel team build resources] ## Set up and update build environment - -You will need to complete the setup according to the {doc}`cranky environment setup ` guide before continuing. +First, we will prepare our build environment and tooling needed for cranking a kernel. ### Update chroot environment -We use [chroot](https://en.wikipedia.org/wiki/Chroot) environments to isolate different sets of tools when doing kernel cranking. `cranky` helps us set up and manage these chroot jails. +We use [chroot](https://en.wikipedia.org/wiki/Chroot) environments to isolate different sets of tools when doing kernel cranking. +`cranky` helps us set up and manage these chroot jails. - Creating the chroot jail is done in two steps. First, create the chroot base: ```bash @@ -37,8 +49,7 @@ This step uses APT to install various packages needed for the crank and may take Update your local clone of `kteam-tools` to the latest commit on the `master` branch: ```{warning} -If your kteam-tools tree isn't clean, be sure to save your work before running the commands. -They will modify the repository! +If your `kteam-tools` tree isn't clean, be sure to save your work before running the commands as this step will modify the repository. Git should warn you if any destructive actions might occur. ``` @@ -48,7 +59,7 @@ git switch master git pull ``` -If the command completes successfully, you've got the latest version of cranky. +Once the command completes successfully, you've got the latest version of cranky. ## Download current version of kernel @@ -56,14 +67,17 @@ You're now ready to clone the `linux-gke` kernel in its current state. - ```bash cranky checkout noble:linux-gke ``` -If the command completes successfully, you should see the following new directory: `~/canonical/kernel/ubuntu/noble/linux-gke/`. Inside, there should be git repositories cloned: `linux-main/`, `linux-meta`, and `linux-signed`. This command can take several minutes to complete. +This command can take several minutes to complete. + +If the command completes successfully, you should see the following new directory: `~/canonical/kernel/ubuntu/noble/linux-gke/`. + +Inside, there should be three git repositories cloned: `linux-main/`, `linux-meta`, and `linux-signed`. -For more information about these, see {term}`linux-meta`, {term}`linux-signed` in the glossary. +For more information, see {term}`linux-meta` and {term}`linux-signed` in the glossary. ## Apply updates from upstream kernel @@ -77,8 +91,8 @@ Update the local (in-tree) helper scripts cranky uses to the latest version: cd ~/canonical/kernel/ubuntu/noble/linux-gke/linux-main/ cranky fix ``` -You should see some output showing that cranky executed several scripts. -You see observe cranky going through the updating process for various scripts in the output terminal. + +You should see that cranky executed several scripts and go through the updating process for various scripts in the output terminal. ### Rebase on top of updated parent @@ -94,20 +108,21 @@ For non-derivative kernels (e.g., `noble:linux`), this step is not required. ### Fix helper scripts (again) -Sometimes after a `cranky rebase`, the helper scripts get updated. It's good practice to always re-run: +Sometimes after a `cranky rebase`, the helper scripts get updated. +It's good practice to always re-run: ```bash cranky fix ``` - ## Commit and review updates Now that we've pulled in all the upstream changes, we are ready to review and apply the commits to the `linux-gke` kernel. ### Add starting commit -Create a commit that signifies the start of a new release. This new commit will contain {term}`ABI` changes and any customization required by backport kernels. +Create a commit that signifies the start of a new release. +This new commit will contain {term}`ABI` changes and any customization required by backport kernels. ```bash cranky open @@ -153,12 +168,15 @@ diff --git a/debian.gke/changelog b/debian.gke/changelog ### Review rebased changes -Sometimes the rebase doesn't get all the changes. So we need to run this command to manually review any outstanding changes: +Sometimes the rebase doesn't get all the changes. +So we need to run this command to manually review any outstanding changes: ```bash cranky review-master-changes ``` -This command will output any outstanding changes in a list with their commit hashes and descriptions. Use `git show ` to view the changes. + +This command will output any outstanding changes in a list with their commit hashes and descriptions. +Use `git show ` to view the changes. ```{terminal} :input: cranky review-master-changes @@ -186,11 +204,11 @@ For the purposes of this tutorial, no additional commits need to be added. ### Link to Launchpad bug tracker -Run the following command to update the corresponding Launchpad tracking bug to this new kernel -version being created. +Run the following command to update the corresponding Launchpad tracking bug to this new kernel version being created. ```{warning} -Use `--dry-run` unless you are actually cranking a kernel. Otherwise, this will overwrite Launchpad and might make destructive changes! +Use `--dry-run` unless you are actually cranking a kernel. +Otherwise, this will overwrite Launchpad and might make destructive changes! ``` ```bash @@ -198,7 +216,8 @@ cranky link-tb --dry-run ``` ```{tip} -If you are running `cranky link-tb` for the first time, you will be directed to the "Authorize application to access Launchpad on your behalf" page. Choose your preferred option before continuing. +If you are running `cranky link-tb` for the first time, you will be directed to the "Authorize application to access Launchpad on your behalf" page. +Choose your preferred option before continuing. ``` This step should update the Launchpad tracking bug -- which, among other things, will be used as an input for subsequent steps -- and create a git commit. @@ -218,7 +237,8 @@ Dry Run -- no changes made ### Update DKMS packages -The `debian.master/dkms-versions` file specifies dkms modules to be packaged with its kernel. This command updates the package versions in `debian.gke/dkms-versions` to match the ones expected for the SRU cycle. +The `debian.master/dkms-versions` file specifies dkms modules to be packaged with its kernel. +This command updates the package versions in `debian.gke/dkms-versions` to match the ones expected for the SRU cycle. ```bash cranky update-dkms-versions @@ -237,7 +257,6 @@ Since no changes are needed at this time, you should observe the following outpu debian.gke/dkms-versions: No changes from kernel-versions ``` - ```{tip} In most cases, no changes are expected as the up-to-date DKMS versions should have been committed on the generic kernel and picked up by the derivative or backport on rebase. ``` @@ -263,6 +282,7 @@ This command is a shortcut that does the following: If successful, you should see a new commit when you run `git show`: + ```diff commit 6c9a5055b22f4c30aa3ba0c9df306762edb29197 (HEAD -> cranky/master-next) Author: Benjamin Wheeler @@ -302,8 +322,11 @@ index ba1191ce3bc2..a9ccd0b375a6 100644 ``` ## Verify the kernel builds successfully + At this point, the kernel is built and packaged. We should test that it builds successfully. + + ### Cloud Builder @@ -395,4 +418,10 @@ cranky push-review -s 2024.10.08 kathleen ``` ## Related topics + Learn more about Ubuntu releases and derivative kernels [here]. + + + +[VPN connection]: https://canonical-kteam-docs.readthedocs-hosted.com/en/latest/how-to/new_starter/newstarter.html#setup-vpn +[Kernel team build resources]: https://canonical-kteam-docs.readthedocs-hosted.com/en/latest/how-to/new_starter/newstarter.html#build-resources From b702358f0025798a0df3b0651ffcc7289ff170cb Mon Sep 17 00:00:00 2001 From: Benjamin Wheeler Date: Thu, 23 Jan 2025 09:52:58 -0500 Subject: [PATCH 22/30] Add cbd.kernel as a git remote before pushing Signed-off-by: Benjamin Wheeler --- docs/tutorial/crank-your-first-kernel.md | 9 +-------- 1 file changed, 1 insertion(+), 8 deletions(-) diff --git a/docs/tutorial/crank-your-first-kernel.md b/docs/tutorial/crank-your-first-kernel.md index 4d69f49..1c4bc66 100644 --- a/docs/tutorial/crank-your-first-kernel.md +++ b/docs/tutorial/crank-your-first-kernel.md @@ -328,21 +328,14 @@ At this point, the kernel is built and packaged. We should test that it builds s ### Cloud Builder - Connect to the canonical VPN. Then, run: - - - - ```bash +git remote add cbd cbd.kernel:noble.git git push cbd ``` - - - The output will initially look like this: ``` Total 0 (delta 0), reused 0 (delta 0), pack-reused 0 From 4a38a25e62863a9386dcdf8fc9661b0fe1b49ca1 Mon Sep 17 00:00:00 2001 From: Benjamin Wheeler Date: Thu, 23 Jan 2025 09:55:53 -0500 Subject: [PATCH 23/30] Add info about reviewing `.debdiff` files after `cranky review *.changes`. Signed-off-by: Benjamin Wheeler --- docs/tutorial/crank-your-first-kernel.md | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/docs/tutorial/crank-your-first-kernel.md b/docs/tutorial/crank-your-first-kernel.md index 1c4bc66..f56d817 100644 --- a/docs/tutorial/crank-your-first-kernel.md +++ b/docs/tutorial/crank-your-first-kernel.md @@ -395,6 +395,10 @@ cd .. cranky review *.changes ``` +This will generate several `.debdiff` files, which show the difference between this deb package and the last one. +You should review these files in a text editor (using a command like `vim *.debdiff`) to ensure there are no unexpected changes. +However, for the sake of this tutorial, we can assume the diffs are all correct. + ## Upload From c4fd75d4fe555d34ea78a4aa5f747191a655fb1d Mon Sep 17 00:00:00 2001 From: Benjamin Wheeler Date: Thu, 23 Jan 2025 10:43:41 -0500 Subject: [PATCH 24/30] Add special case when `cranky close` fails due to annotation changes Signed-off-by: Benjamin Wheeler --- docs/tutorial/crank-your-first-kernel.md | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/docs/tutorial/crank-your-first-kernel.md b/docs/tutorial/crank-your-first-kernel.md index f56d817..d9d4a0e 100644 --- a/docs/tutorial/crank-your-first-kernel.md +++ b/docs/tutorial/crank-your-first-kernel.md @@ -281,6 +281,15 @@ This command is a shortcut that does the following: +`cranky close` should fail, stating that there were changes in `debian.gke/config/annotations`. (If it didn't, move on to the next step.) +In this particular crank, we need to manually review and commit these changes: +``` +git add debian.gke/config/annotations +git commit -m "UBUNTU [Config] gke: Update CONFIG_NVME_KEYRING" +``` +The commit message is a simple description of the updated config. + +Finally, re-run `cranky close`. If successful, you should see a new commit when you run `git show`: ```diff From 7dda2d1936ea0a05b8b3a1d04e3b9f3f2397d47f Mon Sep 17 00:00:00 2001 From: annecyh Date: Fri, 24 Jan 2025 13:42:43 +0800 Subject: [PATCH 25/30] docs(tutorial): Updated commands and descriptions Signed-off-by: annecyh --- .../cranking/set-up-cranky-environment.md | 2 + docs/tutorial/crank-your-first-kernel.md | 153 +++++++++++++----- 2 files changed, 118 insertions(+), 37 deletions(-) diff --git a/docs/how-to/cranking/set-up-cranky-environment.md b/docs/how-to/cranking/set-up-cranky-environment.md index 82560fb..9d44f4e 100644 --- a/docs/how-to/cranking/set-up-cranky-environment.md +++ b/docs/how-to/cranking/set-up-cranky-environment.md @@ -26,7 +26,9 @@ sudo apt install \ kernel-wedge \ libncurses5-dev \ makedumpfile \ + python3-git \ python3-launchpadlib \ + python3-requests \ python3-ruamel.yaml \ schroot \ sharutils \ diff --git a/docs/tutorial/crank-your-first-kernel.md b/docs/tutorial/crank-your-first-kernel.md index d9d4a0e..58e20e8 100644 --- a/docs/tutorial/crank-your-first-kernel.md +++ b/docs/tutorial/crank-your-first-kernel.md @@ -22,7 +22,7 @@ You will need to complete the prerequisite setup below before proceeding with th First, we will prepare our build environment and tooling needed for cranking a kernel. -### Update chroot environment +### Update chroot environment - `cranky chroot` We use [chroot](https://en.wikipedia.org/wiki/Chroot) environments to isolate different sets of tools when doing kernel cranking. `cranky` helps us set up and manage these chroot jails. @@ -61,7 +61,7 @@ git pull Once the command completes successfully, you've got the latest version of cranky. -## Download current version of kernel +## Download current version of kernel - `cranky checkout` You're now ready to clone the `linux-gke` kernel in its current state. @@ -83,7 +83,7 @@ For more information, see {term}`linux-meta` and {term}`linux-signed` in the glo The upstream kernel will possibly have changes that should be propagated down to this kernel. -### Fix helper scripts +### Fix helper scripts - `cranky fix` Update the local (in-tree) helper scripts cranky uses to the latest version: @@ -94,7 +94,7 @@ cranky fix You should see that cranky executed several scripts and go through the updating process for various scripts in the output terminal. -### Rebase on top of updated parent +### Rebase on top of updated parent - `cranky rebase` As `linux-gke` is a derivative kernel, we need to apply updates from its parent, the generic kernel: @@ -106,7 +106,7 @@ cranky rebase For non-derivative kernels (e.g., `noble:linux`), this step is not required. ``` -### Fix helper scripts (again) +### Fix helper scripts (again) - `cranky fix` Sometimes after a `cranky rebase`, the helper scripts get updated. It's good practice to always re-run: @@ -119,7 +119,7 @@ cranky fix Now that we've pulled in all the upstream changes, we are ready to review and apply the commits to the `linux-gke` kernel. -### Add starting commit +### Add starting commit - `cranky open` Create a commit that signifies the start of a new release. This new commit will contain {term}`ABI` changes and any customization required by backport kernels. @@ -166,7 +166,7 @@ diff --git a/debian.gke/changelog b/debian.gke/changelog [...] ``` -### Review rebased changes +### Review rebased changes - `cranky review-master-changes` Sometimes the rebase doesn't get all the changes. So we need to run this command to manually review any outstanding changes: @@ -200,9 +200,10 @@ Listing changes in "debian.master/" since 9f8080a647a9e2c8c9a52b3e471b3f22d4dc0c ``` -For the purposes of this tutorial, no additional commits need to be added. +Since there are commits that have kernel config changes (e.g. commits with the "UBUNTU: [Config]" prefix), this will require an additional commit from our side. +We will address this in the `cranky close` step a little later. -### Link to Launchpad bug tracker +### Link to Launchpad bug tracker - `cranky link-tb` Run the following command to update the corresponding Launchpad tracking bug to this new kernel version being created. @@ -235,7 +236,7 @@ LP: #2093652 (noble/linux-gke: -proposed tracker) 2025.01 Dry Run -- no changes made ``` -### Update DKMS packages +### Update DKMS packages - `cranky update-dkms-versions` The `debian.master/dkms-versions` file specifies dkms modules to be packaged with its kernel. This command updates the package versions in `debian.gke/dkms-versions` to match the ones expected for the SRU cycle. @@ -281,25 +282,61 @@ This command is a shortcut that does the following: -`cranky close` should fail, stating that there were changes in `debian.gke/config/annotations`. (If it didn't, move on to the next step.) -In this particular crank, we need to manually review and commit these changes: +Since there are config-related changes, `cranky close` is expected fail at this point. +You should observe a similar error in the terminal output stating that there were changes in `debian.gke/config/annotations`. + +```{terminal} +:input: cranky close +:user: kernel-engineer +:host: ubuntu-machine +:dir: ~/canonical/kernel/ubuntu/noble/linux-gke/linux-main + +[...] + +check-config: loading annotations from debian.gke/config/annotations +check-config: CONFIG_NVME_KEYRING changed from m to y: policy<{'amd64': 'm', 'arm64': 'm', 'armhf': 'm', 'ppc64el': 'm', 'riscv64': 'm', 's390x': 'm'}>) +check-config: 1 config options have been changed, review them with `git diff` +ERROR: 2 config-check failures detected + +Importing all configurations ... + +* Import configs for amd64-gke ... +* Import configs for arm64-gke ... +make: *** [debian/rules.d/1-maintainer.mk:31: updateconfigs] Error 1 +Script failed +ERROR: Command failed: chroot (args=['run', '--', 'fakeroot', 'debian/rules', 'clean', 'updateconfigs']) +ERROR: Command failed: fdr (args=['clean', 'updateconfigs']) +ERROR: Command failed: close (args=[]) ``` + +```{note} +If there were no unreviewed changes, skip ahead to the next section. +``` + +We will need to manually review and commit these changes with a descriptive +message of the updated config: + +```bash git add debian.gke/config/annotations -git commit -m "UBUNTU [Config] gke: Update CONFIG_NVME_KEYRING" +git commit -m "UBUNTU [Config] gke: Update CONFIG_NVME_KEYRING" -s ``` -The commit message is a simple description of the updated config. Finally, re-run `cranky close`. + +```bash +cranky close +``` + If successful, you should see a new commit when you run `git show`: ```diff commit 6c9a5055b22f4c30aa3ba0c9df306762edb29197 (HEAD -> cranky/master-next) -Author: Benjamin Wheeler +Author: Kernel Engineer Date: Wed Jan 15 12:42:31 2025 -0500 UBUNTU: Ubuntu-gke-6.8.0-1017.21 - Signed-off-by: Benjamin Wheeler + Signed-off-by: Kernel Engineer diff --git a/debian.gke/changelog b/debian.gke/changelog index ba1191ce3bc2..a9ccd0b375a6 100644 @@ -314,7 +351,7 @@ index ba1191ce3bc2..a9ccd0b375a6 100644 - CHANGELOG: Use the insertchanges target to create the final log. + * noble/linux-gke: 6.8.0-1017.21 -proposed tracker (LP: #2093494) -- -- Benjamin Wheeler Wed, 15 Jan 2025 12:04:33 -0500 +- -- Kernel Engineer Wed, 15 Jan 2025 12:04:33 -0500 + [ Ubuntu: 6.8.0-52.53 ] + + * noble/linux: 6.8.0-52.53 -proposed tracker (LP: #2093521) @@ -325,20 +362,17 @@ index ba1191ce3bc2..a9ccd0b375a6 100644 + * CVE-2024-53103 + - hv_sock: Initializing vsk->trans to NULL to prevent a dangling pointer + -+ -- Benjamin Wheeler Wed, 15 Jan 2025 12:42:30 -0500 ++ -- Kernel Engineer Wed, 15 Jan 2025 12:42:30 -0500 linux-gke (6.8.0-1016.20) noble; urgency=medium ``` ## Verify the kernel builds successfully -At this point, the kernel is built and packaged. We should test that it builds successfully. - - +Now that our local `linux-gke` kernel source tree has been updated, we should test that it builds successfully. +We will be using our cloud builder system (CBD) for this purpose. -### Cloud Builder - -Connect to the canonical VPN. Then, run: +Connect to the Canonical VPN and run: ```bash git remote add cbd cbd.kernel:noble.git @@ -346,41 +380,86 @@ git push cbd ``` The output will initially look like this: -``` + +```{code-block} +:emphasize-lines: 4 + Total 0 (delta 0), reused 0 (delta 0), pack-reused 0 remote: *** kernel-cbd ********************************************************* remote: * Queueing builds (your 'cranky/master-next'); ok to interrupt -remote: * For results: ssh cbd ls benjaminwheeler-noble-6c9a5055b22f-szkb +remote: * For results: ssh cbd ls kernelengineer-noble-6c9a5055b22f-szkb ``` -At this point you can interrupt the command with Ctrl-C and periodically check the build status with the ssh command it printed. +At this point you can interrupt the command with {kbd}`Ctrl-C` and periodically check the build status with the `ssh` command it printed. For example: -When running it, you'll see a few files either called `BUILDING` or `BUILD-OK`. +```bash +ssh cbd ls kernelengineer-noble-6c9a5055b22f-szkb +``` -Once all the arches have `BUILD-OK`, we know the kernel built successfully. +You should see the build progress for each architecture with either `BUILDING` or `BUILD-OK` status. +```{terminal} +:input: ssh cbd ls kernelengineer-noble-6c9a5055b22f-szkb +:user: kernel-engineer +:host: ubuntu-machine +:dir: ~/canonical/kernel/ubuntu/noble/linux-gke/linux-main + +2025-01-24 04:38:50 0 amd64/BUILDING +2025-01-24 04:38:51 0 arm64/BUILDING +``` + +For the `linux-gke` kernel, this shows us the builds are still in progress for the amd64 and arm64 architectures. - +Once all the architectures return the `BUILD-OK` status, we know the kernel built successfully. + + + +## Package the kernel for release - `cranky update-dependents` + +To update all dependent packages in this `linux-gke` tree set (e.g. `linux-meta`, `linux-signed`, `linux-lrm`), run: -## Package the kernel for release -Run the following command: - ```bash cranky update-dependents ``` -### Tag commit +If successful, you should see the "SUCCESS: update complete" message in the output. + +### Tag commit - `cranky tags` + +The last step is to tag your commit with the appropriate version tag. + +```{caution} +Once a tag is pushed to the repository it cannot be modified. +So be sure to tag your commit only after verifying that your kernel source tree has been updated correctly and builds successfully. +``` + ```bash cranky tags ``` - -### Verify preparation +You see should something similar returned in the terminal output: + +```{terminal} +:input: cranky tags +:user: kernel-engineer +:host: ubuntu-machine +:dir: ~/canonical/kernel/ubuntu/noble/linux-gke/linux-main + +Tagging with: + git tag -f -s -m Ubuntu-gke-6.8.0-1018.22 Ubuntu-gke-6.8.0-1018.22 +``` + +### Verify preparation - `cranky verify-release-ready` + +Perform one last sanity check on the git trees to screen out any issues with the kernel preparation by running: + ```bash cranky verify-release-ready ``` -The `tag pushed: warning` is an expected warning if you do not have commit rights. +```{note} +The "tag pushed: warning" is an expected warning if you do not have commit rights. +``` ### Pull sources From 27f7ee3ba432e09c396239faf0994520edfefe65 Mon Sep 17 00:00:00 2001 From: annecyh Date: Fri, 24 Jan 2025 14:59:22 +0800 Subject: [PATCH 26/30] docs(conf): ignore anchor check for kteam-docs Signed-off-by: annecyh --- docs/conf.py | 1 + docs/tutorial/crank-your-first-kernel.md | 2 +- 2 files changed, 2 insertions(+), 1 deletion(-) diff --git a/docs/conf.py b/docs/conf.py index 0e2f35a..e5a82ba 100644 --- a/docs/conf.py +++ b/docs/conf.py @@ -201,6 +201,7 @@ r"https://github\.com/.*", r"https://ubuntu.com/about/release-cycle", r"https://snapcraft.io/docs/", + r"https://canonical-kteam-docs.readthedocs-hosted.com/en/latest/how-to/new_starter/newstarter.html", ] diff --git a/docs/tutorial/crank-your-first-kernel.md b/docs/tutorial/crank-your-first-kernel.md index 58e20e8..e39798f 100644 --- a/docs/tutorial/crank-your-first-kernel.md +++ b/docs/tutorial/crank-your-first-kernel.md @@ -262,7 +262,7 @@ debian.gke/dkms-versions: No changes from kernel-versions In most cases, no changes are expected as the up-to-date DKMS versions should have been committed on the generic kernel and picked up by the derivative or backport on rebase. ``` -### Add closing commit +### Add closing commit - `cranky close` We will now create one final commit before preparing a release by running: From f15683a15a494d9f3cd909d8eb8fdccb26d9818a Mon Sep 17 00:00:00 2001 From: Benjamin Wheeler Date: Fri, 24 Jan 2025 10:12:36 -0500 Subject: [PATCH 27/30] Remove unnecessary explanation about `cranky close` Signed-off-by: Benjamin Wheeler --- docs/tutorial/crank-your-first-kernel.md | 13 ++----------- 1 file changed, 2 insertions(+), 11 deletions(-) diff --git a/docs/tutorial/crank-your-first-kernel.md b/docs/tutorial/crank-your-first-kernel.md index e39798f..ede9796 100644 --- a/docs/tutorial/crank-your-first-kernel.md +++ b/docs/tutorial/crank-your-first-kernel.md @@ -270,17 +270,8 @@ We will now create one final commit before preparing a release by running: cranky close ``` -This command is a shortcut that does the following: - - -1. Verifies there are no changes left. -2. Inserts changes from the parent kernel (`noble:linux`) into the debian changelog. -3. Inserts git changes into the changelog. -4. Updates the release series, author and date on the changelog, thus closing the changelog. -5. Creates a commit signifying the finished crank. - - - +This command, among several other things, closes the changelog, verifies changes are handled properly, +and makes a new commit signifying the cranked kernel. Since there are config-related changes, `cranky close` is expected fail at this point. You should observe a similar error in the terminal output stating that there were changes in `debian.gke/config/annotations`. From 0b4634deb9138c7b688c1f10084090886315fe9c Mon Sep 17 00:00:00 2001 From: AnneCYH Date: Tue, 11 Feb 2025 10:49:47 +0800 Subject: [PATCH 28/30] Added Ignore yes Signed-off-by: AnneCYH --- docs/tutorial/crank-your-first-kernel.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/tutorial/crank-your-first-kernel.md b/docs/tutorial/crank-your-first-kernel.md index ede9796..c060983 100644 --- a/docs/tutorial/crank-your-first-kernel.md +++ b/docs/tutorial/crank-your-first-kernel.md @@ -309,7 +309,7 @@ message of the updated config: ```bash git add debian.gke/config/annotations -git commit -m "UBUNTU [Config] gke: Update CONFIG_NVME_KEYRING" -s +git commit -m "UBUNTU [Config] gke: Update CONFIG_NVME_KEYRING" -m "Ignore: yes" -s ``` Finally, re-run `cranky close`. From dc34ff30b5676eb6cec3399d733f67a79abc2b8c Mon Sep 17 00:00:00 2001 From: AnneCYH Date: Tue, 11 Feb 2025 10:52:02 +0800 Subject: [PATCH 29/30] Fixed terminal command for cranky open Signed-off-by: AnneCYH --- docs/tutorial/crank-your-first-kernel.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/tutorial/crank-your-first-kernel.md b/docs/tutorial/crank-your-first-kernel.md index c060983..e375004 100644 --- a/docs/tutorial/crank-your-first-kernel.md +++ b/docs/tutorial/crank-your-first-kernel.md @@ -131,7 +131,7 @@ cranky open You should observe something similar in the output terminal: ```{terminal} -:input: cranky chroot create-base noble:linux-gke +:input: cranky open :user: kernel-engineer :host: ubuntu-machine :dir: ~/canonical/kernel/ubuntu/noble/linux-gke/linux-main From 7ba2a48ed1fd5ddbaf1e3adaa2e0dc74d88ffc65 Mon Sep 17 00:00:00 2001 From: annecyh Date: Wed, 12 Feb 2025 16:40:20 +0800 Subject: [PATCH 30/30] update on top of ben's tutorial Signed-off-by: annecyh --- docs/tutorial/crank-your-first-kernel.md | 199 ++++++++++++++--------- 1 file changed, 125 insertions(+), 74 deletions(-) diff --git a/docs/tutorial/crank-your-first-kernel.md b/docs/tutorial/crank-your-first-kernel.md index e375004..6bd441f 100644 --- a/docs/tutorial/crank-your-first-kernel.md +++ b/docs/tutorial/crank-your-first-kernel.md @@ -3,11 +3,10 @@ Cranking an Ubuntu kernel is the process of applying patches and updates to an existing Ubuntu kernel, packaging it, and preparing it for testing. All this is done using the [cranky](https://kernel.ubuntu.com/gitea/kernel/kteam-tools/src/branch/master/cranky) toolchain. - -In this tutorial, we will crank a 24.04 LTS (Noble Numbat) (codename `noble`) Google cloud kernel (codename `linux-gke`). +In this tutorial, we will crank a 24.04 LTS (Noble Numbat) (codename `noble`) Google cloud kernel (codename `linux-gke`) from the "s2024.12.02" cycle. ```{tip} - Keep these codenames handy for future commands. + Keep these codenames and cycle name handy for future commands. ``` ## Prerequisites @@ -22,6 +21,23 @@ You will need to complete the prerequisite setup below before proceeding with th First, we will prepare our build environment and tooling needed for cranking a kernel. +### Update kteam-tools repository + +Update your local clone of `kteam-tools` to the latest commit on the `master` branch: + +```{warning} +If your `kteam-tools` tree isn't clean, be sure to save your work before running the commands as this step will modify the repository. +Git should warn you if any destructive actions might occur. +``` + +```bash +cd ~/canonical/kteam-tools/ +git switch master +git pull +``` + +Once the command completes successfully, you've got the latest version of cranky. + ### Update chroot environment - `cranky chroot` We use [chroot](https://en.wikipedia.org/wiki/Chroot) environments to isolate different sets of tools when doing kernel cranking. @@ -44,38 +60,19 @@ cranky chroot create-session noble:linux-gke This step uses APT to install various packages needed for the crank and may take several minutes to complete. -### Update kteam-tools repository - -Update your local clone of `kteam-tools` to the latest commit on the `master` branch: - -```{warning} -If your `kteam-tools` tree isn't clean, be sure to save your work before running the commands as this step will modify the repository. -Git should warn you if any destructive actions might occur. -``` - -```bash -cd ~/canonical/kteam-tools/ -git switch master -git pull -``` - -Once the command completes successfully, you've got the latest version of cranky. - -## Download current version of kernel - `cranky checkout` +## Download kernel sources - `cranky checkout` -You're now ready to clone the `linux-gke` kernel in its current state. - - +You're now ready to clone the `linux-gke` kernel for the given cycle (e.g. "s2024.12.02"). ```bash -cranky checkout noble:linux-gke +cranky checkout noble:linux-gke --cycle s2024.12.02 ``` This command can take several minutes to complete. If the command completes successfully, you should see the following new directory: `~/canonical/kernel/ubuntu/noble/linux-gke/`. -Inside, there should be three git repositories cloned: `linux-main/`, `linux-meta`, and `linux-signed`. +Inside, there should be three git repositories cloned: `linux-main`, `linux-meta`, and `linux-signed`. For more information, see {term}`linux-meta` and {term}`linux-signed` in the glossary. @@ -94,18 +91,45 @@ cranky fix You should see that cranky executed several scripts and go through the updating process for various scripts in the output terminal. -### Rebase on top of updated parent - `cranky rebase` +### Rebase on top of parent kernel - `cranky rebase` + +As `linux-gke` is a derivative kernel, we need to apply updates from its parent, the generic kernel in Noble. -As `linux-gke` is a derivative kernel, we need to apply updates from its parent, the generic kernel: +First, add the parent repository as a remote: ```bash -cranky rebase +git remote add parent "git://git.launchpad.net/~ubuntu-kernel/ubuntu/+source/linux/+git/noble" +``` + +Pull the tag for the kernel version we want to rebase on: + +```bash +git fetch parent --no-tags tag Ubuntu-6.8.0-53.55 +``` + +Then rebase to inherit the changes from a specific version of the parent kernel: + +```bash +cranky rebase -l Ubuntu-6.8.0-53.55 ``` ```{tip} For non-derivative kernels (e.g., `noble:linux`), this step is not required. ``` +You should observe something similar in the output terminal: + +```{terminal} +:input: cranky rebase -l Ubuntu-6.8.0-53.55 +:user: kernel-engineer +:host: ubuntu-machine +:dir: ~/canonical/kernel/ubuntu/noble/linux-gke/linux-main (cranky/master-next-s2024.12.02) + +Rebase still needed between Ubuntu-6.8.0-52.53 and Ubuntu-6.8.0-53.55. +II: record=/home/annecyh/canonical/kernel/ubuntu/noble/linux-gke/linux-main/.git/REBASE-SELECTOR +Successfully rebased and updated refs/heads/cranky/master-next-s2024.12.02. +``` + ### Fix helper scripts (again) - `cranky fix` Sometimes after a `cranky rebase`, the helper scripts get updated. @@ -134,12 +158,12 @@ You should observe something similar in the output terminal: :input: cranky open :user: kernel-engineer :host: ubuntu-machine -:dir: ~/canonical/kernel/ubuntu/noble/linux-gke/linux-main +:dir: ~/canonical/kernel/ubuntu/noble/linux-gke/linux-main (cranky/master-next-s2024.12.02) [...] /home/kernel-engineer/canonical/kteam-tools/cranky/cranky startnewrelease --commit Creating new changelog set for 6.8.0-1018.22... -[cranky/master-next 7c8a6e1e1f8b] UBUNTU: Start new release +[cranky/master-next-s2024.12.02 911b0ad106e9] UBUNTU: Start new release 1 file changed, 8 insertions(+) @@ -152,7 +176,7 @@ Run `git show` to verify that this new commit starts with "UBUNTU: Start new rel :input: git show :user: kernel-engineer :host: ubuntu-machine -:dir: ~/canonical/kernel/ubuntu/noble/linux-gke/linux-main +:dir: ~/canonical/kernel/ubuntu/noble/linux-gke/linux-main (cranky/master-next-s2024.12.02) Author: kernel-engineer Date: Mon Jan 20 16:55:52 2025 +0800 @@ -182,21 +206,22 @@ Use `git show ` to view the changes. :input: cranky review-master-changes :user: kernel-engineer :host: ubuntu-machine -:dir: ~/canonical/kernel/ubuntu/noble/linux-gke/linux-main +:dir: ~/canonical/kernel/ubuntu/noble/linux-gke/linux-main (cranky/master-next-s2024.12.02) Listing changes in "debian.master/" since 9f8080a647a9e2c8c9a52b3e471b3f22d4dc0c67... -851f47333771 UBUNTU: [Packaging] debian.master/dkms-versions -- update from kernel-versions (main/2025.01.13) -41b4e628ce36 UBUNTU: Upstream stable to v6.6.55, v6.10.14 -1de0f53d2545 UBUNTU: [Config] updateconfigs for MICROSOFT_MANA -547230d18843 UBUNTU: [Config] updateconfigs for deprecated CONFIG_Z3FOLD -1b64b00b69b7 UBUNTU: [Config] updateconfigs to set ILLEGAL_POINTER_VALUE for riscv64 -14549d19d4b5 UBUNTU: [Config] updateconfigs to select PROC_MEM_ALWAYS_FORCE -3eb67a5e5da8 UBUNTU: [Packaging] Add list of used source files to buildinfo package -3b9e978a6cb4 UBUNTU: [Packaging] Sort build dependencies alphabetically -18d768dbc001 UBUNTU: Upstream stable to v6.6.54, v6.10.13 -0861dae772cb UBUNTU: [Config] update configs for CONFIG_CRYPTO_AES_GCM_P10 -82fbe5ae5484 UBUNTU: Upstream stable to v6.6.53, v6.10.12 +f31315007228 UBUNTU: [Packaging] debian.master/dkms-versions -- update from kernel-versions (main/2025.01.13) +3a8a3b4039e7 UBUNTU: [Packaging] do not attempt to generate BTF header on armhf +59e4c08ecd84 UBUNTU: Upstream stable to v6.6.55, v6.10.14 +fadfbbcc2798 UBUNTU: [Config] updateconfigs for MICROSOFT_MANA +b5bd2549a589 UBUNTU: [Config] updateconfigs for deprecated CONFIG_Z3FOLD +1878fd662392 UBUNTU: [Config] updateconfigs to set ILLEGAL_POINTER_VALUE for riscv64 +267522e21042 UBUNTU: [Config] updateconfigs to select PROC_MEM_ALWAYS_FORCE +366c11c3240e UBUNTU: [Packaging] Add list of used source files to buildinfo package +6b86533cc3d1 UBUNTU: [Packaging] Sort build dependencies alphabetically +52051394e234 UBUNTU: Upstream stable to v6.6.54, v6.10.13 +763dd57e5520 UBUNTU: [Config] update configs for CONFIG_CRYPTO_AES_GCM_P10 +db4045cf4e9a UBUNTU: Upstream stable to v6.6.53, v6.10.12 ``` @@ -229,10 +254,10 @@ But since we used the `--dry-run` option for this tutorial, no changes are made :input: cranky link-tb --dry-run :user: kernel-engineer :host: ubuntu-machine -:dir: ~/canonical/kernel/ubuntu/noble/linux-gke/linux-main +:dir: ~/canonical/kernel/ubuntu/noble/linux-gke/linux-main (cranky/master-next-s2024.12.02) (This is a dry-run) -LP: #2093652 (noble/linux-gke: -proposed tracker) 2025.01.13-1 +LP: #2097956 (noble/linux-gke: -proposed tracker) s2025.01.13-1 Dry Run -- no changes made ``` @@ -251,7 +276,7 @@ Since no changes are needed at this time, you should observe the following outpu :input: cranky update-dkms-versions :user: kernel-engineer :host: ubuntu-machine -:dir: ~/canonical/kernel/ubuntu/noble/linux-gke/linux-main +:dir: ~/canonical/kernel/ubuntu/noble/linux-gke/linux-main (cranky/master-next-s2024.12.02) [...] @@ -277,13 +302,14 @@ Since there are config-related changes, `cranky close` is expected fail at this You should observe a similar error in the terminal output stating that there were changes in `debian.gke/config/annotations`. ```{terminal} +:dir: ~/canonical/kernel/ubuntu/noble/linux-gke/linux-main (cranky/master-next-s2024.12.02) :input: cranky close :user: kernel-engineer :host: ubuntu-machine -:dir: ~/canonical/kernel/ubuntu/noble/linux-gke/linux-main [...] +* Run config-check for arm64-gke ... check-config: loading annotations from debian.gke/config/annotations check-config: CONFIG_NVME_KEYRING changed from m to y: policy<{'amd64': 'm', 'arm64': 'm', 'armhf': 'm', 'ppc64el': 'm', 'riscv64': 'm', 's390x': 'm'}>) check-config: 1 config options have been changed, review them with `git diff` @@ -309,7 +335,7 @@ message of the updated config: ```bash git add debian.gke/config/annotations -git commit -m "UBUNTU [Config] gke: Update CONFIG_NVME_KEYRING" -m "Ignore: yes" -s +git commit -m "UBUNTU [Config] gke: Update configs from 6.8.0-53.55" -m "Ignore: yes" -s ``` Finally, re-run `cranky close`. @@ -321,47 +347,46 @@ cranky close If successful, you should see a new commit when you run `git show`: ```diff -commit 6c9a5055b22f4c30aa3ba0c9df306762edb29197 (HEAD -> cranky/master-next) +commit 6c9a5055b22f4c30aa3ba0c9df306762edb29197 (HEAD -> cranky/master-next-s2024.12.02) Author: Kernel Engineer -Date: Wed Jan 15 12:42:31 2025 -0500 +Date: Wed Feb 12 13:49:40 2025 +0800 - UBUNTU: Ubuntu-gke-6.8.0-1017.21 + UBUNTU: Ubuntu-gke-6.8.0-1018.22 Signed-off-by: Kernel Engineer diff --git a/debian.gke/changelog b/debian.gke/changelog -index ba1191ce3bc2..a9ccd0b375a6 100644 +index fd056a9e15cc..fd8c8d9cd8fb 100644 --- a/debian.gke/changelog +++ b/debian.gke/changelog -@@ -1,10 +1,18 @@ --linux-gke (6.8.0-1017.21) UNRELEASED; urgency=medium -+linux-gke (6.8.0-1017.21) noble; urgency=medium +@@ -1,10 +1,1229 @@ +-linux-gke (6.8.0-1018.22) UNRELEASED; urgency=medium ++linux-gke (6.8.0-1018.22) noble; urgency=medium - CHANGELOG: Do not edit directly. Autogenerated at release. - CHANGELOG: Use the printchanges target to see the curent changes. - CHANGELOG: Use the insertchanges target to create the final log. -+ * noble/linux-gke: 6.8.0-1017.21 -proposed tracker (LP: #2093494) ++ [ Ubuntu: 6.8.0-53.55 ] -- -- Kernel Engineer Wed, 15 Jan 2025 12:04:33 -0500 -+ [ Ubuntu: 6.8.0-52.53 ] -+ -+ * noble/linux: 6.8.0-52.53 -proposed tracker (LP: #2093521) -+ * CVE-2024-53164 -+ - net: sched: fix ordering of qlen adjustment -+ * CVE-2024-53141 -+ - netfilter: ipset: add missing range check in bitmap_ip_uadt -+ * CVE-2024-53103 -+ - hv_sock: Initializing vsk->trans to NULL to prevent a dangling pointer -+ -+ -- Kernel Engineer Wed, 15 Jan 2025 12:42:30 -0500 +- -- Kernel Engineer Wed, 12 Feb 2025 12:43:08 +0800 ++ * noble/linux: 6.8.0-53.55 -proposed tracker (LP: #2093677) ++ * Packaging resync (LP: #1786013) ++ - [Packaging] debian.master/dkms-versions -- update from kernel-versions ++ (main/2025.01.13) +[...] + ++ -- Kernel Engineer Wed, 12 Feb 2025 13:49:40 +0800 - linux-gke (6.8.0-1016.20) noble; urgency=medium + linux-gke (6.8.0-1017.21) noble; urgency=medium ``` ## Verify the kernel builds successfully Now that our local `linux-gke` kernel source tree has been updated, we should test that it builds successfully. -We will be using our cloud builder system (CBD) for this purpose. + +### Build kernel in CBD + +For Canonical users, we can use the cloud builder system (CBD) for this purpose. Connect to the Canonical VPN and run: @@ -393,7 +418,7 @@ You should see the build progress for each architecture with either `BUILDING` o :input: ssh cbd ls kernelengineer-noble-6c9a5055b22f-szkb :user: kernel-engineer :host: ubuntu-machine -:dir: ~/canonical/kernel/ubuntu/noble/linux-gke/linux-main +:dir: ~/canonical/kernel/ubuntu/noble/linux-gke/linux-main (cranky/master-next-s2024.12.02) 2025-01-24 04:38:50 0 amd64/BUILDING 2025-01-24 04:38:51 0 arm64/BUILDING @@ -403,7 +428,32 @@ For the `linux-gke` kernel, this shows us the builds are still in progress for t Once all the architectures return the `BUILD-OK` status, we know the kernel built successfully. - +### Build kernel locally + +Since we have the chroot environment set up already, we can choose to build the kernel locally. + +Clean up any temporary build files and reset the source tree: + +```bash +cranky chroot run noble:linux-gke -- fakeroot debian/rules clean +``` + +Then build and compile the different kernel packages: + +```bash +cranky chroot run noble:linux-gke -- fakeroot debian/rules binary-headers binary-generic binary-perarch +``` + +If the build is successful, you should find several .deb binary package files in the directory above the build root directory. + +```{terminal} +:input: ls ../ +:user: kernel-engineer +:host: ubuntu-machine +:dir: ~/canonical/kernel/ubuntu/noble/linux-gke/linux-main (cranky/master-next-s2024.12.02) + +TBD +``` ## Package the kernel for release - `cranky update-dependents` @@ -434,7 +484,7 @@ You see should something similar returned in the terminal output: :input: cranky tags :user: kernel-engineer :host: ubuntu-machine -:dir: ~/canonical/kernel/ubuntu/noble/linux-gke/linux-main +:dir: ~/canonical/kernel/ubuntu/noble/linux-gke/linux-main (cranky/master-next-s2024.12.02) Tagging with: git tag -f -s -m Ubuntu-gke-6.8.0-1018.22 Ubuntu-gke-6.8.0-1018.22 @@ -501,3 +551,4 @@ Learn more about Ubuntu releases and derivative kernels [here]. [VPN connection]: https://canonical-kteam-docs.readthedocs-hosted.com/en/latest/how-to/new_starter/newstarter.html#setup-vpn [Kernel team build resources]: https://canonical-kteam-docs.readthedocs-hosted.com/en/latest/how-to/new_starter/newstarter.html#build-resources +[Kernel SRU dashboard]: https://kernel.ubuntu.com/reports/kernel-stable-board/ \ No newline at end of file