Skip to content

Add support for static and transparent hugepages - #4864

Merged
piyush-jena merged 2 commits into
bottlerocket-os:developfrom
piyush-jena:hugepages-final
Jul 22, 2026
Merged

Add support for static and transparent hugepages#4864
piyush-jena merged 2 commits into
bottlerocket-os:developfrom
piyush-jena:hugepages-final

Conversation

@piyush-jena

@piyush-jena piyush-jena commented Jun 22, 2026

Copy link
Copy Markdown
Contributor

Closes #

Description of changes:

  • Add support for hugepages

Testing done:
Set the following settings:

[ssm-user@control]$ apiclient apply  <<EOF
[settings.kernel.hugepages."2Mi"]
> count = 200
> [settings.kernel.hugepages."1Gi"]
> count = 2
> EOF
[ssm-user@control]$ apiclient get settings.kernel.hugepages
{
  "settings": {
    "kernel": {
      "hugepages": {
        "1Gi": {
          "count": 2
        },
        "2Mi": {
          "count": 200
        }
      }
    }
  }
}
[ssm-user@control]$ cat /proc/meminfo | grep ^Huge
HugePages_Total:       0
HugePages_Free:        0
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB
Hugetlb:               0 kB

The hugepages are not allocated yet because we have set it up to allocate only at boot.

Post reboot:

  1. Meminfo output:
bash-5.2# cat /proc/meminfo | grep ^Huge
HugePages_Total:     200
HugePages_Free:      200
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB
Hugetlb:         2506752 kB

200 hugepages of default size has been allocated. Hugetlb size = 2048200 + 10485762 = 2506752 kB

  1. Describe node output
    $ kubectl describe node --kubeconfig bottlerocket-cluster-134.kubeconfig
    [Output trimmed for brevity]
Capacity:
  cpu:                128
  ephemeral-storage:  20100288Ki
  hugepages-1Gi:      2Gi
  hugepages-2Mi:      400Mi
  memory:             259892560Ki
  pods:               737
Allocatable:
  cpu:                127610m
  ephemeral-storage:  17450683567
  hugepages-1Gi:      2Gi
  hugepages-2Mi:      400Mi
  memory:             248720720Ki
  pods:               737

As we can see the hugepages show up as allocatable resources.

Terms of contribution:

By submitting this pull request, I agree that this contribution is dual-licensed under the terms of both the Apache License, version 2.0, and the MIT license.

@piyush-jena
piyush-jena force-pushed the hugepages-final branch 4 times, most recently from df10776 to eb46523 Compare June 24, 2026 20:30
@piyush-jena
piyush-jena force-pushed the hugepages-final branch 2 times, most recently from 105068c to 4acc986 Compare July 8, 2026 07:29
Comment thread sources/shared-defaults/defaults.toml
@vigh-m

vigh-m commented Jul 10, 2026

Copy link
Copy Markdown
Contributor

Could you test failure to boot? Using essential= true and then requesting an allocation that isn't possible?

@piyush-jena
piyush-jena force-pushed the hugepages-final branch 2 times, most recently from a5417c7 to 99a8f2b Compare July 16, 2026 18:23
@piyush-jena
piyush-jena marked this pull request as ready for review July 16, 2026 22:42
@piyush-jena
piyush-jena force-pushed the hugepages-final branch 2 times, most recently from 75a7559 to 4452a7c Compare July 21, 2026 05:08
@piyush-jena

Copy link
Copy Markdown
Contributor Author

@vigh-m - Testing related to that is covered here - bottlerocket-os/bottlerocket-core-kit#952.

Some additional testing:
with the following settings:

[ssm-user@control]$ apiclient apply <<EOF
> [settings.kernel.hugepages.static]
essential = false
[settings.kernel.hugepages.static."16Mi"]
count = "200"
[settings.kernel.hugepages.transparent]
enabled = "madvise"
defrag = "defer"
> EOF

the boot fails because 16Mi size passes static checks but fails runtime check (while setting the setting). The instance fails to boot. The syslog says the following:

[    3.281763] corndog[4633]: Failed to apply hugepage settings: Unsupported huge page size 16384 kiB
[FAILED] Failed to start Reserves static hugepages at early boot.
See 'systemctl status configure-hugepages.service' for details.
[DEPEND] Dependency failed for Bottlerocket initial configuration complete.
[DEPEND] Dependency failed for Activate configured.target.
[  OK  ] Finished Sets the hostname.
[  OK  ] Finished Bootstrap Commands.
        2026-07-20T20:42:01+00:00

Signed-off-by: Piyush Jena <jepiyush@amazon.com>
Signed-off-by: Piyush Jena <jepiyush@amazon.com>
@piyush-jena

Copy link
Copy Markdown
Contributor Author

As mentioned in this PR - #4459 metadata migration is not required. Added services migration.

Booted older version and checked for upgrade. Also, as you can see the hugepage settings are not present.

bash-5.2# apiclient get os
{
  "os": {
    "arch": "x86_64",
    "build_id": "d4932ff8",
    "pretty_name": "Bottlerocket OS 1.63.0 (aws-k8s-1.36)",
    "variant_id": "aws-k8s-1.36",
    "version_id": "1.63.0"
  }
}
bash-5.2# updog check-update -a --json
[
  {
    "variant": "aws-k8s-1.36",
    "arch": "x86_64",
    "version": "1.64.0",
    "max_version": "1.64.0",
    "waves": {
      "0": "2026-07-22T01:01:40.491069504Z",
      "20": "2026-07-22T04:01:40.491069504Z",
      "102": "2026-07-23T00:01:40.491069504Z",
      "307": "2026-07-24T00:01:40.491069504Z",
      "819": "2026-07-26T00:01:40.491069504Z",
      "1228": "2026-07-27T00:01:40.491069504Z",
      "1843": "2026-07-28T00:01:40.491069504Z"
    },
    "images": {
      "boot": "bottlerocket-aws-k8s-1.36-x86_64-1.64.0-9850eb44-dirty-boot.ext4.lz4",
      "root": "bottlerocket-aws-k8s-1.36-x86_64-1.64.0-9850eb44-dirty-root.ext4.lz4",
      "hash": "bottlerocket-aws-k8s-1.36-x86_64-1.64.0-9850eb44-dirty-root.verity.lz4"
    }
  }
]
bash-5.2# apiclient apply <<EOF
> [settings.kernel.hugepages.static]
> essential = false
> [settings.kernel.hugepages.static."1Gi"]
> count = "1"
> EOF
Failed to apply settings: Failed to PATCH settings from '-' to '/settings?tx=apiclient-apply-a1ZezHgxniG98K3b': Status 400 when PATCHing /settings?tx=apiclient-apply-a1ZezHgxniG98K3b: Json deserialize error: unknown field `hugepages`, expected one of `lockdown`, `modules`, `sysctl` at line 1 column 22
bash-5.2# updog update -i 1.64.0 -r -n
Starting update to 1.64.0
Reboot scheduled for Wed 2026-07-22 00:07:14 UTC, use 'shutdown -c' to cancel.
Update applied: aws-k8s-1.36 1.64.0

After reboot. The version is 1.64.0 and the hugepages settings work. I downgrade post that.

[ssm-user@control]$ apiclient get os
{
  "os": {
    "arch": "x86_64",
    "build_id": "9850eb44-dirty",
    "pretty_name": "Bottlerocket OS 1.64.0 (aws-k8s-1.36)",
    "variant_id": "aws-k8s-1.36",
    "version_id": "1.64.0"
  }
}
[ssm-user@control]$ apiclient apply <<EOF
[settings.kernel.hugepages.static]
essential = false
[settings.kernel.hugepages.static."2Mi"]
count = "10"
[settings.kernel.hugepages.transparent]
enabled = "madvise"
defrag = "defer"
EOF
[ssm-user@control]$ enter-admin-container
Confirming admin container is enabled...
Waiting for admin container to start...
Entering admin container
          Welcome to Bottlerocket's admin container!
    ╱╲
   ╱┄┄╲   This container provides access to the Bottlerocket host
   │▗▖│   filesystems (see /.bottlerocket/rootfs) and contains common
  ╱│  │╲  tools for inspection and troubleshooting.  It is based on
  │╰╮╭╯│  Amazon Linux 2023, and most things are in the same places you
    ╹╹    would find them on an AL2023 host.

To permit more intrusive troubleshooting, including actions that mutate the
running state of the Bottlerocket host, we provide a tool called "sheltie"
(`sudo sheltie`).  When run, this tool drops you into a root shell in the
Bottlerocket host's root filesystem.
[root@admin]# sheltie
bash-5.2# signpost rollback-to-inactive

After downgrade the settings aren't present.

[ssm-user@control]$ apiclient get settings.kernel.hugepages
{}
[ssm-user@control]$ apiclient get os
{
  "os": {
    "arch": "x86_64",
    "build_id": "d4932ff8",
    "pretty_name": "Bottlerocket OS 1.63.0 (aws-k8s-1.36)",
    "variant_id": "aws-k8s-1.36",
    "version_id": "1.63.0"
  }
}

@piyush-jena piyush-jena changed the title feat: add hugepages support Add support for static and transparent hugepages Jul 22, 2026
@piyush-jena
piyush-jena merged commit efacce9 into bottlerocket-os:develop Jul 22, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants