Skip to content

storage: test large volumes#437

Open
glehmann wants to merge 5 commits intogln/storage-debugger-xva-retention-uxrtfrom
gln/storage-volume-size-support-vzxu
Open

storage: test large volumes#437
glehmann wants to merge 5 commits intogln/storage-debugger-xva-retention-uxrtfrom
gln/storage-volume-size-support-vzxu

Conversation

@glehmann
Copy link
Copy Markdown
Member

@glehmann glehmann commented Mar 26, 2026 •
edited
Loading

This PR is part of a stack containing 18 PRs:

  1. master
  2. storage: avoid removing the xva/vdi before going in the debugger #436
  3. "storage: test large volumes" (this PR)
  4. host: include mdadm RAID devices in disk detection #447
  5. Update randstream to 0.5.0 #446
  6. storage: Avoid writing the whole device in coalesce tests #449
  7. storage: avoid writing the whole device in migration tests #450
  8. xva/vdi: only write a small data amount in large volumes for faster tests #452
  9. storage: test full device write #453
  10. storage: test that we can't create a vdi over its max allowed size #454
  11. storage: add jobs for large volume tests #461
  12. skip large volume tests for zvol and nfsv4 #464
  13. storage: add comprehensive tests for lvmohba storage repositories #470
  14. storage: free space for XVA import by destroying source VM first #471
  15. storage: limit data written per VDI with --write-volume-cap #481
  16. migration: create a xfs sr on the second host for intra/cross-pool migration #497
  17. Enhance block device management #498
  18. Add VHD_MAX and QCOW2_MAX symbolic size constants #500
  19. partially_populate_device: align spans to block size for better performance #509

@glehmann glehmann requested review from a team as code owners March 26, 2026 08:21
@glehmann glehmann force-pushed the gln/storage-volume-size-support-vzxu branch from 01c5e43 to 859fc5c Compare March 26, 2026 08:26
@glehmann glehmann force-pushed the gln/storage-debugger-xva-retention-uxrt branch from 12bb98b to a57c109 Compare March 26, 2026 08:26
@glehmann glehmann force-pushed the gln/storage-volume-size-support-vzxu branch from 859fc5c to 4cd93f5 Compare March 26, 2026 08:28
last-genius
last-genius reviewed Mar 27, 2026
View reviewed changes
Comment thread tests/storage/lvm/test_lvm_sr.py
@glehmann glehmann force-pushed the gln/storage-volume-size-support-vzxu branch from 4cd93f5 to 346eff1 Compare March 27, 2026 10:06
@glehmann glehmann mentioned this pull request Mar 27, 2026
Open
@glehmann glehmann force-pushed the gln/storage-volume-size-support-vzxu branch 2 times, most recently from 5d3fa9b to d96b2a9 Compare March 30, 2026 11:27
@glehmann glehmann force-pushed the gln/storage-debugger-xva-retention-uxrt branch 2 times, most recently from 080ee74 to c130ebc Compare March 31, 2026 18:27
@glehmann glehmann force-pushed the gln/storage-volume-size-support-vzxu branch from d96b2a9 to a11ecb2 Compare March 31, 2026 18:27
@glehmann glehmann changed the title storage: test large volumes gln/storage-volume-size-support-vzxu Mar 31, 2026
This was referenced Mar 31, 2026
Merged
Open
Open
Open
Open
Open
Open
Open
Open
@glehmann glehmann force-pushed the gln/storage-volume-size-support-vzxu branch 2 times, most recently from 9b0818e to 5800d41 Compare March 31, 2026 20:36
@glehmann glehmann changed the title gln/storage-volume-size-support-vzxu storage: test large volumes Apr 6, 2026
@glehmann glehmann mentioned this pull request Apr 7, 2026
Open
@glehmann glehmann force-pushed the gln/storage-volume-size-support-vzxu branch 2 times, most recently from a6a78f3 to 0f07c9b Compare April 7, 2026 13:32
@glehmann glehmann mentioned this pull request Apr 8, 2026
Open
@glehmann glehmann force-pushed the gln/storage-debugger-xva-retention-uxrt branch from 6d2d859 to ef4ead1 Compare April 10, 2026 19:18
@glehmann glehmann force-pushed the gln/storage-volume-size-support-vzxu branch 2 times, most recently from 3563fb6 to f4248cf Compare April 13, 2026 08:20
This was referenced Apr 13, 2026
Open
Open
stormi
stormi reviewed Apr 14, 2026
View reviewed changes
Comment thread tests/storage/storage.py Outdated
vm.export(xva_path, compression)
# check that the zero blocks are not part of the result. Most of the data is from the random stream, so
# compression has little effect. We just check the result is between 500 and 700 MiB
# compression has little effect. We just take into account the system sie
Copy link
Copy Markdown
Member

@stormi stormi Apr 14, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

s/sie/size/

stormi
stormi requested changes Apr 14, 2026
View reviewed changes
Copy link
Copy Markdown
Member

@stormi stormi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry to be demanding, but commit 4 lacks a description of the changes, explaining how it works.

stormi
stormi reviewed Apr 14, 2026
View reviewed changes
Comment thread tests/storage/zfsvol/test_zfsvol_sr.py
Copy link
Copy Markdown
Member

@stormi stormi Apr 14, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fifth commit:

Fix remaing hardcoded device name
The tests was using a hardcoded xvdb which worked because it matches the
default naming.

=>

Fix remaining hardcoded device name
The test was using a hardcoded xvdb which worked because it matches the
default naming.

@glehmann glehmann force-pushed the gln/storage-debugger-xva-retention-uxrt branch from ef4ead1 to cd1c5d4 Compare April 15, 2026 16:18
@glehmann glehmann force-pushed the gln/storage-volume-size-support-vzxu branch from f4248cf to eb0eac0 Compare April 15, 2026 16:18
@glehmann glehmann mentioned this pull request Apr 16, 2026
Open
@glehmann glehmann force-pushed the gln/storage-volume-size-support-vzxu branch 2 times, most recently from 9ade0b0 to a63e581 Compare April 17, 2026 13:52
@glehmann glehmann force-pushed the gln/storage-debugger-xva-retention-uxrt branch from cd1c5d4 to 27d715c Compare April 17, 2026 13:52
@glehmann glehmann force-pushed the gln/storage-volume-size-support-vzxu branch from a63e581 to d7826f6 Compare April 21, 2026 06:08
@glehmann glehmann force-pushed the gln/storage-debugger-xva-retention-uxrt branch from 27d715c to aeaca44 Compare April 24, 2026 14:57
@glehmann glehmann force-pushed the gln/storage-volume-size-support-vzxu branch from d7826f6 to a29798c Compare April 24, 2026 14:57
This was referenced Apr 24, 2026
Open
Open
Open
glehmann added 5 commits April 29, 2026 19:38
@glehmann
storage: make VDI sizes configurable via --volume-size flag
d9aad9b 
Add pytest command-line option to configure default test VDI sizes,
enabling testing of large volumes supported by the new qcow2 feature.
The default remains 1GiB to avoid hardware requirements; larger sizes
will be added as new jobs in jobs.py.

Changes:
- Add parse_size() to handle size string parsing (e.g., '2.5TiB', '1GiB')
- Remove hardcoded integrity checksums to adapt to any volume size
- Introduce randstream() helper to keep code compact and extract checksums
- Update all storage fixtures to use configurable volume_size

Signed-off-by: Gaëtan Lehmann <gaetan.lehmann@vates.tech>
@glehmann
storage: move the vdi fixtures on thick provisioning to function scope
0bda2c3 
At module or package scope, we may have a vdi still hanging around, using the
space we may need for another test

Signed-off-by: Gaëtan Lehmann <gaetan.lehmann@vates.tech>
@glehmann
Use a NFS volume to temporarily store the exported vdis
7a15817 
With larger volume sizes, the exported images may not fit in /tmp.
The large_temp_dir fixture still defaults to /tmp when no nfs configuration
is available, to keep the tests runnable, with small volume sizes, without
a nfs server.

Signed-off-by: Gaëtan Lehmann <gaetan.lehmann@vates.tech>
@glehmann
storage: test XVA export/import with various disk sizes
6f96420 
Resize test VM's VDI to match the --volume-size parameter to support testing
with different volume sizes. Dynamically calculate expected XVA sizes and
handle filesystem resizing on Alpine systems.

Signed-off-by: Gaëtan Lehmann <gaetan.lehmann@vates.tech>
@glehmann
storage: use dynamic device names instead of hardcoded device names
9d399c6 
Remove hardcoded 'xvdb' device name and use the device name returned by
vm.connect_vdi() to support flexible device naming.

Signed-off-by: Gaëtan Lehmann <gaetan.lehmann@vates.tech>
@glehmann glehmann force-pushed the gln/storage-debugger-xva-retention-uxrt branch from aeaca44 to b2250b7 Compare April 29, 2026 17:39
@glehmann glehmann force-pushed the gln/storage-volume-size-support-vzxu branch from a29798c to 9d399c6 Compare April 29, 2026 17:39
@glehmann glehmann mentioned this pull request Apr 29, 2026
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@last-genius last-genius last-genius left review comments

@stormi stormi stormi approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@glehmann @stormi @last-genius