fix incorrect torch version test #2786

namgyu-youn · 2025-08-18T04:55:36Z

Summary:
PyTorch pre-release/dev versions have a0/dev in their name. Therefore, the right order is the following:

2.4.0 (stable) < 2.5.0.dev / 2.5.0.git < 2.5.0 (stable)

For correct order, this PR applies true order within stable and pre-release/dev.

fix: torch_version_at_least semantics are incorrect #2722

Test plan: CI

pytorch-bot · 2025-08-18T04:55:39Z

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/ao/2786

📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❌ 3 New Failures

As of commit 05edf52 with merge base e6b38bb ():

NEW FAILURES - The following jobs have failed:

Run 1xH100 Tests / test (H100, linux.aws.h100, --pre torch torchvision torchaudio fbgemm-gpu-genai --index-url https... / linux-job (gh)
test/integration/test_integration.py::TestAutoQuant::test_autoquant_hp_float
Run 1xL4 Tests / test (SM-89, linux.g6.4xlarge.experimental.nvidia.gpu, --pre torch --index-url https://download.p... / linux-job (gh)
test/integration/test_integration.py::TestAutoQuant::test_autoquant_hp_float
Run Regression Tests / test-nightly (CUDA Nightly, linux.g5.12xlarge.nvidia.gpu, --pre torch --index-url https://downloa... / linux-job (gh)
test/integration/test_integration.py::TestAutoQuant::test_autoquant_hp_float

This comment was automatically generated by Dr. CI and updates every 15 minutes.

jerryzh168 · 2025-08-18T22:48:29Z

torchao/utils.py

+
+    # Parser for local identifiers
+    current_version = re.sub(r"\+.*$", "", torch.__version__)
+    return parse_version(current_version) >= parse_version(min_version)


I have some impression that we don't want to do this, but @msaroufim would have more context here

seems fine to merge altho we probably want to delete the compare_versions function

compare_versions & parse_version are unavailable to used here because parse_version only extracts \d+\.\d+\.\d+ (e.g., 2.5.0→[2, 5, 0]). Therefore, we can inject more parsers (e.g., a0, dev) into parse_version, but I am not certain because check_cpu_version & check_xpu_version are chained with them.

btw we also have

ao/torchao/utils.py

Line 1002 in 43b4106

def is_package_at_least(package_name: str, min_version: str):

, should we just reuse that? not sure if this works for pre-releases as well, could you check?

I vaguely remember at some point that we don't want to use parse version, but don't remember why though

btw we also have

ao/torchao/utils.py

Line 1002 in 43b4106

def is_package_at_least(package_name: str, min_version: str):

, should we just reuse that? not sure if this works for pre-releases as well, could you check?
I vaguely remember at some point that we don't want to use parse version, but don't remember why though

Installation

# 2.8.0 stable pip install torch --index-url https://download.pytorch.org/whl/cpu # 2.9.0 pre-release (nighty) pip install torch --index-url https://download.pytorch.org/whl/nightly/cpu

Test code

test_cases = ["2.8.0a0+git9f17037", "2.8.0", "2.9.0a0+git9f17037", "2.9.0", "2.10.0a0+git9f17037", "2.10.0"] print(torch.__version__) print(torch.version) for test in test_cases: print(torch_version_at_least(test), is_package_at_least(package_name="torch", min_version=test))

Result

# stable 2.8.0+cpu <module 'torch.version' from 'ao/.venv/lib/python3.12/site-packages/torch/version.py'> True False True True False False False False False True False True # pre-release (nighty) 2.9.0.dev20250821+cpu <module 'torch.version' from 'ao/.venv/lib/python3.12/site-packages/torch/version.py'> True True True True True False False True False True False True

It seems that is_package_at_least() doesn't work for both stable/pre-release becauseversion() returns the module object; __version__() returns the actual version string. In my naive guess, torch_version_at_least and is_package_at_least can be consolidated because they share "return true if installed <package/torch> is higher than required" (with PyTorch pre-release detection). How about consolidating them?

andrewor14 · 2025-08-19T13:04:39Z

Hi @namgyu-youn, thanks for fixing this. I think we should actually fix compare_versions and parse_version instead, since they are used in a few different places. I'm actually wondering if we can express the comparisons like this:

# parse_version
2.5.0+cu126 -> [2, 5, 0]
2.5.0 -> [2, 5, 0]
2.5.0a0+git9f17037 -> [2, 5, -1]
2.5.0.dev20240708+cu121 -> [2, 5, -1]
2.4.0 -> [2, 4, 0]

Basically always return a list of 3 numbers, and -1 represents a pre-release. Then we can just compare as follows:

# don't need compare_versions anymore
parse_version(x) >= parse_version(y)

# These are True
[2, 5, -1] < [2, 5, 0]
[2, 5, -1] > [2, 4, 99]

To me this is simpler than the existing compare_versions logic, which I think is a bit unintuitive to understand. What do you think?

namgyu-youn · 2025-08-19T13:46:17Z

Hi @namgyu-youn, thanks for fixing this. I think we should actually fix compare_versions and parse_version instead, since they are used in a few different places. I'm actually wondering if we can express the comparisons like this:
# parse_version
2.5.0+cu126 -> [2, 5, 0]
2.5.0 -> [2, 5, 0]
2.5.0a0+git9f17037 -> [2, 5, -1]
2.5.0.dev20240708+cu121 -> [2, 5, -1]
2.4.0 -> [2, 4, 0]
Basically always return a list of 3 numbers, and -1 represents a pre-release. Then we can just compare as follows:
# don't need compare_versions anymore
parse_version(x) >= parse_version(y)

# These are True
[2, 5, -1] < [2, 5, 0]
[2, 5, -1] > [2, 4, 99]
To me this is simpler than the existing compare_versions logic, which I think is a bit unintuitive to understand. What do you think?

Sounds good to me. Customized parse_version was just a careful suggestion, and I also agree with -1 for better logic. Let me fix it with co-authoring; thanks again!

…versions - Co-authored-by: andrewor14 <[email protected]>

test/test_utils.py

andrewor14 · 2025-08-19T21:12:18Z

torchao/utils.py

+    version = re.sub(r"\+.*$", "", version_string)
+
+    # Check for pre-release indicators (including all common patterns)
+    is_prerelease = bool(re.search(r"(a|b|dev)", version))


Seems like this is simpler:

is_prerelease = bool(re.search(r"(git|dev)", version_string)) match = re.match(r"(\d+)\.(\d+)\.(\d+)", version_string)

any other patterns we need to search for? @jerryzh168

git and dev should be enough I think, we have git locally and dev in nightly

jerryzh168 · 2025-08-21T00:13:25Z

torchao/utils.py

+    Examples: "2.5.0.dev20240708+cu121" -> [2, 5, -1], "2.5.0" -> [2, 5, 0]
+    """
+    # Check for pre-release indicators
+    is_prerelease = bool(re.search(r"(git|dev)", version_string))


please also verify the version string for pytorch, if you build it from source

git and dev should be enough I think, we have git locally and dev in nightly

Aren't the above comments mentioning git and dev enough for a pre-release pattern? Let me know if I am missing something.

Yeah should be git if you build from source, this is mine: 2.9.0a0+git1f19003

yeah I was referring to torchao as well but then realized this is for torch

andrewor14 · 2025-08-22T20:21:08Z

Thanks @namgyu-youn, this looks good to me! The test failures are unrelated.

Summary: version semantics is fixed in pytorch#2786 and we are updating the version check logic accordingly Test Plan: python test/integration/test_integration.py -k test_autoquant_hp_float Reviewers: Subscribers: Tasks: Tags:

Summary: version semantics is fixed in #2786 and we are updating the version check logic accordingly Test Plan: python test/integration/test_integration.py -k test_autoquant_hp_float Reviewers: Subscribers: Tasks: Tags:

fix torch version detector

0ada4b5

meta-cla bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Aug 18, 2025

namgyu-youn changed the title ~~fix torch version detector~~ fix incorrect torch version test Aug 18, 2025

jerryzh168 reviewed Aug 18, 2025

View reviewed changes

jerryzh168 requested a review from msaroufim August 18, 2025 22:48

msaroufim requested a review from andrewor14 August 19, 2025 00:00

andrewor14 added the topic: improvement Use this tag if this PR is an improvement (doesn't fit into any of the other categories) label Aug 19, 2025

namgyu-youn added 3 commits August 20, 2025 01:14

add pre-release parser for torch_version_at_least and remove compare_…

de7a877

…versions - Co-authored-by: andrewor14 <[email protected]>

add pre-release parser for torch_version_at_least and remove compare_…

3818cbf

…versions - Co-authored-by: andrewor14 <[email protected]>

remove local test code

747026b

namgyu-youn requested a review from jerryzh168 August 19, 2025 16:49

andrewor14 reviewed Aug 19, 2025

View reviewed changes

namgyu-youn added 2 commits August 20, 2025 12:03

update PyTorch pre-release version indicator

37b7089

update pre-release patterns

05edf52

jerryzh168 reviewed Aug 21, 2025

View reviewed changes

andrewor14 merged commit 07fbc89 into pytorch:main Aug 22, 2025
15 of 18 checks passed

namgyu-youn deleted the version-fix branch August 22, 2025 20:23

jerryzh168 mentioned this pull request Aug 22, 2025

Fix autoquant after version util changes #2858

Merged

fix incorrect torch version test #2786

fix incorrect torch version test #2786

Conversation

namgyu-youn commented Aug 18, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

pytorch-bot bot commented Aug 18, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/ao/2786

❌ 3 New Failures

Uh oh!

jerryzh168 Aug 18, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

msaroufim Aug 19, 2025

Choose a reason for hiding this comment

Uh oh!

namgyu-youn Aug 19, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jerryzh168 Aug 20, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

namgyu-youn Aug 22, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Installation

Test code

Result

Uh oh!

andrewor14 commented Aug 19, 2025

Uh oh!

namgyu-youn commented Aug 19, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

andrewor14 Aug 19, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jerryzh168 Aug 20, 2025

Choose a reason for hiding this comment

Uh oh!

jerryzh168 Aug 21, 2025

Choose a reason for hiding this comment

Uh oh!

namgyu-youn Aug 22, 2025

Choose a reason for hiding this comment

Uh oh!

andrewor14 Aug 22, 2025

Choose a reason for hiding this comment

Uh oh!

jerryzh168 Aug 22, 2025

Choose a reason for hiding this comment

Uh oh!

andrewor14 commented Aug 22, 2025

Uh oh!

Uh oh!

Uh oh!

namgyu-youn commented Aug 18, 2025 •

edited

Loading

pytorch-bot bot commented Aug 18, 2025 •

edited

Loading

jerryzh168 Aug 18, 2025 •

edited

Loading

namgyu-youn Aug 19, 2025 •

edited

Loading

jerryzh168 Aug 20, 2025 •

edited

Loading

namgyu-youn Aug 22, 2025 •

edited

Loading

namgyu-youn commented Aug 19, 2025 •

edited

Loading

andrewor14 Aug 19, 2025 •

edited

Loading