Skip to content

Fix potential out-of-bound access in int8_mm.py #1751

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 7 commits into from
Feb 25, 2025
Merged

Fix potential out-of-bound access in int8_mm.py #1751

Changes from all commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
ef79557
fix potential out-of-bound access
mark14wu Feb 21, 2025
46fc72b
remove unused EVEN_K
mark14wu Feb 21, 2025
7202d1c
refactor fix with triton.heuristics
mark14wu Feb 24, 2025
7df7a8e
restore EVEN_K as an input
mark14wu Feb 24, 2025
17dd05a
fix typo
mark14wu Feb 24, 2025
1d33af0
fix another typo
mark14wu Feb 24, 2025
62f7334
ruff reformatted
mark14wu Feb 25, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion torchao/prototype/quantized_training/int8_mm.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -54,6 +54,7 @@


@triton.autotune(configs=configs, key=["M", "N", "K", "stride_ak", "stride_bk"])
@triton.heuristics({"EVEN_K": lambda args: args["K"] % args["BLOCK_K"] == 0})
@triton.jit
def _scaled_int8_mm_kernel(
A_ptr,
Expand Down Expand Up @@ -176,7 +177,6 @@ def scaled_int8_mm_cuda(A: Tensor, B: Tensor, row_scale: Tensor, col_scale: Tens
*A.stride(),
*B.stride(),
*C.stride(),
EVEN_K=K % 2 == 0,
COL_SCALE_SCALAR=col_scale.numel() == 1,
)
return C
Loading