Repository navigation
[xegpu] Add modular parameter-free pipeline #293
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
Show all changes
36 commits
Select commit
Hold shift + click to select a range
6531c3b
lowering_common: add gpu-outlining pipeline stage + updates
tkarna 0e1b97f
transform: add transform.alternatives op wrapper
tkarna 2c4906e
smt_ext: fix constrain_params interface registration
tkarna 6e365ff
transform_ext: add matmul_analysis and xegpu_param_selector helpers
tkarna 03ae8df
transform_ext: add infer_xegpu_gemm_params transform op
tkarna 79c781a
transform_ext: add get_param_dict_entry transform op
tkarna b88dc2f
transform_ext: extract_handle has a silenceable bool arg
tkarna 512a519
transform_ext: add emit_definite_failure op
tkarna f1c0601
transform_ext: add compute_num_threads op
tkarna 8714c6d
transform_ext: add compute_sg_layout op
tkarna 3ea2b12
transform_ext: add infer_xegpu_reduction_params
tkarna e09c9a5
matmul analysis: add method to infer wg and k tile sizes from IR
tkarna 57dcacc
matmul_analysis: add tests
tkarna 9f1e7a4
XeGPUParameterSelector: support fixed wg, sg and k tile sizes
tkarna 6461582
infer_xegpu_gemm_params: add force wg/sg/k tile arguments
tkarna 0913e2a
replace_with_fused_attention: tile_size can be a transform param
tkarna 5d55b73
lowering_common: vectorize: hoist after cleanup
tkarna 2b53fda
transform_ext: add infer_xegpu_attention_params op
tkarna 42b3f59
xegpu: add outline_gpu_func schedule
tkarna 0f4ec1a
xegpu: add vectorize schedule
tkarna 8e2a325
xegpu: add bufferize schedule
tkarna 5a64e87
xegpu: add vector_to_xegpu schedule
tkarna 60c48cf
xegpu: add annotate_layouts schedule
tkarna aa04f81
xegpu: add cleanup schedule
tkarna 37cf38a
xegpu: add wg_tiling schedule
tkarna eaa40f3
fix llama3 schedule
tkarna 5628489
inspect_payload: stop tracing producers at matmul op
tkarna f634cea
kernel_bench: use new pipeline via use-gpu-pipeline CLI arg
tkarna 2c64eb0
address comments
tkarna c0327ac
add _schedule suffix to all schedules
tkarna 0caceac
matmul_analysis: add guardrails and tests
tkarna efe4f9b
update matmul_analysis and tests
tkarna 4d68ae9
address comments
tkarna 6603d55
address more comments
tkarna 017c07b
matmul_analysis: more fixes and tests
tkarna 7acf84d
add transform op tests + update docstrings
tkarna File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Some comments aren't visible on the classic Files Changed page.
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
132 changes: 132 additions & 0 deletions
132
lighthouse/dialects/transform/transform_ext/ops/compute_num_threads.py
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,132 @@ | ||
| from collections.abc import Sequence | ||
|
|
||
| from mlir import ir | ||
| from mlir.dialects import ext, transform | ||
| from mlir.dialects.transform import DiagnosedSilenceableFailure | ||
|
|
||
| from lighthouse.dialects.transform.transform_ext import TransformExtensionDialect | ||
|
|
||
|
|
||
| class ComputeNumThreadsOp( | ||
| TransformExtensionDialect.Operation, name="compute_num_threads" | ||
| ): | ||
| """ | ||
| Compute the workgroup thread count from wg and sg tile params. | ||
|
|
||
| Returns `subgroup_size * prod_i(wg_i // sg_i)` over the WG/SG tile | ||
| dimensions, treating an untiled (0) dim as 1 to avoid division by zero. | ||
|
|
||
| This op is intended to be used to compute the number of threads for a | ||
| workgroup based on the provided WG and SG tile sizes. | ||
|
|
||
| Example: | ||
|
|
||
| wg_tile, sg_tile, _ = tr_ext.infer_xegpu_attention_params(anchor_op) | ||
| nb_threads = transform_ext.compute_num_threads( | ||
| wg_tile, sg_tile, base=NB_WORKITEMS | ||
| ) | ||
| xegpu.set_gpu_launch_threads(launch_op, threads=[nb_threads, 1, 1]) | ||
|
|
||
| The wg and sg tiles are each passed as one or more param operands via the | ||
| single `tiles` operand list; `num_wg_handles` marks how many leading | ||
| operands make up the wg tile (the rest make up the sg tile). Each operand | ||
| may itself carry several i64 attrs, so the tiles may be supplied either as | ||
| one param-per-dim handle or as a single handle holding all dims. All params | ||
| are read at apply time, so the tile rank may vary with the payload. | ||
|
|
||
| Args: | ||
| tiles: wg tile param operand(s) followed by sg tile param operand(s). | ||
| num_wg_handles: Number of leading `tiles` operands forming the wg tile. | ||
| base: Multiplier applied to the subgroup count (e.g., the subgroup | ||
| size). | ||
| Return: | ||
| Param holding the thread count as a single i64. | ||
| """ | ||
|
|
||
| tiles: Sequence[ext.Operand[transform.AnyParamType]] | ||
| num_wg_handles: ir.IntegerAttr | ||
| base: ir.IntegerAttr = ext.attribute( | ||
| default_factory=lambda: ir.IntegerAttr.get(ir.IntegerType.get_signless(64), 1) | ||
| ) | ||
| num_threads: ext.Result[transform.AnyParamType[()]] = ext.infer_result() | ||
|
|
||
| @classmethod | ||
| def attach_interface_impls(cls, ctx=None): | ||
| cls.TransformOpInterfaceModel.attach(cls.OPERATION_NAME, context=ctx) | ||
| cls.MemoryEffectsOpInterfaceModel.attach(cls.OPERATION_NAME, context=ctx) | ||
|
|
||
| class TransformOpInterfaceModel(transform.TransformOpInterface): | ||
| @staticmethod | ||
| def apply( | ||
| op: "ComputeNumThreadsOp", | ||
| _rewriter: transform.TransformRewriter, | ||
| results: transform.TransformResults, | ||
| state: transform.TransformState, | ||
| ) -> DiagnosedSilenceableFailure: | ||
| operands = list(op.tiles) | ||
| num_wg = op.num_wg_handles.value | ||
| wg = [ | ||
| ir.IntegerAttr(a).value | ||
| for handle in operands[:num_wg] | ||
| for a in state.get_params(handle) | ||
| ] | ||
| sg = [ | ||
| ir.IntegerAttr(a).value | ||
| for handle in operands[num_wg:] | ||
| for a in state.get_params(handle) | ||
| ] | ||
| if len(wg) != len(sg): | ||
| return DiagnosedSilenceableFailure.SilenceableFailure | ||
|
|
||
| count = op.base.value | ||
| for w, s in zip(wg, sg): | ||
| # Treat an untiled (0) dim as 1 to avoid division by zero. | ||
| factor = (w or 1) // (s or 1) | ||
| assert factor > 0 | ||
| count *= factor | ||
|
|
||
| i64 = ir.IntegerType.get_signless(64) | ||
| results.set_params(op.num_threads, [ir.IntegerAttr.get(i64, count)]) | ||
| return DiagnosedSilenceableFailure.Success | ||
|
|
||
| @staticmethod | ||
| def allow_repeated_handle_operands(_op: "ComputeNumThreadsOp") -> bool: | ||
| return False | ||
|
|
||
| class MemoryEffectsOpInterfaceModel(ir.MemoryEffectsOpInterface): | ||
| @staticmethod | ||
| def get_effects(op: ir.Operation): | ||
| return ( | ||
| transform.only_reads_handle(op.op_operands) | ||
| + transform.produces_handle(op.results) | ||
| + transform.only_reads_payload() | ||
| ) | ||
|
|
||
|
|
||
| def compute_num_threads( | ||
| wg_tile: ir.Value[transform.AnyParamType] | Sequence[ir.Value], | ||
| sg_tile: ir.Value[transform.AnyParamType] | Sequence[ir.Value], | ||
| base: int | ir.IntegerAttr = 1, | ||
| ) -> ir.Value[transform.AnyParamType]: | ||
| """ | ||
| snake_case wrapper to create a ComputeNumThreadsOp. | ||
|
|
||
| Args: | ||
| wg_tile: Param(s) holding the workgroup tile sizes. Either a single | ||
| handle with one i64 per dim, or a list of per-dim handles. | ||
| sg_tile: Param(s) holding the subgroup tile sizes, in the same layout as | ||
| `wg_tile`. | ||
| base: Multiplier applied to the subgroup count (e.g. the subgroup size). | ||
| Return: | ||
| Param holding the thread count as a single i64. | ||
| """ | ||
| i64 = ir.IntegerType.get_signless(64) | ||
| wg_handles = list(wg_tile) if isinstance(wg_tile, (list, tuple)) else [wg_tile] | ||
| sg_handles = list(sg_tile) if isinstance(sg_tile, (list, tuple)) else [sg_tile] | ||
| if not isinstance(base, ir.IntegerAttr): | ||
| base = ir.IntegerAttr.get(i64, base) | ||
| return ComputeNumThreadsOp( | ||
| tiles=wg_handles + sg_handles, | ||
| num_wg_handles=ir.IntegerAttr.get(i64, len(wg_handles)), | ||
| base=base, | ||
| ).num_threads |
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.