Skip to content

[xegpu] Remove reduction_schedule - #313

Merged
tkarna merged 17 commits into
llvm:mainfrom
tkarna:xegpu-rm-reduction-schedule
Oct 26, 2026
Merged

tkarna merged 17 commits into
llvm:mainfrom
tkarna:xegpu-rm-reduction-schedule

Conversation

@tkarna

@tkarna tkarna commented Oct 23, 2026 •

Copy link
Copy Markdown
Contributor
  • Implements missing reduction kernel functionality to the parameter-free gpu pipeline.
  • The common gpu pipeline is now defined in lighthouse/schedule/xegpu/xegpu_pipeline.py.
  • Migrates layer_norm and softmax examples and the kernel_bench routine to use it.
  • Removes now obsolete reduction_schedule.

Assisted-by: Claude

@tkarna
tkarna force-pushed the xegpu-rm-reduction-schedule branch from e4164a9 to 917e42b Compare October 23, 2026 19:07
@tkarna

tkarna commented Oct 23, 2026

Copy link
Copy Markdown
Contributor Author

@charithaintc

@charithaintc charithaintc left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Thanks.

schedules.append(Runner.get_bench_wrapper_schedule(self.payload_function_name))
schedules.extend(
get_xegpu_pipeline(parameters=parameters[0], stop_at_stage=stop_at_stage)
xegpu.get_xegpu_pipeline(

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

does this also use the reduction schedule? is then worth updating the PR desc (it does not mention about fused attention?)

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, previously fused_attention defined its own get_xegpu_pipeline function. In this PR it's updated to use the common one defined in xegpu_pipeline.py. There no change in the attention lowering.

if stop_at_stage == "xegpu-initial":
return schedules
schedules.append(
xegpu.annotate_layouts_schedule(

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it seems nb_prefetch, prefetch_tile, q_load_tile or v_load_tile are ignored (according to Claude skill I have).

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

They are used in the annotate_attention function to override hard-coded defaults, e.g.:

nb_prefetch = 1
if force_nb_prefetch is not None:
nb_prefetch = force_nb_prefetch
prefetch_shape = [red_tile, d_head] # prefetch the whole local tile
prefetch_sg_data = [16, 32]
if force_prefetch_tile is not None:
assert len(force_prefetch_tile) == 2, "force_prefetch_tile must have length 2"
prefetch_sg_data = list(force_prefetch_tile)

# TODO Infer meaningful values.
wg_rows = 64
red_size = 32
elif isinstance(target_op, vector.MultiDimReductionOp):

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Claude found a crash with rank > 2 vectors here. Check RMSNorm with vector<8x64x128xbf16> shape. main seems to fail but with a good error message.

Consider taking a look

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, I've added a check for 2D shape. I don't think we currently have an example/test for rms norm with that shape. I suggest we generalize this once we have a concrete use case. This PR just ports the existing reduction_schedule functionality to the new pipeline.

Comment thread examples/xegpu/softmax.py
schedules.append(
reduction_schedule(
payload_func_name=self.payload_function_name,
schedules.extend(

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

out of curiosity, how do we fully automate things like layout decisions number of prefetches etc?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's a good question. I'm hoping we can implement heuristics for setting valid prefetch tile size configs (e.g., if we know the shape of the tile to be fetched and have (constraints for) thread layout, it's doable). This already exists for gemm kernels. But to get best performance I think we'll have run autotuner (also already exists for gemms).

@tkarna
tkarna merged commit a59b5da into llvm:main Oct 26, 2026
3 checks passed
@tkarna
tkarna deleted the xegpu-rm-reduction-schedule branch October 26, 2026 20:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants