Repository navigation
[xegpu] Remove reduction_schedule - #313
Conversation
e4164a9 to
917e42b
Compare
| schedules.append(Runner.get_bench_wrapper_schedule(self.payload_function_name)) | ||
| schedules.extend( | ||
| get_xegpu_pipeline(parameters=parameters[0], stop_at_stage=stop_at_stage) | ||
| xegpu.get_xegpu_pipeline( |
There was a problem hiding this comment.
does this also use the reduction schedule? is then worth updating the PR desc (it does not mention about fused attention?)
There was a problem hiding this comment.
No, previously fused_attention defined its own get_xegpu_pipeline function. In this PR it's updated to use the common one defined in xegpu_pipeline.py. There no change in the attention lowering.
| if stop_at_stage == "xegpu-initial": | ||
| return schedules | ||
| schedules.append( | ||
| xegpu.annotate_layouts_schedule( |
There was a problem hiding this comment.
it seems nb_prefetch, prefetch_tile, q_load_tile or v_load_tile are ignored (according to Claude skill I have).
There was a problem hiding this comment.
They are used in the annotate_attention function to override hard-coded defaults, e.g.:
lighthouse/lighthouse/schedule/xegpu/annotate_layouts_schedule.py
Lines 273 to 280 in 917e42b
| # TODO Infer meaningful values. | ||
| wg_rows = 64 | ||
| red_size = 32 | ||
| elif isinstance(target_op, vector.MultiDimReductionOp): |
There was a problem hiding this comment.
Claude found a crash with rank > 2 vectors here. Check RMSNorm with vector<8x64x128xbf16> shape. main seems to fail but with a good error message.
Consider taking a look
There was a problem hiding this comment.
Thanks, I've added a check for 2D shape. I don't think we currently have an example/test for rms norm with that shape. I suggest we generalize this once we have a concrete use case. This PR just ports the existing reduction_schedule functionality to the new pipeline.
| schedules.append( | ||
| reduction_schedule( | ||
| payload_func_name=self.payload_function_name, | ||
| schedules.extend( |
There was a problem hiding this comment.
out of curiosity, how do we fully automate things like layout decisions number of prefetches etc?
There was a problem hiding this comment.
That's a good question. I'm hoping we can implement heuristics for setting valid prefetch tile size configs (e.g., if we know the shape of the tile to be fetched and have (constraints for) thread layout, it's doable). This already exists for gemm kernels. But to get best performance I think we'll have run autotuner (also already exists for gemms).
lighthouse/schedule/xegpu/xegpu_pipeline.py.reduction_schedule.Assisted-by: Claude