Skip to content

Commit f75ba24

Browse files
authored
[xegpu] Remove mlp_schedule (#314)
- Remove `mlp_schedule` that is now superseded by the common xegpu pipeline. - Updates examples: - `matmul.py`: uses xegpu pipeline. - `mlp.py`: uses xegpu pipeline. - `torch_matmul.py`: uses xegpu pipeline. - `kernel_bench.py`: uses xegpu pipeline for gemm/mlp kernels. Drops now obsolete `--use-gpu-pipeline` CLI arg. - `nanoGPT_schedule.py`: uses `annotate_gemm` to add xegpu layout annotations to matmuls. - `llama3_schedule.py`: uses `annotate_gemm` to add xegpu layout annotations to matmuls. - Verified `tune_matmul_costmodel.py` works correctly. - Verified `kernel_bench.py` gemm parameter tuning and parameter disk-caching works. - Updated the only xegpu related yaml pipeline file `pipeline/descriptors/xegpu/matmul/bf16.yaml` to point to the xegpu pipeline. It's not used in Lighthouse at the moment.
1 parent a59b5da commit f75ba24

11 files changed

Lines changed: 202 additions & 196 deletions

File tree

‎examples/xegpu/README.md‎

Lines changed: 4 additions & 63 deletions
Original file line numberDiff line numberDiff line change
@@ -149,71 +149,12 @@ python mlp.py --bias --relu ...
149149

150150
## Kernel tuning
151151

152-
### Exhaustive grid search
152+
### Matmul kernel tuning based on cost modeling
153153

154-
`tune_matmul_gridsearch.py` performs an exhaustive grid search on a matrix multiplication kernel. It takes similar arguments as the `matmul.py` benchmark:
154+
`tune_matmul_costmodel.py` uses cost-model estimates to select promising matrix multiplication configurations for benchmarking. Results are saved to `out_costmodel.csv`.
155155

156156
```bash
157-
python tune_matmul_gridsearch.py --sizes 1024 2048 4096 --bias --relu --no-accumulate-c
157+
python tune_matmul_costmodel.py --sizes 1024 2048 4096 --dry-run
158158
```
159159

160-
The executed parameter combinations are stored in `out_gridsearch.csv` file along with the obtained performance metrics:
161-
162-
```txt
163-
m,n,k,wg_m,wg_n,sg_m,sg_n,k_tile,load_a_m,load_a_k,load_b_k,load_b_n,prefetch_a_m,prefetch_a_k,prefetch_b_k,prefetch_b_n,prefetch_nb,time (us),GFLOPS/s
164-
4096,4096,4096,64,256,32,32,64,8,16,16,16,8,16,8,16,1,???,???
165-
...
166-
```
167-
168-
To get information about the search space (e.g., tile parameter choices) without actually executing the kernels run with `--dry-run` flag:
169-
170-
```bash
171-
python tune_matmul_gridsearch.py --dry-run
172-
```
173-
174-
Example output:
175-
176-
```txt
177-
Matmul problem size: [4096, 4096, 4096]
178-
ab_type='f16'
179-
c_type='f32'
180-
has_bias=False
181-
has_relu=False
182-
accumulate_c=True
183-
Variable set:
184-
wg_m=[64, 128, 256]
185-
wg_n=[64, 128, 256]
186-
sg_m=[32, 64, 128]
187-
sg_n=[32, 64, 128]
188-
k_tile=[16, 32, 64, 128, 256]
189-
load_a_m=[8, 16, 32]
190-
load_a_k=[8, 16, 32]
191-
load_b_k=[8, 16, 32]
192-
load_b_n=[8, 16, 32]
193-
prefetch_a_m=[8, 16, 32]
194-
prefetch_a_k=[8, 16, 32]
195-
prefetch_b_k=[8, 16, 32]
196-
prefetch_b_n=[8, 16, 32]
197-
prefetch_nb=[1]
198-
Total complexity: 2657205 configurations
199-
Number of executed configurations: 5292
200-
```
201-
202-
Total complexity is the number of parameter combinations without any filtering. The number of executed configurations shows the number of valid combinations, i.e. ones that satisfy appropriate (e.g., hardware) constraints.
203-
204-
To dump the best found configurations as JSON files at the end of the search, use `--dump-json n` flag where `n` stands for the number of best configurations. The files are named as `matmul_params_*_00.json` with increasing integer suffix (best configuration being 00).
205-
206-
> [!NOTE]
207-
> Running the grid search typically takes several hours to complete.
208-
209-
### Adaptive sampling with Genetic Algorithm
210-
211-
`tune_matmul_ga.py` employs a genetic algorithm for adaptive sampling to explore the kernel tuning search space. This approach is typically an order of magnitude faster while discovering high throughput parameter combinations.
212-
213-
The command-line interface is similar to `tune_matmul_gridsearch.py`:
214-
215-
```bash
216-
python tune_matmul_ga.py --sizes 1024 2048 4096 --bias --relu --dump-json 10
217-
```
218-
219-
The executed parameter combinations are stored in `out_genetic_algorithm.csv` file.
160+
Omit `--dry-run` to run the tuning and benchmark selected configurations; use `--dump-json 5` to save the five best configurations.

‎examples/xegpu/kernel_bench.py‎

Lines changed: 11 additions & 28 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
# RUN: %PYTHON %s -l 2 -b 9 --dump-kernel=xegpu-wg | FileCheck %s
2-
# RUN: %PYTHON %s -l 2 -b 9 --dump-kernel=xegpu-wg --use-gpu-pipeline | FileCheck %s
3-
# RUN: %PYTHON %s -l 1 -b 97 --input-shapes 32x32x512x64 32x32x512x64 32x32x512x64 --dump-kernel xegpu-wg --use-gpu-pipeline | FileCheck %s
2+
# RUN: %PYTHON %s -l 2 -b 9 --dump-kernel=xegpu-wg | FileCheck %s
3+
# RUN: %PYTHON %s -l 1 -b 97 --input-shapes 32x32x512x64 32x32x512x64 32x32x512x64 --dump-kernel xegpu-wg | FileCheck %s
44
# REQUIRES: torch
55
# REQUIRES: kernel_bench
66

@@ -67,7 +67,6 @@
6767
from lighthouse.schedule.parameters import ScheduleParameters
6868
from lighthouse.pipeline.driver import TransformDriver
6969
from lighthouse.schedule.xegpu import (
70-
mlp_schedule,
7170
elemwise_schedule,
7271
)
7372
from lighthouse.pipeline.helper import PipelineInterrupt
@@ -438,7 +437,6 @@ def infer_params_and_lower(
438437
params_cache_json: str | None = "kb_params.json",
439438
dump_parameters: bool = True,
440439
enable_tuning: bool = True,
441-
use_gpu_pipeline: bool = False,
442440
verbose: int = 0,
443441
) -> ir.Module:
444442
"""
@@ -530,7 +528,6 @@ def infer_params_and_lower(
530528
stop_at_stage=stop_at_stage,
531529
benchmark=benchmark,
532530
payload_func_name=payload_func_name,
533-
use_gpu_pipeline=use_gpu_pipeline,
534531
)
535532

536533
if stop_at_stage:
@@ -551,32 +548,26 @@ def lower_to_llvm(
551548
stop_at_stage: str | None,
552549
benchmark: bool,
553550
payload_func_name: str,
554-
use_gpu_pipeline: bool = False,
555551
) -> ir.Module:
556552
"""Lower payload module to LLVM using the specified schedule and parameters."""
557553
Runner.make_function_callable(mod, payload_func_name)
558-
if use_gpu_pipeline:
554+
if schedule_kind in ("mlp", "reduction", "attention"):
555+
parameters = schedule_params[0]
556+
if schedule_kind == "mlp" and len(schedule_params) > 1:
557+
# TODO fix parameter propagation for multiple layers
558+
parameters = None
559559
pipeline = xegpu.get_xegpu_pipeline(
560-
parameters=schedule_params[0], stop_at_stage=stop_at_stage
560+
parameters=parameters,
561+
stop_at_stage=stop_at_stage,
562+
device=device,
561563
)
562564
else:
563-
if schedule_kind == "mlp":
564-
schedule = mlp_schedule(
565-
params=schedule_params,
566-
payload_func_name=payload_func_name,
567-
device=device,
568-
stop_at_stage=stop_at_stage,
569-
)
570-
elif schedule_kind == "elemwise":
565+
if schedule_kind == "elemwise":
571566
schedule = elemwise_schedule(
572567
params=schedule_params,
573568
payload_func_name=payload_func_name,
574569
stop_at_stage=stop_at_stage,
575570
)
576-
elif schedule_kind == "reduction":
577-
raise NotImplementedError("Reduction kernels require --use-gpu-pipeline.")
578-
elif schedule_kind == "attention":
579-
raise NotImplementedError("Attention kernels require --use-gpu-pipeline.")
580571
else:
581572
raise ValueError(f"Unsupported schedule kind: {schedule_kind}")
582573
pipeline = [schedule]
@@ -611,7 +602,6 @@ def lower_and_execute_benchmark(
611602
stop_at_stage: str | None = None,
612603
verbose: int = 0,
613604
dump_parameters: bool = True,
614-
use_gpu_pipeline: bool = False,
615605
) -> dict:
616606
"""
617607
High-level function to lower and execute a KernelBench benchmark.
@@ -694,7 +684,6 @@ def lower_and_execute_benchmark(
694684
params_cache_json=f"kb_params_level{level}-{id}.json",
695685
dump_parameters=dump_parameters,
696686
enable_tuning=execute,
697-
use_gpu_pipeline=use_gpu_pipeline,
698687
verbose=verbose,
699688
)
700689
backend = gpu_backend(
@@ -939,11 +928,6 @@ def parser_cli_args():
939928
"final",
940929
],
941930
)
942-
parser.add_argument(
943-
"--use-gpu-pipeline",
944-
action="store_true",
945-
help="Use the parameter-free GPU pipeline for execution.",
946-
)
947931
parser.add_argument(
948932
"--nruns",
949933
type=int,
@@ -1038,7 +1022,6 @@ def parser_cli_args():
10381022
verbose=args.verbose,
10391023
stop_at_stage=stop_at_stage,
10401024
dump_parameters=args.dump_parameters,
1041-
use_gpu_pipeline=args.use_gpu_pipeline,
10421025
)
10431026
if stop_at_stage:
10441027
continue

‎examples/xegpu/llama3_schedule.py‎

Lines changed: 16 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -53,7 +53,7 @@
5353
)
5454
from lighthouse.schedule import schedule_boilerplate
5555
from lighthouse.schedule.xegpu.attention_fusion import enable_fastmath_optimizations
56-
from lighthouse.schedule.xegpu.mlp_schedule import xegpu_wg_annotation_for_mlp_layer
56+
from lighthouse.schedule.xegpu.annotate_layouts_schedule import annotate_gemm
5757
from lighthouse.schedule.xegpu.lowering_common import convert_to_gpu_launch
5858
from llama3_payload import F32
5959

@@ -600,7 +600,21 @@ def mm_thread_count(mmp):
600600
for gm, kind in zip(gpu_mods, kinds):
601601
gf = match(gm, ops={"gpu.func"})
602602
if kind == "matmul":
603-
xegpu_wg_annotation_for_mlp_layer(gf, **mm_params_list[mi])
603+
dpas_op = match_and_split(gf, ops={"xegpu.dpas"}, nhandles=1)[0]
604+
p = mm_params_list[mi]
605+
annotate_gemm(
606+
gf,
607+
dpas_op,
608+
force_wg_tile=[p["wg_m"], p["wg_n"]],
609+
force_sg_tile=[p["sg_m"], p["sg_n"]],
610+
force_k_tile=p["k_tile"],
611+
force_load_tile_a=[p["load_a_m"], p["load_a_k"]],
612+
force_load_tile_b=[p["load_b_k"], p["load_b_n"]],
613+
force_prefetch_tile_a=[p["prefetch_a_m"], p["prefetch_a_k"]],
614+
force_prefetch_tile_b=[p["prefetch_b_k"], p["prefetch_b_n"]],
615+
force_prefetch_a_nb=p["prefetch_a_nb"],
616+
force_prefetch_b_nb=p["prefetch_b_nb"],
617+
)
604618
mi += 1
605619
elif kind == "fused_attention":
606620
xegpu_fa_annotation(gf, fa_params)

‎examples/xegpu/matmul.py‎

Lines changed: 14 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -31,7 +31,7 @@
3131
MemoryManager,
3232
GPUMemoryManager,
3333
)
34-
from lighthouse.schedule.xegpu import mlp_schedule, xegpu_to_binary
34+
from lighthouse.schedule import xegpu
3535
from lighthouse.utils.numpy import mlir_to_numpy_dtype
3636
from lighthouse.ingress.mlir_gen import generate_gpu_matmul_payload, get_mlir_elem_type
3737
from lighthouse.schedule.xegpu import XeGPUParameterSelector
@@ -210,20 +210,17 @@ def schedule_modules(
210210
schedules = []
211211
schedules.append(Runner.get_bench_wrapper_schedule(self.payload_function_name))
212212

213-
schedules.append(
214-
mlp_schedule(
215-
payload_func_name=self.payload_function_name,
216-
device=device,
213+
schedules.extend(
214+
xegpu.get_xegpu_pipeline(
215+
parameters=parameters[0],
217216
stop_at_stage=stop_at_stage,
218-
params=parameters,
217+
device=device,
219218
)
220219
)
221220

222221
if stop_at_stage and stop_at_stage != "final":
223222
return schedules
224223

225-
schedules.append(xegpu_to_binary())
226-
227224
return schedules
228225

229226
def shared_libs(self) -> list[str]:
@@ -472,11 +469,6 @@ def parse_cli_args(description):
472469
[
473470
{
474471
"layer_kind": "matmul",
475-
"m": m,
476-
"n": n,
477-
"k": k,
478-
"transpose_a": transpose_a,
479-
"transpose_b": transpose_b,
480472
}
481473
]
482474
)
@@ -502,18 +494,24 @@ def parse_cli_args(description):
502494
if args.prefetch_b_nb is not None:
503495
cli_params["prefetch_b_nb"] = args.prefetch_b_nb
504496

505-
# By default the tile size parameters are left undefined
506497
layer_params = params[0]
507498
if args.json:
508499
# Override parameters with values from JSON file if provided
509500
json_params = ScheduleParameters.from_json(args.json)
510501
layer_params.update(json_params[0] if json_params else {})
511502
# Override with CLI params
512503
layer_params.update(cli_params)
513-
elif cli_params:
504+
else:
514505
# Get default parameters from selector
515506
param_selector = XeGPUParameterSelector(device=args.target)
516-
def_params = param_selector.get_parameters((m, n, k), transpose_a, transpose_b)
507+
def_params = param_selector.get_parameters(
508+
(m, n, k),
509+
transpose_a,
510+
transpose_b,
511+
wg_tile=args.wg_tile,
512+
sg_tile=args.sg_tile,
513+
k_tile=args.k_tile,
514+
)
517515
layer_params.update(def_params[0])
518516
# Override with CLI params
519517
layer_params.update(cli_params)

‎examples/xegpu/mlp.py‎

Lines changed: 6 additions & 35 deletions
Original file line numberDiff line numberDiff line change
@@ -34,8 +34,7 @@
3434
GPUMemoryManager,
3535
)
3636
from lighthouse.utils.numpy import mlir_to_numpy_dtype
37-
from lighthouse.schedule.parameters import ScheduleParameters
38-
from lighthouse.schedule.xegpu import mlp_schedule, xegpu_to_binary
37+
from lighthouse.schedule import xegpu
3938
from lighthouse.ingress.mlir_gen import (
4039
generate_gpu_mlp_payload,
4140
get_mlir_elem_type,
@@ -247,27 +246,18 @@ def payload_module(self) -> ir.Module:
247246
def schedule_modules(
248247
self,
249248
stop_at_stage: str | None = None,
250-
parameters: ScheduleParameters | None = None,
251249
device: str | None = None,
252250
) -> list[ir.Module]:
253-
assert parameters is not None, "Schedule parameters must be provided"
254251
schedules = []
255252
schedules.append(Runner.get_bench_wrapper_schedule(self.payload_function_name))
256253

257-
schedules.append(
258-
mlp_schedule(
259-
payload_func_name=self.payload_function_name,
260-
device=device,
254+
schedules.extend(
255+
xegpu.get_xegpu_pipeline(
261256
stop_at_stage=stop_at_stage,
262-
params=parameters,
257+
device=device,
263258
)
264259
)
265260

266-
if stop_at_stage and stop_at_stage != "final":
267-
return schedules
268-
269-
schedules.append(xegpu_to_binary())
270-
271261
return schedules
272262

273263
def shared_libs(self) -> list[str]:
@@ -426,40 +416,21 @@ def parse_cli():
426416
ab_type = wload.ab_type
427417
acc_type = wload.acc_type
428418

429-
# Initialize layer parameters
430-
params = []
431-
for i, (M, N, K) in enumerate(matmuls):
432-
layer_params = {
433-
"layer_kind": "matmul",
434-
"m": M,
435-
"n": N,
436-
"k": K,
437-
"transpose_a": tr_a if i == 0 else False,
438-
"transpose_b": tr_b,
439-
}
440-
params.append(layer_params)
441-
params = ScheduleParameters(params)
442-
443419
if args.dump_kernel or args.dump_schedule:
444420
pipeline = TransformDriver(
445421
wload.schedule_modules(
446422
stop_at_stage=args.dump_kernel,
447-
parameters=params,
448423
device=args.target,
449424
)
450425
)
451426
payload = pipeline.apply(wload.payload_module())
452427
if args.dump_kernel:
453428
print(payload)
454429
if args.dump_schedule:
455-
for schedule_module in wload.schedule_modules(
456-
parameters=params, device=args.target
457-
):
430+
for schedule_module in wload.schedule_modules(device=args.target):
458431
print(schedule_module)
459432
else:
460-
pipeline = TransformDriver(
461-
wload.schedule_modules(parameters=params, device=args.target)
462-
)
433+
pipeline = TransformDriver(wload.schedule_modules(device=args.target))
463434
payload = pipeline.apply(wload.payload_module())
464435
runner = Runner(
465436
payload,

0 commit comments

Comments
 (0)