Minimum number of GPUs needed to pretrain llama4_17bx16e - 8 ?

[githubsgi]

Going by the config files it would be 8 H100 class GPUs, Is 8 a reasonable number ?

[tianyu-l]

where does it say so?

[githubsgi]

@tianyu-l , that I inferred from the toml file, which has TP=8, and everything else set to 1.

[tianyu-l]

This is -1 not 1.
https://github.com/pytorch/torchtitan/blob/main/torchtitan/experiments/llama4/train_configs/llama4_17bx16e.toml#L42

We did say it's 64 gpus.
https://github.com/pytorch/torchtitan/blob/main/torchtitan/experiments/llama4/train_configs/llama4_17bx16e.toml#L1