Add FLUX model to Hub - #2944
Conversation
There was a problem hiding this comment.
Code Review
This pull request refactors the FLUX model implementation in KerasHub, including converting DoubleStreamBlock to a Layer, optimizing tensor reshaping and attention logic for backend-agnostic static shape inference, and adding preset configurations and weight conversion utilities for FLUX.1. Feedback on these changes focuses on replacing placeholder organization names in Kaggle handles, adhering to repository style guides by saving initialization arguments as attributes and implementing get_config() for DoubleStreamBlock, removing redundant monkey-patching in the checkpoint conversion script, and correcting default input shapes in FluxBackbone to match the actual FLUX architecture.
| image_shape=(None, 3072), | ||
| text_shape=(None, 3072), | ||
| image_ids_shape=(None, 3), | ||
| text_ids_shape=(None, 3), | ||
| y_shape=(128,), |
There was a problem hiding this comment.
The default shapes for image_shape, text_shape, and y_shape do not match the actual dimensions used by the FLUX model (which has 64 channels for latent images, 4096 channels for T5 text embeddings, and 768 dimensions for the pooled vector y). If a user instantiates the model with default arguments, they will encounter shape mismatch errors when feeding standard FLUX inputs. Please update the defaults to match the actual FLUX architecture.
| image_shape=(None, 3072), | |
| text_shape=(None, 3072), | |
| image_ids_shape=(None, 3), | |
| text_ids_shape=(None, 3), | |
| y_shape=(128,), | |
| image_shape=(None, 64), | |
| text_shape=(None, 4096), | |
| image_ids_shape=(None, 3), | |
| text_ids_shape=(None, 3), | |
| y_shape=(768,), |
This PR fixes FLUX model to KerasHub, including the model implementation, checkpoint conversion utilities, and preset generation.
Fixes: #2938
Changes -
checkpoint format.
layers and
Sequential-based implementations.Model Configuration -
[16, 56, 56]Validation -
FluxBackbone.Checklist