Skip to content

Add FLUX model to Hub - #2944

Open
maitry63 wants to merge 5 commits into
keras-team:masterfrom
maitry63:fix_flux_model
Open

Add FLUX model to Hub#2944
maitry63 wants to merge 5 commits into
keras-team:masterfrom
maitry63:fix_flux_model

Conversation

@maitry63

Copy link
Copy Markdown
Contributor

This PR fixes FLUX model to KerasHub, including the model implementation, checkpoint conversion utilities, and preset generation.

Fixes: #2938

Changes -

  • Added the FLUX backbone implementation to KerasHub.
  • Added FLUX transformer layers and supporting math utilities.
  • Added support for:
    • Double-stream transformer blocks.
    • Single-stream transformer blocks.
    • Timestep embeddings.
    • Positional embeddings.
    • Modulation and AdaLN layers.
    • Attention and MLP projections.
    • Final output projection.
  • Updated the FLUX model construction and Functional graph handling.
  • Added FLUX configuration conversion from the Hugging Face configuration.
  • Added FLUX checkpoint weight conversion from the original PyTorch/Safetensors
    checkpoint format.
  • Supports conversion of modulation weights for both dedicated modulation
    layers and Sequential-based implementations.
  • Added the FLUX.1 Schnell checkpoint-to-KerasHub preset generation script.
  • Generated a KerasHub preset from the official FLUX.1 Schnell checkpoint.
  • Ensured PyTorch linear weights are transposed correctly when loading them into Keras layers.

Model Configuration -

  • Hidden size: 3072
  • Attention heads: 24
  • Double-stream blocks: 19
  • Single-stream blocks: 38
  • Input channels: 64
  • MLP ratio: 4.0
  • Positional embedding axes: [16, 56, 56]
  • Guidance embedding: disabled

Validation -

  • Successfully constructed the FluxBackbone.
  • Successfully converted the FLUX.1 Schnell Safetensors checkpoint.
  • Successfully mapped the checkpoint weights into the Keras model.
  • Successfully generated the KerasHub FLUX.1 Schnell preset.
  • Verified the conversion workflow end-to-end.

Checklist

  • I have added all the necessary unit tests for my change.
  • I have verified that my change does not break existing code and works with all backends (TensorFlow, JAX, and PyTorch).
  • My PR is based on the latest changes of the main branch (if unsure, rebase the code).
  • I have followed the Keras Hub Model contribution guidelines in making these changes.
  • I have followed the Keras Hub API design guidelines in making these changes.
  • I have signed the Contributor License Agreement.

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request refactors the FLUX model implementation in KerasHub, including converting DoubleStreamBlock to a Layer, optimizing tensor reshaping and attention logic for backend-agnostic static shape inference, and adding preset configurations and weight conversion utilities for FLUX.1. Feedback on these changes focuses on replacing placeholder organization names in Kaggle handles, adhering to repository style guides by saving initialization arguments as attributes and implementing get_config() for DoubleStreamBlock, removing redundant monkey-patching in the checkpoint conversion script, and correcting default input shapes in FluxBackbone to match the actual FLUX architecture.

Comment thread keras_hub/src/models/flux/flux_presets.py Outdated
Comment thread keras_hub/src/models/flux/flux_presets.py Outdated
Comment thread keras_hub/src/models/flux/flux_layers.py
Comment thread keras_hub/src/models/flux/flux_layers.py
Comment thread tools/checkpoint_conversion/convert_flux_checkpoints.py Outdated
Comment on lines +71 to +75
image_shape=(None, 3072),
text_shape=(None, 3072),
image_ids_shape=(None, 3),
text_ids_shape=(None, 3),
y_shape=(128,),

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

The default shapes for image_shape, text_shape, and y_shape do not match the actual dimensions used by the FLUX model (which has 64 channels for latent images, 4096 channels for T5 text embeddings, and 768 dimensions for the pooled vector y). If a user instantiates the model with default arguments, they will encounter shape mismatch errors when feeding standard FLUX inputs. Please update the defaults to match the actual FLUX architecture.

Suggested change
image_shape=(None, 3072),
text_shape=(None, 3072),
image_ids_shape=(None, 3),
text_ids_shape=(None, 3),
y_shape=(128,),
image_shape=(None, 64),
text_shape=(None, 4096),
image_ids_shape=(None, 3),
text_ids_shape=(None, 3),
y_shape=(768,),

@laxmareddyp laxmareddyp changed the title Fix: FLUX model, checkpoint conversion, and preset support Add FLUX model to Hub Aug 13, 2026
@laxmareddyp laxmareddyp added the new model For PRs that contribute a new model to the Keras Hub registry. label Aug 13, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

new model For PRs that contribute a new model to the Keras Hub registry.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Flux has no presets: is a checkpoint conversion planned?

2 participants