Skip to content

Add diffusion [Draft : wip] #327

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
shubhammalhotra28 wants to merge 14 commits into
base: smonga/vlm+diffusion
Choose a base branch
from
Open

Add diffusion [Draft : wip] #327

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
14 commits
Select commit Hold shift + click to select a range
88a5d69
apple ios changes
shubhammalhotra28 Feb 2, 2026
5d93b6a
staging changes
shubhammalhotra28 Feb 2, 2026
338176b
updates
shubhammalhotra28 Feb 2, 2026
dcc3000
updates
shubhammalhotra28 Feb 2, 2026
ff05d72
updates
shubhammalhotra28 Feb 2, 2026
9f7da15
changes
shubhammalhotra28 Feb 2, 2026
ac2a957
tweaking some changes
shubhammalhotra28 Feb 2, 2026
c431a64
fixnig ios
shubhammalhotra28 Feb 2, 2026
4509331
minor tweaks
shubhammalhotra28 Feb 2, 2026
f88f570
minor changes - will need to tweak again - since diff models have dif…
shubhammalhotra28 Feb 2, 2026
a37fb27
Optimised llamacpp_backend.cpp
VyasGuru Feb 4, 2026
5028c39
Add diffusion model support and TTS router for Kotlin SDK
shubhammalhotra28 Feb 5, 2026
cf742f8
Merge pull request #330 from VyasGuru/llamabackend
shubhammalhotra28 Feb 5, 2026
bfca896
Merge branch 'main' into add-diffusion
shubhammalhotra28 Feb 5, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions .gitignore
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -340,6 +340,9 @@ sdk/runanywhere-swift/Sources/RunAnywhere/Data/Network/DevelopmentConfig.swift
# Swift SDK - Local XCFrameworks
sdk/runanywhere-swift/Binaries/*.xcframework

# Swift SDK - Vendor directory (locally built xcframeworks)
sdk/runanywhere-swift/vendor/

# Kotlin SDK - Local JNI libs
sdk/runanywhere-kotlin/src/androidMain/jniLibs/

Expand Down
35 changes: 35 additions & 0 deletions ...d/RunAnywhereAI/app/src/main/java/com/runanywhere/runanywhereai/RunAnywhereApplication.kt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -358,6 +358,41 @@ class RunAnywhereApplication : Application() {
)
Log.i("RunAnywhereApp", "✅ ONNX STT/TTS models registered")

// ============================================================================
// Register Diffusion models (ONNX - cross-platform)
// Using models from HuggingFace for ONNX Runtime inference
// ============================================================================

// Stable Diffusion 1.5 ONNX - UNet model (main inference component)
// Source: https://huggingface.co/onnx-community/stable-diffusion-v1-5-ONNX
// Note: Full SD requires multiple ONNX files (text_encoder, unet, vae_decoder, vae_encoder)
// For now, registering the UNet as the primary model file
RunAnywhere.registerModel(
id = "sd15-onnx-unet",
name = "SD 1.5 UNet (ONNX)",
url = "https://huggingface.co/onnx-community/stable-diffusion-v1-5-ONNX/resolve/main/unet/model.onnx",
framework = InferenceFramework.ONNX,
modality = ModelCategory.IMAGE_GENERATION,
memoryRequirement = 3_400_000_000, // ~3.4GB for UNet alone
)

// Recommended mobile-optimized models (architecturally compressed):
// These BK-SDM models are much smaller and faster for on-device inference:
//
// BK-SDM-Tiny (0.50B params, ~500MB) - Best for mobile!
// https://huggingface.co/nota-ai/bk-sdm-tiny
//
// BK-SDM-Small (0.66B params, ~660MB)
// https://huggingface.co/nota-ai/bk-sdm-small
//
// BK-SDM-Base (0.76B params, ~760MB)
// https://huggingface.co/nota-ai/bk-sdm-base
//
// Note: These are in Diffusers/safetensors format and need ONNX conversion.
// Use the Olive tool to convert: https://github.com/microsoft/Olive

Log.i("RunAnywhereApp", "✅ Diffusion models registered (ONNX from HuggingFace)")

Log.i("RunAnywhereApp", "🎉 All modules and models registered")
}
}
Loading