Skip to content

Add CentML as an inference provider #1394

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 16 commits into
base: main
Choose a base branch
from

Conversation

V2arK
Copy link

@V2arK V2arK commented Apr 28, 2025

✨ Add CentML server-less support to huggingface.js/inference

What’s in this PR

Area Change
README Adds CentML to the supported-provider tables and links to the partner-model list.
Provider wiring * src/lib/getProviderHelper.ts – registers CentML for conversational .
* src/providers/centml.ts – new task classes
Docs Links to https://huggingface.co/api/partners/centml/models.
Misc Uses the current CentML base URL https://api.centml.com.

Why

* CentML just opened public server-less endpoints; supporting it keeps huggingface.js a one-stop client for all HF partners.
* CentML routes are OpenAI-shaped, so we extend the existing base classes.

How I tested

enable tests in packages/inference/test/InferenceClient.spec.ts, and did

pnpm test -- \
  --reporter verbose \
  test/InferenceClient.spec.ts -t "CentML"

> @huggingface/[email protected] test /Users/honglin/centML/huggingface.js/packages/inference
> vitest run --config vitest.config.mts "--reporter" "verbose" "test/InferenceClient.spec.ts" "-t" "CentML"


 RUN  v0.34.6 /Users/honglin/centML/huggingface.js/packages/inference

stderr | unknown test
Set HF_TOKEN in the env to run the tests for better rate limits

 ✓ test/InferenceClient.spec.ts (100) 4226ms
   ✓ InferenceClient (100) 4225ms
     ↓ backward compatibility (1) [skipped]
       ↓ works with old HfInference name [skipped]
     ↓ HF Inference (48) [skipped]
       ↓ throws error if model does not exist [skipped]
       ↓ fillMask [skipped]
       ↓ works without model [skipped]
       ↓ summarization [skipped]
       ↓ questionAnswering [skipped]
       ↓ tableQuestionAnswering [skipped]
       ↓ documentQuestionAnswering [skipped]
       ↓ documentQuestionAnswering with non-array output [skipped]
       ↓ visualQuestionAnswering [skipped]
       ↓ textClassification [skipped]
       ↓ textGeneration - gpt2 [skipped]
       ↓ textGeneration - openai-community/gpt2 [skipped]
       ↓ textGenerationStream - meta-llama/Llama-3.2-3B [skipped]
       ↓ textGenerationStream - catch error [skipped]
       ↓ textGenerationStream - Abort [skipped]
       ↓ tokenClassification [skipped]
       ↓ translation [skipped]
       ↓ zeroShotClassification [skipped]
       ↓ sentenceSimilarity [skipped]
       ↓ FeatureExtraction [skipped]
       ↓ FeatureExtraction - auto-compatibility sentence similarity [skipped]
       ↓ FeatureExtraction - facebook/bart-base [skipped]
       ↓ FeatureExtraction - facebook/bart-base, list input [skipped]
       ↓ automaticSpeechRecognition [skipped]
       ↓ audioClassification [skipped]
       ↓ audioToAudio [skipped]
       ↓ textToSpeech [skipped]
       ↓ imageClassification [skipped]
       ↓ zeroShotImageClassification [skipped]
       ↓ objectDetection [skipped]
       ↓ imageSegmentation [skipped]
       ↓ imageToImage [skipped]
       ↓ imageToImage blob data [skipped]
       ↓ textToImage [skipped]
       ↓ textToImage with parameters [skipped]
       ↓ imageToText [skipped]
       ↓ request - openai-community/gpt2 [skipped]
       ↓ tabularRegression [skipped]
       ↓ tabularClassification [skipped]
       ↓ endpoint - makes request to specified endpoint [skipped]
       ↓ chatCompletion modelId - OpenAI Specs [skipped]
       ↓ chatCompletionStream modelId - OpenAI Specs [skipped]
       ↓ chatCompletionStream modelId Fail - OpenAI Specs [skipped]
       ↓ chatCompletion - OpenAI Specs [skipped]
       ↓ chatCompletionStream - OpenAI Specs [skipped]
       ↓ custom mistral - OpenAI Specs [skipped]
       ↓ custom openai - OpenAI Specs [skipped]
       ↓ OpenAI client side routing - model should have provider as prefix [skipped]
     ↓ Fal AI (4) [skipped]
       ↓ textToImage - black-forest-labs/FLUX.1-schnell [skipped]
       ↓ textToImage - SD LoRAs [skipped]
       ↓ textToImage - Flux LoRAs [skipped]
       ↓ automaticSpeechRecognition - openai/whisper-large-v3 [skipped]
     ↓ Featherless (3) [skipped]
       ↓ chatCompletion [skipped]
       ↓ chatCompletion stream [skipped]
       ↓ textGeneration [skipped]
     ↓ Replicate (10) [skipped]
       ↓ textToImage canonical - black-forest-labs/FLUX.1-schnell [skipped]
       ↓ textToImage canonical - black-forest-labs/FLUX.1-dev [skipped]
       ↓ textToImage canonical - stabilityai/stable-diffusion-3.5-large-turbo [skipped]
       ↓ textToImage versioned - ByteDance/SDXL-Lightning [skipped]
       ↓ textToImage versioned - ByteDance/Hyper-SD [skipped]
       ↓ textToImage versioned - playgroundai/playground-v2.5-1024px-aesthetic [skipped]
       ↓ textToImage versioned - stabilityai/stable-diffusion-xl-base-1.0 [skipped]
       ↓ textToSpeech versioned [skipped]
       ↓ textToSpeech OuteTTS -  usually Cold [skipped]
       ↓ textToSpeech Kokoro [skipped]
     ↓ SambaNova (3) [skipped]
       ↓ chatCompletion [skipped]
       ↓ chatCompletion stream [skipped]
       ↓ featureExtraction [skipped]
     ↓ Together (4) [skipped]
       ↓ chatCompletion [skipped]
       ↓ chatCompletion stream [skipped]
       ↓ textToImage [skipped]
       ↓ textGeneration [skipped]
     ↓ Nebius (3) [skipped]
       ↓ chatCompletion [skipped]
       ↓ chatCompletion stream [skipped]
       ↓ textToImage [skipped]
     ↓ 3rd party providers (1) [skipped]
       ↓ chatCompletion - fails with unsupported model [skipped]
     ↓ Fireworks (2) [skipped]
       ↓ chatCompletion [skipped]
       ↓ chatCompletion stream [skipped]
     ↓ Hyperbolic (4) [skipped]
       ↓ chatCompletion - hyperbolic [skipped]
       ↓ chatCompletion stream [skipped]
       ↓ textToImage [skipped]
       ↓ textGeneration [skipped]
     ↓ Novita (2) [skipped]
       ↓ chatCompletion [skipped]
       ↓ chatCompletion stream [skipped]
     ↓ Black Forest Labs (2) [skipped]
       ↓ textToImage [skipped]
       ↓ textToImage URL [skipped]
     ↓ Cohere (2) [skipped]
       ↓ chatCompletion [skipped]
       ↓ chatCompletion stream [skipped]
     ↓ Cerebras (2) [skipped]
       ↓ chatCompletion [skipped]
       ↓ chatCompletion stream [skipped]
     ↓ Nscale (3) [skipped]
       ↓ chatCompletion [skipped]
       ↓ chatCompletion stream [skipped]
       ↓ textToImage [skipped]
     ↓ Groq (2) [skipped]
       ↓ chatCompletion [skipped]
       ↓ chatCompletion stream [skipped]
     ✓ CentML (4) 4225ms
       ✓ chat completions (4) 4225ms
         ✓ basic chat completion 1047ms
         ✓ chat completion with multiple messages 970ms
         ✓ chat completion with parameters 1555ms
         ✓ chat completion stream 653ms

 Test Files  1 passed (1)
      Tests  4 passed | 96 skipped (100)
   Start at  01:20:35
   Duration  4.72s (transform 254ms, setup 9ms, collect 297ms, tests 4.23s, environment 0ms, prepare 54ms)

Thanks for reviewing! 🙏

@V2arK V2arK marked this pull request as draft April 28, 2025 17:09
@julien-c julien-c changed the title Add CentML as an inference provide Add CentML as an inference provider Apr 28, 2025
@HuggingFaceDocBuilderDev

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

@V2arK V2arK marked this pull request as ready for review April 29, 2025 07:24
Copy link
Contributor

@hanouticelina hanouticelina left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, thank you @V2arK for the contribution!
I tested the PR (using the free credits you offer for new users) and it works as expected ✅

@V2arK
Copy link
Author

V2arK commented May 4, 2025

Is there anything else I need to do to get it merged?

@V2arK V2arK requested a review from julien-c May 5, 2025 20:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants