Skip to content

Conversation

Wauplin
Copy link
Contributor

@Wauplin Wauplin commented May 21, 2025

Followed implementation from huggingface/huggingface.js#1422

cc @evalstate if you can have a look at it

How to test

import os

from huggingface_hub import ChatCompletionInputMessage, ChatCompletionStreamOutput, MCPClient


async def main():
    async with MCPClient(
        provider="nebius",
        model="Qwen/Qwen2.5-72B-Instruct",
        api_key=os.environ["HF_TOKEN"],
    ) as client:
        await client.add_mcp_server(type="sse", url="https://evalstate-flux1-schnell.hf.space/gradio_api/mcp/sse")

        messages = [
            {
                "role": "user",
                "content": "Generate a picture of a cat on the moon",
            }
        ]

        async for chunk in client.process_single_turn_with_tools(messages):
            # Log messages
            if isinstance(chunk, ChatCompletionStreamOutput):
                delta = chunk.choices[0].delta
                if delta.content:
                    print(delta.content, end="")

            # Or tool calls
            elif isinstance(chunk, ChatCompletionInputMessage):
                print(
                    f"\nCalled tool '{chunk.name}'. Result: '{chunk.content if len(chunk.content) < 1000 else chunk.content[:1000] + '...'}'"
                )


if __name__ == "__main__":
    import asyncio

    asyncio.run(main())

Output

Called tool 'evalstate_flux1_schnell_infer'. Result: '[Binary Content: Image image/webp, 38474 bytes]
The task is complete and the content accessible to the User
Image URL: https://evalstate-flux1-schnell.hf.space/gradio_api/file=/tmp/gradio/c178b5c1aae663d896b21cf6d0090e8e868e8a87e2d89b272243a9f135faf3ad/image.webp
42'

@Wauplin Wauplin requested review from julien-c and hanouticelina May 21, 2025 09:46
@hanouticelina hanouticelina mentioned this pull request May 21, 2025
2 tasks
Copy link
Contributor

@hanouticelina hanouticelina left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Very nice and clean! thank you

@HuggingFaceDocBuilderDev

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

async def add_mcp_server(self, type: Literal["sse"], **params: Unpack[SSEServerParameters_T]): ...

@overload
async def add_mcp_server(self, type: Literal["streamablehttp"], **params: Unpack[StreamableHTTPParameters_T]): ...
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
async def add_mcp_server(self, type: Literal["streamablehttp"], **params: Unpack[StreamableHTTPParameters_T]): ...
async def add_mcp_server(self, type: Literal["http"], **params: Unpack[StreamableHTTPParameters_T]): ...

in the JS implem we've just used "http", and i think i've seen in some clients (Copilot config for instance) they went for just "http" too.

As it's expected to become the main method, i would keep it super short and simple

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for noticing. Indeed much better to have "http". Addressed in 25726d8

@Wauplin Wauplin merged commit 10d4494 into main May 21, 2025
25 checks passed
@Wauplin Wauplin deleted the mcp-sse-http-support branch May 21, 2025 17:30
@evalstate
Copy link
Collaborator

I was in the air while this happened (saw the email) @Wauplin don't think you can make that any shorter!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants