fix: filter Azure-incompatible dynamic MCP tool schemas - #2379
Open
zsxh1990 wants to merge 2 commits into
Open
Conversation
![claude[bot]](https://avatars.githubusercontent.com/in/1236702?s=60&v=4){kind=small}
Contributor
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (2)
🚧 Files skipped from review as they are similar to previous changes (2)
WalkthroughMCP HTTP/SSE and stdio configurations now support ChangesMCP tool filtering
Estimated code review effort: 3 (Moderate) | ~20 minutes Sequence Diagram(s)sequenceDiagram
participant MCPServer
participant RemoteMCPToolset
participant RemoteMCPTool
MCPServer->>RemoteMCPToolset: Return discovered tools
RemoteMCPToolset->>RemoteMCPToolset: Filter excluded and incompatible tools
RemoteMCPToolset->>RemoteMCPTool: Create remaining tools
Possibly related PRs
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
✅ Deploy Preview for holmes-docs ready!
To edit notification comments on pull requests, go to your Netlify project configuration. |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Actionable comments posted: 1
🧹 Nitpick comments (2)
tests/test_mcp_toolset.py (2)
817-817: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low valueMove
_is_azure_incompatible_schemato the module import section.The test methods repeat the same local import. Import the helper once at the top of
tests/test_mcp_toolset.py.
tests/test_mcp_toolset.py#L817-L817: remove the local import.tests/test_mcp_toolset.py#L831-L831: remove the local import.tests/test_mcp_toolset.py#L845-L845: remove the local import.tests/test_mcp_toolset.py#L858-L858: remove the local import.As per coding guidelines: “Always place Python imports at the top of the file, not inside functions or methods.”
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@tests/test_mcp_toolset.py` at line 817, In tests/test_mcp_toolset.py, add the _is_azure_incompatible_schema import to the module-level import section, then remove the repeated local imports at lines 817-817, 831-831, 845-845, and 858-858; no other changes are needed.Source: Coding guidelines
917-926: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low valueUse descriptive test fixture names.
tool1andtool2do not state each fixture role. Use names such asincluded_toolandexcluded_tool.
tests/test_mcp_toolset.py#L917-L926: renametool1andtool2for the configured-exclusion case.tests/test_mcp_toolset.py#L992-L1001: renametool1andtool2for the stdio exclusion case.As per coding guidelines: “Use semantic, descriptive names for variables, functions, and components.”
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@tests/test_mcp_toolset.py` around lines 917 - 926, Rename the generic tool1 and tool2 fixtures to descriptive role-based names in tests/test_mcp_toolset.py lines 917-926 for the configured-exclusion case and lines 992-1001 for the stdio-exclusion case, using names such as included_tool and excluded_tool and updating all references in each test.Source: Coding guidelines
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@holmes/plugins/toolsets/mcp/toolset_mcp.py`:
- Around line 938-946: Update the schema validation logic around required and
properties so non-dictionary properties values are treated as incompatible
before iterating required keys or performing membership checks. Preserve the
existing behavior for missing or invalid required lists, and add a regression
test covering required ["args"] with properties set to null, ensuring one
invalid remote tool does not prevent compatible tools from loading.
---
Nitpick comments:
In `@tests/test_mcp_toolset.py`:
- Line 817: In tests/test_mcp_toolset.py, add the _is_azure_incompatible_schema
import to the module-level import section, then remove the repeated local
imports at lines 817-817, 831-831, 845-845, and 858-858; no other changes are
needed.
- Around line 917-926: Rename the generic tool1 and tool2 fixtures to
descriptive role-based names in tests/test_mcp_toolset.py lines 917-926 for the
configured-exclusion case and lines 992-1001 for the stdio-exclusion case, using
names such as included_tool and excluded_tool and updating all references in
each test.
🪄 Autofix
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro Plus
Run ID: 64327267-7272-4523-985d-1c36a23d47ae
📒 Files selected for processing (2)
holmes/plugins/toolsets/mcp/toolset_mcp.pytests/test_mcp_toolset.py
Some MCP tools define dynamic object arguments (e.g. 'args') where 'required' lists the key but 'properties' is absent. Azure OpenAI rejects the entire MCP server when encountering such schemas, which disables all tools from that server. This commit adds: 1. Auto-detection via _is_azure_incompatible_schema() - logs and excludes tools with schemas Azure rejects 2. excluded_tools config option for manual tool filtering 3. Both features work together for maximum flexibility 4. Comprehensive test coverage (8 new tests) Closes HolmesGPT#2297 Signed-off-by: zsxh1990 <zsxh1990@users.noreply.github.com> Signed-off-by: zsxh1990 <445655361@qq.com>
zsxh1990
force-pushed
the
fix/azure-mcp-schema-filter
branch
from
August 11, 2026 11:56
0776154 to
2557055
Compare
Fixes CodeRabbit review comment on PR HolmesGPT#2379. When a remote tool returns `{"required": ["args"], "properties": null}`, the previous code raised TypeError on the membership test. Now treats non-dictionary properties as incompatible before iterating required keys. Added regression test for null properties case. Signed-off-by: zsxh1990 <445655361@qq.com>
Author
|
Thanks for the review! The if not isinstance(properties, dict):
return TrueWhen |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This PR adds automatic detection and filtering of MCP tool schemas that Azure OpenAI rejects.
Problem
Azure OpenAI requires that all keys in
requiredmust also appear inproperties. Some MCP tools define dynamic object arguments (e.g.,args) whererequiredlists the key butpropertiesis absent. This causes Azure to reject the entire schema, disabling all tools from that MCP server.Solution
Two complementary features:
_is_azure_incompatible_schema()detects problematic schemas and logs a warning while excluding the tool automaticallyexcluded_toolsconfig option for fine-grained control over which tools to loadChanges
holmes/plugins/toolsets/mcp/toolset_mcp.py:_is_azure_incompatible_schema()helper functionexcluded_toolsfield to bothMCPConfigandStdioMCPConfig_load_remote_tools()to filter tools automaticallytests/test_mcp_toolset.py:TestAzureIncompatibleSchemaclass with 8 comprehensive testsExample Config
Test Coverage
Closes #2297
Summary by CodeRabbit
New Features
Bug Fixes