Skip to content

Conversation

@jpshackelford
Copy link
Contributor

@jpshackelford jpshackelford commented Jan 8, 2026

Summary

Implements #1645 - adds the ability to fetch plugins from remote sources (GitHub repositories, git URLs) and cache them locally. This is a building block for the Plugin Directory feature (OpenHands/OpenHands#12088).

Changes

New Function: Plugin.fetch()

@classmethod
def fetch(
    cls,
    source: str,
    cache_dir: Path | None = None,
    ref: str | None = None,
    update: bool = True,
) -> Path:
    """Fetch a plugin from a remote source and return the local cached path."""

Source Parsing

The parse_plugin_source() function handles various source formats:

  • "github:owner/repo" - GitHub repository shorthand
  • "https://github.com/owner/repo.git" - Full git URL
  • "[email protected]:owner/repo.git" - SSH git URL
  • "/local/path" - Local path (returned as-is)

Caching Strategy

  • Cache location: ~/.openhands/cache/plugins/{name}-{hash}/
  • If cached and exists: optionally update (git fetch) or use as-is
  • If not cached: clone the repository
  • Supports shallow clones for efficiency (--depth 1)

Usage Example

# Fetch and load a plugin
path = Plugin.fetch("github:owner/my-plugin")
plugin = Plugin.load(path)

# With specific version
path = Plugin.fetch("github:owner/my-plugin", ref="v1.0.0")
plugin = Plugin.load(path)

# Fetch and load in one step
plugin = Plugin.load(Plugin.fetch("github:owner/my-plugin"))

Files Changed

  • openhands-sdk/openhands/sdk/plugin/fetch.py - New module with fetching logic
  • openhands-sdk/openhands/sdk/plugin/plugin.py - Added fetch() classmethod
  • openhands-sdk/openhands/sdk/plugin/__init__.py - Export new types
  • tests/sdk/plugin/test_plugin_fetch.py - 34 new unit tests

Checklist

  • Plugin.fetch("github:owner/repo") clones and returns local path
  • Plugin.fetch("https://...") works with any git URL
  • Plugin.fetch("/local/path") returns path unchanged
  • Caching works - second fetch uses cached version
  • ref parameter checks out specific branch/tag
  • Errors are handled gracefully with PluginFetchError
  • Unit tests cover all source types and caching behavior (34 tests)

Related Issues

@jpshackelford can click here to continue refining the PR


Agent Server images for this PR

GHCR package: https://github.com/OpenHands/agent-sdk/pkgs/container/agent-server

Variants & Base Images

Variant Architectures Base Image Docs / Tags
java amd64, arm64 eclipse-temurin:17-jdk Link
python amd64, arm64 nikolaik/python-nodejs:python3.12-nodejs22 Link
golang amd64, arm64 golang:1.21-bookworm Link

Pull (multi-arch manifest)

# Each variant is a multi-arch manifest supporting both amd64 and arm64
docker pull ghcr.io/openhands/agent-server:4b4b564-python

Run

docker run -it --rm \
  -p 8000:8000 \
  --name agent-server-4b4b564-python \
  ghcr.io/openhands/agent-server:4b4b564-python

All tags pushed for this build

ghcr.io/openhands/agent-server:4b4b564-golang-amd64
ghcr.io/openhands/agent-server:4b4b564-golang_tag_1.21-bookworm-amd64
ghcr.io/openhands/agent-server:4b4b564-golang-arm64
ghcr.io/openhands/agent-server:4b4b564-golang_tag_1.21-bookworm-arm64
ghcr.io/openhands/agent-server:4b4b564-java-amd64
ghcr.io/openhands/agent-server:4b4b564-eclipse-temurin_tag_17-jdk-amd64
ghcr.io/openhands/agent-server:4b4b564-java-arm64
ghcr.io/openhands/agent-server:4b4b564-eclipse-temurin_tag_17-jdk-arm64
ghcr.io/openhands/agent-server:4b4b564-python-amd64
ghcr.io/openhands/agent-server:4b4b564-nikolaik_s_python-nodejs_tag_python3.12-nodejs22-amd64
ghcr.io/openhands/agent-server:4b4b564-python-arm64
ghcr.io/openhands/agent-server:4b4b564-nikolaik_s_python-nodejs_tag_python3.12-nodejs22-arm64
ghcr.io/openhands/agent-server:4b4b564-golang
ghcr.io/openhands/agent-server:4b4b564-java
ghcr.io/openhands/agent-server:4b4b564-python

About Multi-Architecture Support

  • Each variant tag (e.g., 4b4b564-python) is a multi-arch manifest supporting both amd64 and arm64
  • Docker automatically pulls the correct architecture for your platform
  • Individual architecture tags (e.g., 4b4b564-python-amd64) are also available if needed

Implements #1645 - adds the ability to fetch plugins from remote sources
(GitHub repositories, git URLs) and cache them locally.

Changes:
- Add Plugin.fetch() classmethod to fetch from remote sources
- Add parse_plugin_source() to parse various source formats:
  - GitHub shorthand: 'github:owner/repo'
  - Git URLs: HTTPS, SSH, git:// protocol
  - Local paths (returned as-is)
- Add PluginFetchError exception for fetch failures
- Implement caching at ~/.openhands/cache/plugins/
- Support shallow clones for efficiency
- Support specific ref (branch/tag/commit) checkout
- Add comprehensive unit tests (34 new tests)

Co-authored-by: openhands <[email protected]>
@github-actions
Copy link
Contributor

github-actions bot commented Jan 8, 2026

Coverage

Coverage Report •
FileStmtsMissCoverMissing
openhands/sdk/plugin
   plugin.py1552285%274–276, 278–283, 299–302, 311–315, 331–332, 350–351
TOTAL15851488069% 

- Add blank line after imports in fetch.py
- Remove unused import PluginFetchError in plugin.py
- Reformat long lines in test file

Co-authored-by: openhands <[email protected]>
- Create GitHelper class to encapsulate all git operations (clone, fetch, checkout, etc.)
- Refactor fetch.py to use GitHelper via dependency injection
- Update unit tests to mock GitHelper instead of subprocess.run
- Add integration tests for real git operations
- Add support for file:// URLs for local testing

This improves test coverage by:
1. Allowing unit tests to execute actual fetch.py logic while mocking git operations
2. Providing separate integration tests that use real git operations

Co-authored-by: openhands <[email protected]>
openhands-agent and others added 6 commits January 8, 2026 21:36
Add comprehensive tests to improve coverage for the plugin module:

- fetch.py: Add tests for set_git_helper(), relative path parsing,
  default cache_dir fallback, and PluginFetchError re-raise
- git_helper.py: Add unit tests for error handling paths including
  CalledProcessError and TimeoutExpired for all git operations
- plugin.py: Add tests for plugin loading edge cases including
  manifest parsing, skills/agents/commands loading, and error handling
- types.py: Add tests for AgentDefinition and CommandDefinition
  frontmatter parsing including complex field handling

Simplify integration test to avoid skill loading complexity.

Co-authored-by: openhands <[email protected]>
The --forked flag was preventing coverage from being collected properly
because pytest-forked uses os.fork() which doesn't combine coverage data.

Changes:
- Split SDK test run: forked tests run separately, then non-forked tests
  with --cov-append to combine coverage data
- Add 'forked' pytest marker to identify tests needing process isolation
- Add coverage configuration with parallel mode and branch coverage
- Add coverage report exclusions for common non-coverable patterns

This allows most tests to run without forking for accurate coverage,
while still supporting tests that need process isolation.

Co-authored-by: openhands <[email protected]>
The change to run tests without --forked exposed pre-existing test pollution
issues in test_state_serialization.py where tests share state incorrectly.

Reverting to the original --forked approach until those tests are fixed.
Coverage collection will remain affected by --forked, but tests will pass.

Co-authored-by: openhands <[email protected]>
Copy link
Contributor Author

Requested Changes: Add subpath parameter

Context

During testing, we identified a bug where plugin paths (e.g., github:owner/repo/plugins/sub-plugin) were being rejected by parse_plugin_source() because it validates that GitHub shorthand has only one / in the repo path. Rather than embedding the path in the source string, we've decided to keep path as a separate parameter throughout the entire stack for consistency and clarity.

Changes Needed

openhands-sdk/openhands/sdk/plugin/fetch.py - Add subpath parameter to fetch_plugin():

def fetch_plugin(
    source: str,
    cache_dir: Path | None = None,
    ref: str | None = None,
    update: bool = True,
    subpath: str | None = None,  # NEW PARAMETER
    git_helper: GitHelper | None = None,
) -> Path:
    """Fetch a plugin from a remote source and return the local cached path.

    Args:
        source: Plugin source - "github:owner/repo", git URL, or local path
        cache_dir: Directory for caching
        ref: Optional branch, tag, or commit
        update: If True, update cached repo
        subpath: Optional subdirectory path within the repo
        git_helper: GitHelper instance

    Returns:
        Path to the plugin directory (with subpath applied if specified)
    """
    source_type, url = parse_plugin_source(source)
    
    # ... existing clone/update logic ...
    
    # NEW: Apply subpath if specified (add before final return)
    if subpath:
        final_path = plugin_path / subpath.strip('/')
        if not final_path.exists():
            raise PluginFetchError(
                f"Subdirectory '{subpath}' not found in plugin repository"
            )
        return final_path
    
    return plugin_path

openhands-sdk/openhands/sdk/plugin/plugin.py - Add subpath parameter to Plugin.fetch():

@classmethod
def fetch(
    cls,
    source: str,
    cache_dir: Path | None = None,
    ref: str | None = None,
    update: bool = True,
    subpath: str | None = None,  # NEW PARAMETER
) -> Path:
    """Fetch a plugin from a remote source."""
    return fetch_plugin(
        source, 
        cache_dir=cache_dir, 
        ref=ref, 
        update=update,
        subpath=subpath,  # NEW
    )

This maintains API shape consistency across all layers (see OpenHands/OpenHands#12087 (comment) for the full API design).

jpshackelford pushed a commit that referenced this pull request Jan 12, 2026
Add plugin_path field to StartConversationRequest and pass it as
subpath parameter to Plugin.fetch(). This enables fetching plugins
from subdirectories within repositories (e.g., monorepos with
multiple plugins).

Changes:
- models.py: Add plugin_path optional field to StartConversationRequest
- conversation_service.py: Pass plugin_path as subpath to Plugin.fetch()
- test_conversation_service.py: Update tests to verify subpath handling

Note: This depends on PR #1647 adding the subpath parameter to Plugin.fetch()

Co-authored-by: openhands <[email protected]>
Add optional subpath parameter to both fetch_plugin() and Plugin.fetch()
methods. This allows fetching plugins from subdirectories within a
repository, which is needed when plugin paths are specified separately
rather than embedded in the source string.

Changes:
- Add subpath parameter to fetch_plugin() in fetch.py
- Add subpath parameter to Plugin.fetch() in plugin.py
- Apply subpath to both local paths and remote repositories
- Strip leading/trailing slashes from subpath
- Raise PluginFetchError if subpath doesn't exist
- Add 8 new unit tests for subpath functionality

Addresses review comment on PR #1647.

Co-authored-by: openhands <[email protected]>
@openhands-ai
Copy link

openhands-ai bot commented Jan 12, 2026

Looks like there are a few issues preventing this PR from being merged!

  • GitHub Actions are failing:
    • Pre-commit checks

If you'd like me to help, just leave a comment, like

@OpenHands please fix the failing actions on PR #1647 at branch `feat/plugin-fetch`

Feel free to include any additional details that might help me get this PR into a better state.

You can manage your notification settings

jpshackelford pushed a commit that referenced this pull request Jan 12, 2026
Add plugin_path field to StartConversationRequest and pass it as
subpath parameter to Plugin.fetch(). This enables fetching plugins
from subdirectories within repositories (e.g., monorepos with
multiple plugins).

Changes:
- models.py: Add plugin_path optional field to StartConversationRequest
- conversation_service.py: Pass plugin_path as subpath to Plugin.fetch()
- test_conversation_service.py: Update tests to verify subpath handling

Note: This depends on PR #1647 adding the subpath parameter to Plugin.fetch()

Co-authored-by: openhands <[email protected]>


class GitHelper:
"""Abstraction for git operations, enabling easy mocking in tests."""
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we already have git clone in the codebase?

raise GitError(f"Fetch timed out after {timeout}s") from e

def checkout(self, repo_path: Path, ref: str, timeout: int = 30) -> None:
"""Checkout a ref (branch, tag, or commit).
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we already needed to do git checkout, so I wonder if this code duplicates it. Maybe in the workspace package?

I believe we checkout:

  • user's repo (I think this is a workspace thing)
  • organization skills repo, e.g. github/acme/.openhands/skills or so

This kind of feature feels like the second, like checking out a repo for this user/org from some remote?

except subprocess.TimeoutExpired as e:
raise GitError(f"Reset timed out after {timeout}s") from e

def get_current_branch(self, repo_path: Path, timeout: int = 10) -> str | None:
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ditto, we must have this code already, is this a duplicate?

]

# Coverage configuration - needed for --forked tests to collect coverage data
[tool.coverage.run]
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This doesn't seem part of this PR, maybe it could be in a separate PR if you'd like it in?

@enyst
Copy link
Collaborator

enyst commented Jan 12, 2026

@OpenHands Install yourself gh if you don't already have it, and then get all this PR diff and the inline comments to it.

GOAL: answer a code design question: does this feature implementation belong in the sdk package of the SDK (like this PR), in the workspace package of the SDK (Workspace classes), or in the client apps?

I want you to make a throughout research in the SDK packages and try to understand the git-related operations and responsibilities. Answer directly in a github comment.

@openhands-ai
Copy link

openhands-ai bot commented Jan 12, 2026

I'm on it! enyst can track my progress at all-hands.dev

Copy link
Collaborator

enyst commented Jan 12, 2026

Thanks for flagging this — I dug through the SDK packages to see where “git-ish” responsibilities currently live.

What exists today

SDK (openhands-sdk) already does small, self-contained git + caching for SDK-level assets.

  • openhands.sdk.context.skills.utils.update_skills_repository() maintains a local clone under ~/.openhands/cache/skills/... and does git clone/fetch/reset via subprocess.
    • This is not workspace-scoped: it’s a pure “fetch a read-only dependency into a user cache” helper.

Workspace (openhands-workspace) is about running commands / managing environments, not about bringing in additional repos into ~/.openhands caches.

  • The workspace layer exposes git introspection APIs (git_changes, git_diff) for the repo inside the workspace, but it does not implement git clone / remote fetching logic. It’s basically “operate on whatever repo is already in the workspace”.

So where should Plugin.fetch() live?

Assessment: Plugin.fetch() belongs in openhands-sdk (the sdk package), but ideally as a shared “remote repo fetch + cache” utility used by both skills + plugins, not in Workspace or client apps.

Why SDK (not Workspace):

  • The feature is “resolve a plugin source string → ensure a local cached copy exists → return a filesystem path”. That mirrors the existing public-skills behavior and is independent of any particular workspace implementation (Docker, API remote, cloud, etc).
  • The cache location is under the user’s home (~/.openhands/cache/...), which is outside the workspace’s conceptual boundary. Workspaces should generally avoid mutating host-level caches (especially for remote workspaces).
  • Workspace APIs today are about a repo that already exists in the workspace (diff/changes), not “bring external dependencies into existence”.

Why not client apps:

  • If every client re-implements “git clone with caching + ref checkout + update semantics”, we’ll end up with duplicated, inconsistent behavior. This is a reusable SDK primitive (plugins, skills registry, future org registries, etc).
  • Keeping it in the SDK makes it testable and available to multiple clients.

Design suggestion (to reduce duplication)

Right now the PR introduces openhands.sdk.plugin.git_helper.GitHelper while openhands.sdk.git.utils and skills/utils.py already contain git wrappers.

I’d recommend unifying these as:

  1. a single openhands.sdk.git abstraction for both "git introspection" and "git fetch/clone" operations, and
  2. small domain wrappers:
    • skills calls something like sdk.git.cached_clone_or_update(...)
    • plugin calls the same and adds “parse plugin source → choose cache key/name”.

That keeps git operations centralized and avoids the “we have 2-3 ways to call git” situation.

@openhands-ai
Copy link

openhands-ai bot commented Jan 12, 2026

Summary of work completed

  • Installed the GitHub CLI (gh) in the environment (it was not present).
  • Retrieved PR feat(plugin): Add Plugin.fetch() for remote plugin fetching and caching #1647 metadata and the full diff (gh pr diff 1647 …) and reviewed the implementation (Plugin.fetch + parsing + caching + new git helper).
  • Pulled all PR-level comments (gh pr view --comments) and all inline review comments via the GitHub API endpoint for pull request review comments (/pulls/1647/comments), and reviewed them (notably the concern about duplicating existing git clone/checkout logic and whether this belongs in workspace).
  • Performed a repo-wide audit of existing git responsibilities across packages:
    • Found existing SDK git operations and caching behavior in openhands-sdk, specifically openhands.sdk.context.skills.utils.update_skills_repository() which clones/fetches/resets into ~/.openhands/cache/skills.
    • Confirmed openhands-workspace focuses on workspace lifecycle + executing commands and does not implement remote clone/fetch-to-cache logic; SDK workspace exposes git introspection APIs (git_changes, git_diff) for repos already present.
    • Reviewed openhands.sdk.git.* utilities (safe git command wrapper, diff/changes helpers) to understand current patterns.
  • Posted a GitHub PR comment answering the design question directly:
    • Conclusion: Plugin.fetch() belongs in openhands-sdk, not Workspace or client apps.
    • Additional recommendation: reduce duplication by centralizing “clone/fetch/update cached repo” primitives under openhands.sdk.git and have both skills and plugin use that shared utility.

Checklist vs user request

  • Install gh if missing
  • Get PR diff
  • Get inline comments
  • Research SDK packages for git-related responsibilities
  • Answer the design question directly in a GitHub comment

Code changes / pushes

  • No repository code changes were made.
  • No commits were created.
  • Nothing to push.

View full conversation

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat(plugin): Add Plugin.fetch() for remote plugin fetching and caching

4 participants