Skip to content

PelicanFS doesn't let you mount to subpaths in a federation #125

@jhiemstrawisc

Description

@jhiemstrawisc

I was recently playing around with PelicanFS when I instantiated a PelicanFileSystem in a way that I expected to work:

>>> pelfs = PelicanFileSystem("pelican://osg-htc.org/ospool/ap40/justin.hiemstra")
>>> pelfs.ls("/")
DEBUG:fsspec.pelican:Ensuring that / is a pelican compatible path...
DEBUG:fsspec.pelican:Compatible path: /
DEBUG:fsspec.pelican:Finding the collections endpoint for /...
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/jhiemstra/.local/share/mamba/envs/test-pelicanfs/lib/python3.12/site-packages/fsspec/asyn.py", line 118, in wrapper
    return sync(self.loop, func, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/jhiemstra/.local/share/mamba/envs/test-pelicanfs/lib/python3.12/site-packages/fsspec/asyn.py", line 103, in sync
    raise return_result
  File "/home/jhiemstra/.local/share/mamba/envs/test-pelicanfs/lib/python3.12/site-packages/fsspec/asyn.py", line 56, in _runner
    result[0] = await coro
                ^^^^^^^^^^
  File "/home/jhiemstra/snakemake-executor-dev/pelicanfs/src/pelicanfs/core.py", line 704, in wrapper
    data_url, director_response = await self.get_dirlist_url(path)
                                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/jhiemstra/snakemake-executor-dev/pelicanfs/src/pelicanfs/core.py", line 630, in get_dirlist_url
    raise BadDirectorResponse()
pelicanfs.exceptions.BadDirectorResponse

I think the issue is that a PelicanFileSystem expects only the pelican://osg-htc.org bit, not the additional path. However, it feels like I as a user should be able to instantiate the "filesystem" at a "sub directory" like I tried to do in this example, such that any pelfs.<operation>("/some/path") is relative to the path I provided when I set things up.

I'm not sure whether this is a feature request or a bug report because I don't know what was intended.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions