Track and unregister loaders in tests manually

[katafrakt]

Changes in #292 turned out to not be enough. While they removed the error of ObjectSpace not being enabled, it still did not work correctly, presumably because it was enabled too late. As a result, ObjectSpace.each_object(Zeitwerk::Loader) was yielding empty set and the loader were not unregistered successfully, leading to errors like this:

     6.1) Failure/Error: loader.push_dir @dir.join("lib", "ns").realpath, namespace: MyNamespace
          
          Zeitwerk::Error:
            loader
          
            #<Zeitwerk::Loader:0x3206174f
             @autoloaded_dirs=[],
             @autoloads={},
             @collapse_dirs=#<Set: {}>,
             @collapse_glob_patterns=#<Set: {}>,
             @dirs_autoload_monitor=#<Monitor:0x795f5d51>,
             @eager_load_exclusions=#<Set: {}>,
             @eager_loaded=false,
             @ignored_glob_patterns=#<Set: {}>,
             @ignored_paths=#<Set: {}>,
             @inceptions=
              #<Zeitwerk::Cref::Map:0x43aeb5e0
               @map={},
               @mutex=#<Thread::Mutex:0x65383667>>,
             @inflector=#<Zeitwerk::Inflector:0x63cd2cd2>,
             @initialized_at=2025-12-04 15:50:21.900213 +0100,
             @logger=nil,
             @mutex=#<Thread::Mutex:0x6deee370>,
             @namespace_dirs=
              #<Zeitwerk::Cref::Map:0x49c17ba4
               @map={},
               @mutex=#<Thread::Mutex:0x423c5404>>,
             @on_load_callbacks={},
             @on_setup_callbacks=[],
             @on_unload_callbacks={},
             @reloading_enabled=false,
             @roots={},
             @setup=false,
             @shadowed_files=#<Set: {}>,
             @tag="1c773a",
             @to_unload={}>
          
          
            wants to manage directory /tmp/d20251204-104206-b05t9p/lib/ns, which is already managed by
          
            #<Zeitwerk::Loader:0xb112b13
             @autoloaded_dirs=["/tmp/d20251204-104206-b05t9p/lib/ns/nested"],
             @autoloads={},
             @collapse_dirs=#<Set: {}>,
             @collapse_glob_patterns=#<Set: {}>,
             @dirs_autoload_monitor=#<Monitor:0x5fbe155>,
             @eager_load_exclusions=#<Set: {}>,
             @eager_loaded=false,
             @ignored_glob_patterns=#<Set: {}>,
             @ignored_paths=#<Set: {}>,
             @inceptions=
              #<Zeitwerk::Cref::Map:0x1386313f
               @map={},
               @mutex=#<Thread::Mutex:0x433c6abb>>,
             @inflector=#<Zeitwerk::Inflector:0x288f173f @overrides={}>,
             @initialized_at=2025-12-04 15:50:21.824222 +0100,
             @logger=nil,
             @mutex=#<Thread::Mutex:0x35e26d05>,
             @namespace_dirs=
              #<Zeitwerk::Cref::Map:0x29fa6b65
               @map={},
               @mutex=#<Thread::Mutex:0x47406941>>,
             @on_load_callbacks={},
             @on_setup_callbacks=[],
             @on_unload_callbacks={},
             @reloading_enabled=false,
             @roots={"/tmp/d20251204-104206-b05t9p/lib/ns" => MyNamespace},
             @setup=true,
             @shadowed_files=#<Set: {}>,
             @tag="1013b4",
             @to_unload={}>
          # ./spec/integration/container/auto_registration/component_dir_namespaces/autoloading_loader_spec.rb:111:in 'block in <main>'

This replaces ZeitwerkHelpers with ZeitwerkLoaderRegistry, which keeps track of created loaders and allows to unregister them. This is used in tests when we need to have multiple loaders but cannot allow them to interfere with each other.

This replaces former solutions to the problem:

• Using Zeitwerk::Loaders - it's a private API, not guaranteed to
  persist
• Using ObjectSpace - this does not work well in JRuby

[timriley]

Thank you for persisting with this, @katafrakt!

We actually did use Zeitwerk::Registry.loaders here in the past, but @fxn encouraged we take a different approach, because Zeitwerk::Registry is private API and may change at any time.

See #259 and #260 for that discussion.

The last thing I want is to make @fxn unhappy (😆), so if ObjectSpace is not easy for us to use under JRuby, I wonder if there's a third option we could take here? Is there a way we could track our own loaders more clearly, e.g. our own version of Zeitwerk::Registry? We could e.g. assign all the loaders we create to a Dry::System::Plugins::Zeitwerk.loaders (or similar) for the purposes of managing them during tests? Maybe that could be something we add just during the execution of our tests?

[fxn]

That is the idea.

Zeitwerk does not have interface to fetch loaders because loaders are supposed to be private to gems, I cannot expose and bless accessing them.

But a project that needs to know about its loaders can have its own registry.

[katafrakt]

Thank you for some historical context @timriley and for confirming the direction @fxn. I created our own loaders registry for tests, which tracks the loaders and allows to unregister them. No change to non-test code was necessary after all.

[timriley]

Thanks @katafrakt! Looks like we're heading in the right direction now. I did have one question for you, though. It seems like we may be missing the cleanup of the default loaders created by the plugin, for the times when we're not passing them in explicitly?

[timriley]

Do the loaders created in this example group get torn down? Here we're letting the plugin create the loader itself (which is just a plain old ::Zeitwerk::Loader.new), rather than assigning a loader that we've created ourselves via ZeitwerkLoaderRegistry.new_loader.

(This question applies to any of the spec files that follow a similar approach to this one)

[katafrakt]

Hmm, good point. One thing to remedy this could be to just stub Zeitwerk::Loader.new in tests with ZeitwerkLoaderRegistry.new_loader, but I'm not sure how we feel about this.

[timriley]

One thing to remedy this could be to just stub Zeitwerk::Loader.new in tests with ZeitwerkLoaderRegistry.new_loader, but I'm not sure how we feel about this.

I think this is no worse than what we were doing to iterate over ObjectSpace and find the classes we want. So I think it's a reasonable move at this point just to keep things us moving forward and unlock JRuby support.

We should at least file an issue about finding a cleaner way of doing this.