Wind Waker: Support for Universal Tracker.

[lonegraywolf2000]

What is this fixing or adding?

This adds the following features for Universal Tracker:

• Sequence break support: UT now recognizes which checks you can get out of sequence without completely breaking the game. Only the checks involving Levels 1-3 of Obscure and Precision are in here: Barrier Skip is not considered a valid sequence break. A unit test demonstrating the basics of how UT interprets sequence breaks is provided.
• YAML-less Generation: We now have the ability to fully parse the options from the original YAML file and fill the slots as appropriate.
• Entrance Shuffle Support: You are now told what rooms have items that you can access. If you are unsure of the path, use /get_logical_path or /explain in UT.
• Opt-In Deferred Entrance Support: We parse the game's memory to temporarily disconnect entrances in UT. At the bottom of the checks you know you have access to, you will be told of entrances you can access but haven't yet. Once you access them for the first time, the entrances are connected and the check lists behave as before. If you do not wish to have deferred entrances in your games, you can always set your host.yaml file to exclude them.

How was this tested?

Multiple seed generations were created, with each seed being loaded into both Universal Tracker and Dolphin for extensive comparing and contrasting.

Logs were previously present to ensure the data coming through was correct, but these were removed for the production push.

The unit test for sequence break support is based off of one I wrote for the Ship of Harkinian APWorld, and that was verified to work.

If this makes graphical changes, please attach screenshots.

While I didn't make a graphical change directly, I do want to show the result of dungeon entrance shuffle in Universal Tracker.

Additional credits go here I guess.

• Faris for making Universal Tracker.
• Tanjo for allowing me to give this a shot.
• lx5 for confirming the missing link to enable deferred entrance shuffle.

[Mysteryem]

There are some changes that need to be made, a question about datastorage keys, and a thought about the LogicMixin.

I have not yet had a deeper look at the actual implementation of unrandomizing random options from slot data, or the new tests.

From running the UT fuzzer hook I didn't see any UT-related failures.

[Mysteryem]

The PopTracker pack can already connect entrances using just the list of visited stages, why are all these new datastorage keys needed?

[lonegraywolf2000]

I was following the example of Waffles: https://github.com/TheLX5/Archipelago/blob/waffles/worlds/waffles/Client.py#L458

[Mysteryem]

That doesn't really answer my question. Is there some technical requirement of Universal Tracker for these new datastorage keys to be required?

Either way, the client shouldn't be duplicating the same visited stages data across both the existing list datastorage key and all the new individual datastorage keys, so one of the two should be removed, though it is not clear to me which one should be removed currently.

[lonegraywolf2000]

Based on earlier Discord discussion, we can hopefully resolve this one if the original message sent also contains room for the (mostly unused by all except devs) team parameter. How easy is it to update poptracker to account for that?

[Mysteryem]

Updating the PopTracker pack to change the datastorage key to include the team is like a 3 line change or so, it's very little work.

[lonegraywolf2000]

Unless I hear otherwise, I'll presume you will go for tww_visited_stages_%i_%i then.

[Mysteryem]

Ultimately, the tracker pack has to follow the client, so decide on the new datastorage key's format, and the poptracker pack will follow.

[lonegraywolf2000]

Implemented on latest commit.

[Mysteryem]

These are not relevant to logic, so do not need to be included in slot data for UT support.

[lonegraywolf2000]

Possible misunderstanding here, but why would these not be relevant to logic? The location of the keys, at least, kind of determines which places you can go, right?

[Mysteryem]

All these options do is control where the keys get placed in the multiworld. The keys are real AP items and not events, so Universal Tracker discards them from its internal generation. When the key items get received from the multiworld, then Universal Tracker's logic updates to account for the keys.

[Mysteryem]

These are not relevant to logic, so do not need to be included in slot data UT support.

In the case of num_required_bosses, generation with UT can calculate the count based on the number of required bosses enabled that this PR is putting into slot data.

[Mysteryem]

These are not relevant to logic, so do not need to be included in slot data.

In the case of mix_entrances, generation with UT can just assume that entrances are always mixed, because all non-mixed ER combinations are a subset of mixed ER combinations.

In the case of randomize_charts, unless you're intending to add deferred entrance support for randomized charts, where entrances would only be connected when the player opens the chart in-game, generation with UT can just assume that charts are always randomized, because non-randomized charts is a possible randomized charts combination.

[lonegraywolf2000]

I debated deferred entrance support for randomized charts, but considering that you can either easily open up the charts in a few seconds or a future option to just have them pre-opened, I didn't feel that was worth it.

Mix entrances, I get.

Randomize Starting Island, I am torn. If you don't start with the Swift Sail or Ballad of Gales, and get placed in an island with no accessible checks, doesn't that cause a problem?

[Mysteryem]

Randomize Starting Island does not affect the world's logic in any way.

Playing without the Swift Sail is not really recommended, but even if you don't have the Swift Sail enabled, you can still slowly cruise across the ocean like when you don't have the sail out.

[Mysteryem]

"required_bosses" is already in use for the Required Bosses option, please use a different name.

[Mysteryem]

I don't really like slowing down regular generation just to add Universal Tracker support.

I'm not sure if there is a good solution to this, but it might be possible to, when generating with UT, replace the _tww_obscure/precise_1/2/3 methods, that would normally get added onto CollectionState through the TWWLogic LogicMixin's AutoLogicRegister metaclass, with UT-specific versions that include the or self.has("Glitched", player, 1) checks (which can just be or self.has("Glitched", player) because 1 is the default count).

[lonegraywolf2000]

I did that pattern similar to what was done for Ship of Harkinian, and I was explicitly suggested to place it in the trick section. https://github.com/lonegraywolf2000/Archipelago-SoH/blob/oot-soh/worlds/oot_soh/LogicHelpers.py#L387

[Mysteryem]

If they want to put up with their regular generation being slower just to support UT glitched logic, then that's up to them.

Ideally, there would be a better way where UT glitched logic is only checked when generation is occurring under UT. Depending on how worlds have set up their logic, separating out the UT glitched logic functions could be easier or more difficult.

If there is a way to support UT glitched logic without slowing down regular generations, and without being highly complicated, then I think it would be a good thing to do.

This is similar to the idea of making logic rules that never needs to check what Options are enabled because different logic is set depending on what the enabled Options are.

[lonegraywolf2000]

Based on earlier discord discussions, I may want to see how you handle things in the future with Banjo Tooie before bringing it over.

[Mysteryem]

Banjo Tooie isn't really comparable to Wind Waker because they set up their logic differently. In Banjo Tooie's case, their logic is all defined through methods on a specific class that the world creates an instance of. This means that a subclass of this class can exist for UT generation which overrides the methods that check logic difficulty to additionally check for the UT 'glitched' item. This means that generation with UT can just swap out the normal class for the UT-specific class. jjjj12212#3

Wind Waker, however, does not define its logic on a class, and the specific part that checks the logic difficulty is done through a LogicMixin, which is a special class that has its methods 'mixed into' the CollectionState class of Core AP. The actual class itself isn't used during generation, so subclassing it is useless.

What I think can be done for Wind Waker with Universal Tracker, is to define a new LogicMixin class when generation with Universal Tracker starts, to replace some of the methods in Wind Waker's normal LogicMixin to versions that check for UT's 'glitched' item. The important thing is that this UT-specific LogicMixin class must never be defined unless it is known that generation is occurring with Universal Tracker.

An alternative to defining a new LogicMixin class at runtime could be manually setting replacement methods onto the CollectionState class (basically manually implementing the behaviour of defining a LogicMixin class), but I feel like that would be slightly more prone to breaking if AP were to change how LogicMixins work in the future.

[lonegraywolf2000]

I think I understand what you are saying here. I may as well confirm something. Are you allowing a refactor of the rules to go to a fully class approach here that may remove LogicMixins, or do you still wish to keep the LogicMixin approach? I don't want to jump the gun again with programming.

[Mysteryem]

Huh? I'm not saying any refactor should be done. Banjo Tooie and TWW set up their logic differently to one another, and that doesn't need to change.

I'm saying that when generation is occurring under Universal Tracker, I think it should work for the TWW world to define a new, UT-specific, LogicMixin subclass that causes the _tww_obscure/precise_# methods to be replaced with new methods that additionally check for the UT 'glitched' item.

In generate_early, or maybe even __init__ if it is possible to detect that generation is occurring under UT at that point, the TWW world would define a new subclass of LogicMixin with each of the _tww_obscure/precise_# methods that include checking the UT 'glitched' item, causing the pre-existing mixed-in methods with the same names from TWW's normal LogicMixin subclass to be replaced with the new methods.

That way, TWW would use the methods from its normal LogicMixin in normal generation, and only when generating with Universal Tracker would the Universal Tracker versions of _tww_obscure/precise_# get defined and replace the normal versions of those methods.

[Mysteryem]

Ah, damn, I looked at LogicMixin's metaclass and it will error when attempting to define a new LogicMixin with some of the same methods defined, so I guess the replacing of the mixed-in methods on the CollectionState would have to be done manually.

[Mysteryem]

I haven't tested it, but my thought was something like this. The main problem I can think of is that your test that enables the glitched item won't work any more, and I wouldn't recommend calling mix_in_universal_tracker_logic() from within tests because it makes permanent changes to the CollectionState class for the currently running Python process.

A small test that the glitched logic is working could probably instead be added within mix_in_universal_tracker_logic(), if mix_in_universal_tracker_logic() is updated to take the TWWWorld instance as an argument, so that a CollectionState can be created and tested against. The logic_obscure/precise_1/2/3 attributes on the TWWWorld class would probably also need to be changed to default to False for a test to work because they are currently only defined on the instance at the end of generate_early(), so testing against the mixed-in methods would raise an AttributeError due to the attributes not being defined on the TWWWorld instance yet.

Index: worlds/tww/Rules.py
IDEA additional info:
Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP
<+>UTF-8
===================================================================
diff --git a/worlds/tww/Rules.py b/worlds/tww/Rules.py
--- a/worlds/tww/Rules.py	(revision db56e26df9f4aa0ebe3a708f7115d8e898e5b72a)
+++ b/worlds/tww/Rules.py	(date 1767929704424)
@@ -75,6 +75,41 @@
         return self.multiworld.worlds[player].logic_tuner_logic_enabled
 
 
+def mix_in_universal_tracker_logic():
+    def _tww_obscure_1(self, player: int) -> bool:
+        return self.multiworld.worlds[player].logic_obscure_1 or self.has("Glitched", player)
+
+    def _tww_obscure_2(self, player: int) -> bool:
+        return self.multiworld.worlds[player].logic_obscure_2 or self.has("Glitched", player)
+
+    def _tww_obscure_3(self, player: int) -> bool:
+        return self.multiworld.worlds[player].logic_obscure_3 or self.has("Glitched", player)
+
+    def _tww_precise_1(self, player: int) -> bool:
+        return self.multiworld.worlds[player].logic_precise_1 or self.has("Glitched", player)
+
+    def _tww_precise_2(self, player: int) -> bool:
+        return self.multiworld.worlds[player].logic_precise_2 or self.has("Glitched", player)
+
+    def _tww_precise_3(self, player: int) -> bool:
+        return self.multiworld.worlds[player].logic_precise_3 or self.has("Glitched", player)
+
+    mix_in_methods = {k: v for k, v in locals().items() if k.startswith("_tww_")}
+
+    if len(mix_in_methods) != 6:
+        raise Exception(f"Expected 6 mix-in methods to replace, but got {len(mix_in_methods)}")
+
+    for k, v in mix_in_methods.items():
+        if not hasattr(TWWLogic, k):
+            raise Exception(f"{k} was must be present on TWWLogic")
+        if not hasattr(CollectionState, k):
+            raise Exception(f"{k} must be present on CollectionState")
+        if getattr(CollectionState, k) is not getattr(TWWLogic, k):
+            raise Exception(f"CollectionState.{k} should be TWWLogic.{k}")
+        # Replace the normal mixed-in method with the Universal Tracker version.
+        setattr(CollectionState, k, v)
+
+
 def set_rules(world: "TWWWorld") -> None:  # noqa: F405
     """
     Define the logic rules for locations in The Wind Waker.
Index: worlds/tww/__init__.py
IDEA additional info:
Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP
<+>UTF-8
===================================================================
diff --git a/worlds/tww/__init__.py b/worlds/tww/__init__.py
--- a/worlds/tww/__init__.py	(revision db56e26df9f4aa0ebe3a708f7115d8e898e5b72a)
+++ b/worlds/tww/__init__.py	(date 1767929704430)
@@ -24,7 +24,7 @@
 from .randomizers.Entrances import ALL_EXITS, BOSS_EXIT_TO_DUNGEON, MINIBOSS_EXIT_TO_DUNGEON, EntranceRandomizer
 from .randomizers.ItemPool import generate_itempool
 from .randomizers.RequiredBosses import RequiredBossesRandomizer
-from .Rules import set_rules
+from .Rules import set_rules, mix_in_universal_tracker_logic
 
 VERSION: tuple[int, int, int] = (3, 0, 0)
 
@@ -281,6 +281,12 @@
         """
         Run before any general steps of the MultiWorld other than options.
         """
+        if hasattr(self.multiworld, "generation_is_fake") and not getattr(TWWWorld, "_ut_logic_mixed_in", False):
+            # Replace normal LogicMixin methods with
+            mix_in_universal_tracker_logic()
+            # Prevent re-replacing the mixin methods if they have already been replaced.
+            setattr(TWWWorld, "_ut_logic_mixed_in", True)
+
         options = self.options
 
         # Only randomize secret cave inner entrances if both puzzle secret caves and combat secret caves are enabled.

[lonegraywolf2000]

Implemented.

[Mysteryem]

This attribute will need to be prefixed with _tww or similar. While I think it is unlikely that other worlds are setting a _ut_logic_mixed_in attribute, all worlds share the same CollectionState class, so adding a world-specific prefix should avoid possible issues.

[Mysteryem]

I think for general critique of the structure of the changes in this PR is that __init__.py already has a lot in it, and Universal Tracker-specific code isn't relevant to most generations, so I think it would be good to move most of the actual code for handling Universal Tracker into a new module, with the code in __init__.py importing and calling functions from this new module.