Bug: Stardew Valley slot data breaks the WebHost

[OriginalTomPouce]

What happened?

Hello !

I'm building a silly tracker that displays game inventories from a multiworld (a normal AP multiworld tracker). I'm using the WebHost API to fetch the multiworld data, including the slot data in order to get some goal info.

I'm using this API call :
/slot_data_tracker/<suuid:tracker>

However, some multiworlds send me back a 500 error when I try to get the slot_data. After some investigation (a good number of local gens and local hosts), Stardew Valley appaears to always break that API call.

The reason why is because of the generated slot data. Stardew Valley uses an hybrid structure to store their bundles. That structure, once set on the database, always breaks the Webhost when it tries to retrieve it.

File that cause the issue : /worlds/stardew_valley/init.py , line 363

        for room in self.modified_bundles:
            bundles[room.name] = dict()
            for bundle in room.bundles:
                bundles[room.name][bundle.name] = {"number_required": bundle.number_required}
                for i, item in enumerate(bundle.items):
                    bundles[room.name][bundle.name][i] = f"{item.get_item()}|{item.amount}|{item.quality}"

I've checked the Stardew Valley mod to see if the slot_data is used. It is, and we have to keep the number_required key on the same level as their associated items.

Github of the mod : https://github.com/agilbert1412/StardewArchipelago
An example of a class that parse the bundle structure : https://github.com/agilbert1412/StardewArchipelago/blob/main/StardewArchipelago/Bundles/ItemBundle.cs

The safest way to solve this would be to add the bundle items as a key, by casting the index into a str :

                    bundles[room.name][bundle.name][str(i)] = f"{item.get_item()}|{item.amount}|{item.quality}"

I set this as an issue because I don't have a copy of Stardew Valley. I cannot do an end to end test.

What were the expected results?

The /slot_data_tracker/<suuid:tracker> should send me the slot_data as usual. Or maybe not take into account any invalid data.

I'm aware that slot_data can be too versatile.

Software

Website

[ScipioWright]

@agilbert1412

[agilbert1412]

That's strange. We can retrieve it fine from the mod and from poptracker. What makes this different?

[Mysteryem]

The issue is that Stardew Valley is putting int keys into slot data, when json keys must be str.

While json.dumps() normally happily serialises int keys into str, if the json serialisation is being performed with sort_keys=True, then the serialisation fails when there is a dictionary containing a mix of int and str keys because the keys cannot be compared.

No exception:
json.dumps({1: "string", "str_key": "string2"}, sort_keys=False)

Raises TypeError: '<' not supported between instances of 'str' and 'int':
json.dumps({1: "string", "str_key": "string2"}, sort_keys=True)

tests.general.test_implemented.test_slot_data already checks for keys only being str, but it is currently only checking the top-level dict, and is not checking recursively through the slot data, so that could probably be improved.

Here is an example traceback from webhost:

  File "D:\Program Files\Python313\Lib\site-packages\flask\app.py", line 1511, in wsgi_app
    response = self.full_dispatch_request()
  File "D:\Program Files\Python313\Lib\site-packages\flask\app.py", line 920, in full_dispatch_request
    return self.finalize_request(rv)
           ~~~~~~~~~~~~~~~~~~~~~^^^^
  File "D:\Program Files\Python313\Lib\site-packages\flask\app.py", line 939, in finalize_request
    response = self.make_response(rv)
  File "D:\Program Files\Python313\Lib\site-packages\flask\app.py", line 1231, in make_response
    rv = self.json.response(rv)
  File "D:\Program Files\Python313\Lib\site-packages\flask\json\provider.py", line 214, in response
    f"{self.dumps(obj, **dump_args)}\n", mimetype=self.mimetype
       ~~~~~~~~~~^^^^^^^^^^^^^^^^^^
  File "D:\Program Files\Python313\Lib\site-packages\flask\json\provider.py", line 179, in dumps
    return json.dumps(obj, **kwargs)
           ~~~~~~~~~~^^^^^^^^^^^^^^^
  File "D:\Program Files\Python313\Lib\json\__init__.py", line 242, in dumps
    **kw).encode(obj)
          ~~~~~~^^^^^
  File "D:\Program Files\Python313\Lib\json\encoder.py", line 202, in encode
    chunks = self.iterencode(o, _one_shot=True)
  File "D:\Program Files\Python313\Lib\json\encoder.py", line 263, in iterencode
    return _iterencode(o, 0)
TypeError: '<' not supported between instances of 'int' and 'str'

[agilbert1412]

Thanks to Mysteryem and Berserker for clarifying, now I understand the issue. I fixed it in the 7.x.x branch.

If this is for some reason urgent (which would be surprising, considering it's been like that for years and this is the first complaint), tell me and I'll PR a cherry picked fix as well. But I'd prefer not, to avoid conflicts.

[duckboycool]

From what I've heard before, a world issue (or core really) should stay open until a fix actually gets merged.

[agilbert1412]

Oh, alright. I wasn't aware of this process. Then consider this "submitted".