Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Custom key deserialiser registered for Object.class in nested Map object is ignored when Map key type not defined #4680

Closed
1 task done
devdanylo opened this issue Aug 27, 2024 · 12 comments · Fixed by #4684
Labels
2.19 Issues planned at 2.19 or later

Comments

@devdanylo
Copy link

devdanylo commented Aug 27, 2024

Search before asking

  • I searched in the issues and found nothing similar.

Describe the bug

I'm working on a piece of software which accepts arbitrary JSON, flattens it, and then writes it to CSV. The main issue here is that we don't know the schema of the JSON—it can be anything. This, in turn, results in untyped deserialization.

Before starting processing of the parsed object, the field names must be sanitised (yes, it's crucial to sanitise the field names during the parsing phase). Unfortunately UntypedObjectDeserializerNR ignores the custom key deserialiser I'm trying to provide:

package libraries;

import com.fasterxml.jackson.databind.DeserializationContext;
import com.fasterxml.jackson.databind.KeyDeserializer;
import com.fasterxml.jackson.databind.ObjectMapper;
import com.fasterxml.jackson.databind.module.SimpleModule;
import org.junit.jupiter.api.Assertions;
import org.junit.jupiter.api.Test;

import java.util.Map;

public class ObjectMapperTest {

    @Test
    void customKeyDeserializerShouldBeUsedWhenTypeNotDefined() throws Exception {
        // given
        var json = """
                {
                    "name*": "Erik",
                    "address*": {
                        "city*": {
                            "id*": 1,
                            "name*": "Berlin"
                        },
                        "street*": "Elvirastr"
                    }
                }
                                
                """;

        var objectMapper = new ObjectMapper();
        var keySanitizationModule = new SimpleModule("key-sanitization");
        keySanitizationModule.addKeyDeserializer(String.class, new KeyDeserializer() {
            @Override
            public Object deserializeKey(String key, DeserializationContext ctxt) {
                return key.replace("*", "_");
            }
        });
        objectMapper.registerModule(keySanitizationModule);

        // when
        var result = objectMapper.readValue(json, Object.class);

        // then
        Assertions.assertEquals("Erik", ((Map<String, Object>) result).get("name_"));

        var addressMap = (Map<String, Object>) ((Map<String, Object>) result).get("address_");
        Assertions.assertEquals("Elvirastr", addressMap.get("street_"));

        var cityMap = (Map<String, Object>) addressMap.get("city_");
        Assertions.assertEquals(1, cityMap.get("id_"));
        Assertions.assertEquals("Berlin", cityMap.get("name_"));
    }
}

I have 2 questions in regards of this behaviour:

  1. Is that something expected?
  2. Is there a workaround for the problem in question?

Version Information

2.14.3

Reproduction

No response

Expected behavior

No response

Additional context

No response

@devdanylo devdanylo added the to-evaluate Issue that has been received but not yet evaluated label Aug 27, 2024
@JooHyukKim
Copy link
Member

Have you tried mapper.readValue(json, new TypeReference<Map<String, Object>>(){})?

@devdanylo
Copy link
Author

@JooHyukKim in such case for the root level object the custom key deserialiser is going to work, but for the nested objects it's still going to be ignored.

@cowtowncoder
Copy link
Member

I think the problem is that unless you can provide tight type definition for nested Maps, key deserializer is looked up for type Object and not String. Whether this is a bug or not in Jackson is debatable (it can be definitely be unexpected); not 100% sure if changing it would be safe (esp. for backwards-compatibility).

But I think you can register same KeyDeserializer for Object.class too. Either by adding another registration call for SimpleModule, or implementing KeyDeserializers callback which could return same instance for different JavaTypes.
Second approach could be helpful for debugging as well.

If registering of key deserializer for Object.class doesn't work, that's a bug.

@cowtowncoder cowtowncoder changed the title Custom key deserialiser is ignored when type not defined Custom key deserialiser registered for String.class is ignored when Map key type not defined Aug 27, 2024
@cowtowncoder cowtowncoder added 2.18 and removed to-evaluate Issue that has been received but not yet evaluated labels Aug 27, 2024
@JooHyukKim
Copy link
Member

@JooHyukKim in such case for the root level object the custom key deserialiser is going to work, but for the nested objects it's still going to be ignored.

I see, so the input could be nested. Then what @cowtowncoder said above would suffice.

@devdanylo
Copy link
Author

devdanylo commented Aug 28, 2024

If registering of key deserializer for Object.class doesn't work, that's a bug.

@cowtowncoder I've tried keySanitizationModule.addKeyDeserializer(Object.class, ...); and the result is the same.

FYI: if you look at the implementation of UntypedObjectDeserializerNR, you will find out that it doesn't use any external key deserializers.

@JooHyukKim
Copy link
Member

@devdanylo is right. on depth 2 it no longer works.

    @Test
    void customKeyDeserializerShouldBeUsedWhenTypeNotDefined() throws Exception {
        // GIVEN
        var json = """
                {
                    "name*": "Erik",
                    "address*": {
                        "city*": {
                            "id*": 1,
                            "name*": "Berlin"
                        },
                        "street*": "Elvirastr"
                    }
                }
                                
                """;

        var keySanitizationModule = new SimpleModule("key-sanitization");
        keySanitizationModule.addKeyDeserializer(String.class, new KeyDeserializer() {
            @Override
            public String deserializeKey(String key, DeserializationContext ctxt) {
                return key.replace("*", "_");
            }
        });

        keySanitizationModule.addKeyDeserializer(Object.class, new KeyDeserializer() {
            @Override
            public Object deserializeKey(String key, DeserializationContext ctxt) {
                throw new RuntimeException("Should be called, but never called");
            }
        });

        var mapper = JsonMapper.builder().addModule(keySanitizationModule).build();

        // WHEN
        var result = mapper.readValue(json, new TypeReference<Map<String, Object>>() { });

        // THEN
        // depth 1 works as expected
        Assertions.assertEquals("Erik", result.get("name_"));

        // depth 2 does NOT work as expected
        var addressMap = (Map<String, Object>) result.get("address_");
        // null?? Fails here
        Assertions.assertEquals("Elvirastr", addressMap.get("street_"));
        var cityMap = (Map<String, Object>) addressMap.get("city_");
        Assertions.assertEquals(1, cityMap.get("id_"));
        Assertions.assertEquals("Berlin", cityMap.get("name_"));
    }

@cowtowncoder
Copy link
Member

Ok. So it sounds like we have a bug here. I suspect it might be because nested Maps will probably be deserialized using "UntypedObjectDeserializer" (being java.lang.Object). And maybe that does not try to fetch deserializer correctly wrt custom configuration.

@cowtowncoder cowtowncoder added 2.19 Issues planned at 2.19 or later and removed 2.18 labels Aug 28, 2024
@JooHyukKim
Copy link
Member

hmmmm, I tried to come up with a fix, but failing.
The deserialization is being directed to UntypedObjectDeserializerNR and _deserializeNR() is invoked, but I don't think I am correctly understanding the mechanism of deserializing nested (therefore recursive) JSON objects as Object.class.

Any pointers u might want to share here?

@cowtowncoder
Copy link
Member

Ah .NR meaning "Non-recursive". There should be reference to (optional) custom Map deserializer, which would be thing created with custom key deserializer. But I don't immediately recall how locating that works.

@JooHyukKim
Copy link
Member

I just realized this issue may no longer be about String.class but Object.class for key deseriailzation.

@JooHyukKim
Copy link
Member

Should we change issue title from

Custom key deserialiser registered for String.class is ignored when Map key type not defined

to something like below?

Custom key deserialiser registered for Object.class in nested Map object

@cowtowncoder cowtowncoder changed the title Custom key deserialiser registered for String.class is ignored when Map key type not defined Custom key deserialiser registered for Object.class in nested Map object is ignored when Map key type not defined Nov 5, 2024
cowtowncoder added a commit that referenced this issue Nov 5, 2024
@cowtowncoder
Copy link
Member

@JooHyukKim Thank you -- changed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
2.19 Issues planned at 2.19 or later
Projects
None yet
3 participants