Misleading exception for input source when processing byte buffer with start offset (see JsonProcessingException)

[gwittel]

Describe the bug

If I use the ObjectMapper.readValue(byte[], int offset, int len, TypeReference) interface to deserialize a value AND the deserialize fails (e.g. bad data not conforming to the expected type), the exception refers to the entire buffer rather than the buffer starting at offset.

This leads to a misleading exception where it appears as the entire buffer is the problem, where the real problem is starting somewhere else.

This came up when processing byte buffers of JSON line separated data (so the buffer may contain more than one JSON object).

Version information
Which Jackson version(s) was this for?

2.11.0

To Reproduce
If you have a way to reproduce this with:

@Test
public void simpleJsonTest()
        throws Exception
{
    String json = "{\"k1\":\"v1\"}\n[\"oops\"]\n{\"k2\":\"v2\"}";
    byte[] buffer = json.getBytes(UTF_8);
    int itemStart = json.indexOf('\n') + 1; // Get the second record start
    int itemEnd = json.indexOf('\n', itemStart); // End of second record
    int itemLength = itemEnd - itemStart;

    ObjectMapper mapper = new ObjectMapper();

    TypeReference<Map<String, String>> typeReference = new TypeReference<>()
    {
    };

    Map<String, String> decoded = mapper.readValue(buffer, itemStart, itemLength, typeReference);
}

Stack trace with error:

com.fasterxml.jackson.databind.exc.MismatchedInputException: Cannot deserialize instance of `java.util.LinkedHashMap<java.lang.Object,java.lang.Object>` out of START_ARRAY token
 at [Source: (byte[])"{"k1":"v1"}
["oops"]
{"k2":"v2"}"; line: 1, column: 1]

	at com.fasterxml.jackson.databind.exc.MismatchedInputException.from(MismatchedInputException.java:59)
	at com.fasterxml.jackson.databind.DeserializationContext.reportInputMismatch(DeserializationContext.java:1464)
	at com.fasterxml.jackson.databind.DeserializationContext.handleUnexpectedToken(DeserializationContext.java:1238)
	at com.fasterxml.jackson.databind.DeserializationContext.handleUnexpectedToken(DeserializationContext.java:1148)
	at com.fasterxml.jackson.databind.deser.std.StdDeserializer._deserializeFromEmpty(StdDeserializer.java:639)
	at com.fasterxml.jackson.databind.deser.std.MapDeserializer.deserialize(MapDeserializer.java:360)
	at com.fasterxml.jackson.databind.deser.std.MapDeserializer.deserialize(MapDeserializer.java:29)
	at com.fasterxml.jackson.databind.ObjectMapper._readMapAndClose(ObjectMapper.java:4482)
	at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:3521)
	at XXX.simpleJsonTest(XXX.java:152)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
	at org.testng.internal.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:124)
	at org.testng.internal.Invoker.invokeMethod(Invoker.java:583)
	at org.testng.internal.Invoker.invokeTestMethod(Invoker.java:719)
	at org.testng.internal.Invoker.invokeTestMethods(Invoker.java:989)
	at org.testng.internal.TestMethodWorker.invokeTestMethods(TestMethodWorker.java:125)
	at org.testng.internal.TestMethodWorker.run(TestMethodWorker.java:109)
	at org.testng.TestRunner.privateRun(TestRunner.java:648)
	at org.testng.TestRunner.run(TestRunner.java:505)
	at org.testng.SuiteRunner.runTest(SuiteRunner.java:455)
	at org.testng.SuiteRunner.runSequentially(SuiteRunner.java:450)
	at org.testng.SuiteRunner.privateRun(SuiteRunner.java:415)
	at org.testng.SuiteRunner.run(SuiteRunner.java:364)
	at org.testng.SuiteRunnerWorker.runSuite(SuiteRunnerWorker.java:52)
	at org.testng.SuiteRunnerWorker.run(SuiteRunnerWorker.java:84)
	at org.testng.TestNG.runSuitesSequentially(TestNG.java:1208)
	at org.testng.TestNG.runSuitesLocally(TestNG.java:1137)
	at org.testng.TestNG.runSuites(TestNG.java:1049)
	at org.testng.TestNG.run(TestNG.java:1017)
	at com.intellij.rt.testng.IDEARemoteTestNG.run(IDEARemoteTestNG.java:66)
	at com.intellij.rt.testng.RemoteTestNGStarter.main(RemoteTestNGStarter.java:109)

Expected behavior

In this case, I would expect the exception error message to point to the start offset (here its the ["oops"] chunk). Due to the supplied source context, the "line 1 column 1" guides the reader to point to the start of the (valid) first record in the buffer, rather than the actual problem which is line 1 column 1 in the second invalid record.

Additional context

Add any other context about the problem here.

In the example, the underlying exceptions' JsonLocation looks like this:

_totalBytes = 0
_totalChars = -1
_lineNr = 1
_columnNr = 1
_sourceRef= The whole buffer

[cowtowncoder]

Ah. So input source description appears to ignore offset.

[gwittel]

Yes, that appears to be the case.

[cowtowncoder]

Ah. This will be tricky to handle -- problem is that only byte[] (and char[] in other cases) is passed to JsonLocation, so it literally has no knowledge of offsets. Will see what constructs location for such case, and if that could then construct wrapper, which in turn would need to have handling to indicate content in range.

[cowtowncoder]

Hmmh. In theory, yes. Source reference (opaque Object) is passed to IOContext at low level, and that in turn could be constructed to use a more usable wrapper. Downside is that this could break handling that expected to see byte[] get returned.
Given timing I think such change would unfortunately be bit risky for 2.12, so I'll tag this as 2.13 to give more time to think of what wrapper type to use -- and whether there should be some extension point for more customization since I suspect there are some aspects frameworks might be interested in using (wrt error reporting, typically).

[cowtowncoder]

(moved to jackson-core since this where relevant code is, regardless of how it gets called)

[cowtowncoder]

Quick notes:

1. Working on solving the reporting issue for 2.13
2. Original use case is bit obsolete: Jackson can handle line-delimited content on its own, see retain attribute entity references instead of choking when (and this functionality has been available for long time)