Bug fix for issue 745 : Fix string duplication on escaped chunk boundaries

[dwalend]

Bug #745 tokens[F, String] has the same chunk-boundary escape duplication bug as #515

Description

Issue #515 (fixed in 1.8.1 by commit 18a8ce2) fixed a bug where chunk boundaries falling inside JSON escape sequences caused string content duplication. The fix added T.mark(context) in slowString_ (the slow path) after seeing a backslash, but missed the identical pattern in string_ (the fast path) at line 55-58.

When the fast-path string_ encounters a backslash, it calls appendMarked, advance, then enters slowString_ -- but without calling mark first. If the chunk boundary falls right after the backslash, slowString_ calls appendMarked again using the stale mark position, re-appending already-consumed content.

This affects tokens[F, String] because String chunks are typically small (one per emit). The original #515 test used tokens[F, Byte] with chunkLimit(55), which happened to align chunks differently and not hit the fast-path code.

Fix

One line -- add T.mark(context) in string_ after seeing a backslash, matching what slowString_ already does:

--- a/json/src/main/scala/fs2/data/json/internal/JsonTokenParser.scala
+++ b/json/src/main/scala/fs2/data/json/internal/JsonTokenParser.scala
@@ -55,6 +55,7 @@
         case '\\' =>
           T.appendMarked(context, acc)
           T.advance(context)
+          T.mark(context)
           slowString_(key, StringState.SeenBackslash, 0, acc)

PR with fix and tests: closes #745.

[satabin]

Hi @dwalend, thanks for reporting and proposing a fix for this. I will look into this change later today.

[dwalend]

Thanks! Let me know if you want me to make any changes.

[satabin]

Just two small comments on the test, otherwise LGTM. Thanks a lot for spotting this.

[dwalend]

Both changes are in. Thanks again for taking up my bug. Let me know if you want any other changes.

I've got an AI shaggy dog story to go with it. ("The left coprocessor does not know what the right coprocessor is up to.")

[satabin]

Thanks for the contribution. I will merge it and backport it to release 1.12.1 with this fix immediately.