Quoted string specification

[arnaud-secondlayer]

I'm facing issues when LLMs are generating strings with quotes. I wonder if we could update the String specification to make it more robust to these problems.

The major issue is when the LLM generates a string with a starting quote.
Here is an example: "hello" said Alice
Most of the time, the LLM would generate something like:

text: "hello" said Alice

and this leads to a parsing error.
In theory the LLM should have generated the following to get the proper results, but I never see it happen.

text: "\"hello\" said Alice"

The quoted string specification is very clear and logic, but TOON is still a new format and I'm not sure LLMs trained on enough data to be able to output proper quoted strings.

Anyone else facing the same problem?