Add a nightly feature which allows to manipulate spans to underline which part of the template is failing compilation

[GuillaumeGomez]

We can manipulate rustc's spans only in nightly. With it, since we know where we are in the template source, we can then generate code with a span associated to this exact location. It would be a crazy improvement over the current situation and I don't think it would be too difficult to do as well.

Gonna reserve this issue for myself. =D

[Kijewski]

At some point (far in the future, maybe) I would love to go a step further, and convert the current code that fills a String to get parsed as a TokenStream. Instead we could generate a TokenStream directly. This way it might be possible to click on a variable in the template to get to the field definition in the struct, and otherway around: click on the field definition to find the usages, and to see that it's used in the template code.

[GuillaumeGomez]

Sadly, it's not going to be possible with the current API because we don't have access to compiler Span's data, making it only possible to generate better errors for inlined templates, which likely represent only the minority for all templates.

So after some thinking, I proposed a new API for the proc-macro library: rust-lang/rust#140943

Let's see if it gains some traction.

[Kijewski]

Have a look at https://github.com/Kijewski/proc-macro-include-std-test.

[GuillaumeGomez]

Ok, that's a very good lead. When I have time I'll try to work around it. Nightly is not even needed for that, that's crazy.

[GuillaumeGomez]

So now remains to make it work with path and other included files. Something similar to https://github.com/Kijewski/proc-macro-include-std-test should allow us to make it work. Just curious what the output will look like.

[Kijewski]

I think this is what has to be done in order to implement this feature:

• Generate TokenStream instead of String
  • Generate a TokenStream rather than a String #558
• Don't use &'a str as parse stream, but LocatingSlice<&'i str>
  • parser: use LocatingSlice<&str> instead of &str #560
• Replace Span(&str) with Span { bytes: Range<usize> }
  • Use Parser::[with_]span() to get byte range and use byte range for proc_macro::Span #571
• Have an input file span like https://github.com/Kijewski/proc-macro-include-std-test.
  • Use Parser::[with_]span() to get byte range and use byte range for proc_macro::Span #571

Optional, but might be useful to make spans more accurate later on:

• Make the parser::State part of the InputStream state
  • Use Parser::[with_]span() to get byte range and use byte range for proc_macro::Span #571
• Remove WithSpan. Let Node, Expr, Target know the Span of its leading token instead. E.g. Node::If should know the span of the if in the input stream.
• For Expr, replace our recursive descent parser with a double-e infix expression parser.
• extract the tokenization into its own crate