Add basic jsdoc tag types

[sandersn]

This PR adds basic support for the JSDoc tags @type, @param, @returns, @typedef and @template. The implementation is quite different from Strada, but Corsa will now have examples of all 3 major approaches we've planned for JS inference we've planned for JS inference in Corsa, meaning that the rest is just filling out support.

Functionality

@type applies to variable declarations and property declarations:

/** @type {number} */
const x = 1
class C {
  /** @type {1 | 2 | 3} */
  p = 1
}

It also works as a cast on parenthesised expressions:

let x = /** @type {12} */(12)

And it behaves as a cast on property assignments, export assignments, and return statements:

const o = {
  /** @type {number} */
  p: 2
}
/** @type {Level} */
export = { e: 1, m: 1 }
function f() {
  /** @type {1 | 2 | 3} */
  return 1
}

I ported this support from Strada, and it's useful for @param/@returns (see next), but it doesn't really make sense for @type. I might cut it, or perhaps add a real Type field to property assignment/export assignment/return statement; the checker currently acts as if they do because of porting from Strada.

@param and @returns apply to parameters and return types of functions. They work on anything function-like, including in lower positions:

/**
 * @param {string} s
 * @returns {never}
 */
function fail(s) { throw new Error(s) }
/**
 * @param {*} input
 * @returns {string}
 */
const log = input => input.toString()
class C {
  /** @returns {string} */
  f = () => ""
}
const o = {
  /** @returns {string} */
  f: () => ""
}
/** @returns {string} */
export = () => ""
function f() {
  /** @returns {string} */
  return () => ""
}

@typedef supports both single-line types and multi-line object types with @param. Nested object types are not yet supported. As in Strada, @typedefs are implicitly exported, but this PR is missing the fanciness to make that work right with global scripts or CommonJS modules. Currently, @typedef always turns a file into a module.

/** @typedef {'a' | 'b' | 'c'} ABC */
/** @typedef {Object} Composite
 * @property {number} one - some documentation
 * @property {string} two - more documentation
 */

@template tags work with @typedef, function-likes and classes.

/**
 * @template T
 * @typedef {Object} Box
 * @property {T} value
 */
/**
 * @template T
 * @param {T} x
 * @returns T
 */
function id(x) { 
}

Architecture

Corsa now uses 3 different approaches to support JS inference:

• expando assignments (eg f.p = expandable), including to class properties, are (or will be) handled in the binder, just like Strada. Unfortunately, this means that every BinaryExpression includes DeclarationBase.
• TS-equivalent type and modifier tags (eg @type @template @private) produce synthetic nodes during parsing. After a node is parsed and its JSDoc is parsed, missing fields on the node can be set to nodes from the tag instead.
• TS-equivalent statements (eg @typedef) produces synthetic statements, which the parser inserts right before the just-parsed node. These statements have their own node kinds and type to allow for JS-specific semantics, although this PR doesn't have any @typedef-specific semantics vs type aliases.
  Worth noting: I have no ability yet to support API users like TSDoc, who in Strada could request all tags for a variable declaration and get all tags on the variable declaration statement, for example. I expect eventually we will have to port the old tag lookup code just for uses like this.

Expando representation

For an expando assignment f.p = expandable, I could emit a synthetic jsnamespace f { jsexport var p = expandable }. This might be more efficient than treating BinaryExpression as a declaration in the binder. Class properties would need an equivalent jsprop p = expandable, and this declaration would either need to be lifted to the class body or retain special support in the binder/checker to allow (JS) property declarations to occur anywhere inside a class.

More information on the synthetic nodes:

Hosts

Corsa searches for a host similar to the way Strada does, but the search is (1) top-down (2) at parse-time instead of repeated each time on-demand. In other words, in Strada, hosts would search up the tree for applicable tags. In Corsa, tags search down the tree for applicable hosts. However, this makes it harder to track multiple hosts, so for this PR, the search stops at the first host, which is by far the most common case.
The bottom-up on-demand nature of Strada's tag-to-host attachment meant that if multiple hosts find a tag, the tag applies to multiple hosts. In Corsa, tags look for hosts, and they apply to the first host only.
This should be much more efficient. This means, at least currently, that each tag can only have one host.

Also note that 'host' generally means the entire signature in Strada, but is a specific parameter type annotation, for example, in Corsa.

Clones

Currently, only the top node of a hosted type is cloned into its new position. It's not a deep clone, so children of the type still retain their original positions and parents. That means when walking the tree top-down, the checker may suddenly be looking at JSDoc nodes that originated from a different place in the tree.

This is important because if the checker needs to walk back up the tree, the parent pointers have not been changed. To fix this, I had to track a type's host on JSDocTypeExpression:

JSDocTypeExpression

JSDocTypeExpression is the JSDoc node kind just above all the types that are cloned. Because of this, when there is a bottom-up lookup, JSDocTypeExpression is the signal that a type was cloned from JSDoc. JSDocTypeExpressions store a pointer to the host node of its type, and name resolution, for example, uses it to jump from a JSDoc type to its host.

Overall, a deep clone with parent pointer fixup might be better. But I looked at DeepClone and it doesn't change the parent or provide hooks to change it.

Implementation

• ClassLikeBase now has LocalsContainerBase because @typedef can be nested in a class body. Questionable.
• I simplified the representation of JSDocTemplateTag, which previously tracked its single constraint per-tag, but now copies the constraint down to each type parameter. Thinking about this, this probably breaks visits and needs to be undone.
• JSDocTypedefTag can now only have an Identifier as a name, not QualifiedName, so I simplified that in this PR too.
• I simplified other places that no longer need to check for param/return/type/template tags. A good example is name resolution, which is much simpler, although it now has to follow a JSDocTypeExpression's host pointer when walking up the tree.
• JSTypeAliasDeclaration is exempt from the grammar checks that apply to TypeAliasDeclaration. Otherwise, a lot of the diffs are from pasting JSTypeAliasDeclaration wherever TypeAliasDeclaration occurred.
• Statement parsing uses parseListIndex, not parseDelimitedList, so I only needed to change the former. After parsing, I check whether a new parser property reparseList contains elements and add them before the just-returned element if so. Then I empty reparseList.
• JSTypeAliasDeclaration emits as jstype T = U. I have no idea if that's right, but normal JSDoc emit should already emit the @typedef (and JSTypeAliasDeclaration should be erased by the type eraser.)
• I had to change the baseline writer to skip synthetic nodes, as far as I remember.

Limits and Future Work

• Host search stops at the first host. I need to figure out whether tags should even support multiple hosts, then figure out to make it work.
• Make host search recursive, if it makes sense to support that.
• Over 2000 baseline diffs still remaining.
• Find out to how permanently accept intentional diffs (or normalise them with Strada's baselines).
• CommonJS
• Class property expando assignments
• Limited constructor function support

[sandersn]

Unlike type, @typedef is allowed as a local inside classes.

I haven't checked to see how common this is so I added support for now as a way of fixing a nil panic. But I will remove it if it's not common.

[jakebailey]

Are the type baseline writer changes still needed after position info was fixed?

[jakebailey]

The PR is too big for me to leave comments on, so I'll list some stuff here:

• Are the type baseline writer changes still needed after position info was fixed?
• checkJsdocTypeTagOnExportAssignment5 indicates that the symbol baseliner isn't properly skipping JSDoc nodes that we were skipping before. This seems to be most of the new symbol baseline diffs.

[sandersn]

Re positions: good question. The skip code is needed because the position of a synthetic child is outside the span of its parent node -- for example, a jsdoc type that gets put on a parameter. The code that the baseliner uses to emit unchanged spans then produces invalid slices of lines.

Re skipping JSDoc: It's actually the synthetic node that's emitting here, and it is useful information. However, it bloats the differences between the Strada and Corsa baselines. And skipping all reparsed nodes avoids needing the position-based skip code, so I'll remove the existing check in favour of one that addresses both concerns.

[jakebailey]

Seems like some checking here isn't working?

[jakebailey]

Seems like it's intentional due to Array meaning actual Array and not evolving array.

[sandersn]

I found very few examples of the synonyms String -> string, Array -> Array<any>, etc when I looked, and over the years I've had quite a few complaints about them from heavy JSDoc users, especially object. So they're not supported anymore.

This includes the syntax for index signatures Object.<K,V> -> { [x: K]: V }, which we ran into elsewhere in the diffs.

I'll add these files to submoduleAccepted.txt when the list of diffs gets smaller.

[jakebailey]

Seems like a bug in the type assertion code that may need to read the synthesized code.

[sandersn]

Could be the checker needs to handle synthetic type assertions the same as it handles normal ones. I'll look after the list of diffs grows smaller.