[E2E] Streamlining server codegen project scaffolding

[allenjzhang]

Overview:

The Service Codegen can streamline the development process with project scaffolding, making it crucial to scaffold the server project effectively to help users get started efficiently.

The scaffolded project may include various components that experience different levels of modification as the source project evolves:

• Modifiable Components:

  • Unchanged Project Components: These include files such as .csproj and .settings that generally remain untouched.

  • Potentially Changing Components: Files like program.cs may require modifications as the project evolves.

  • Always Impacted by Spec Changes: For example, mock classes based on implementation interfaces will often need to be regenerated in response to specification updates.

• Always Regenerated Components:

  • These components are not expected to be modified by users. They should reside in isolated folders to avoid accidental changes and are outside the scope of this discussion.

  • There are different use cases for when and how users may want to scaffold a project. Striking a balance between requiring explicit steps and always scaffolding the project is crucial to prevent accidental overwrites of user-modified code.

Option 1: Explicit Invocation

Users can run a command to explicitly scaffold the project. To lower the barrier for new users, a standard CLI command such as tsp scaffold could be introduced.

Pros:

• Users’ explicit action reduces the risk of accidental overwrites, with sufficient warnings provided.

Cons:

• Users must be aware that an extra step is required, which could pose a barrier for new users unfamiliar with the workflow.

Option 2: Automatic Invocation in compile Path

In this approach, the project is always scaffolded during the compile process. This requires classifying files and implementing various levels of intelligence to handle modifications appropriately.

Pros:

• Provides a seamless user experience by eliminating the need for additional steps.

Cons:

• More complex to implement, as it requires advanced logic to detect and handle modifications.

• May not align with some users’ expectations if their custom changes are unintentionally affected.

Option 3: Split the scaffolding between init & compile

[allenjzhang]

After discussion, proposal has been narrowed down. The conversation has mostly converged on 1. Explicit invocation. But I feel we have not given 2. a little more thoughts. Please comment.

1. Explicit Invocation

1. Each service emitter will provide a standard init-project.ts/js script to allow initial project scaffolding
   • This script will create both static files
   • This script will create spec specific mock files. This may involves emitter being invoked compile with special flag
   • If file exists, warning will be generated and prompt Y/n for continuation
2. tsp init will place a custom script command init-project in TypeSpec project's package.json to allow user to invoke the script
   • if there are multiple server projects, single npx run init-project will invoke all server projects' scaffolding script
3. Each emitter will produce a README.md with What's next section. Template TBD.
4. Do we need a README.md in the TypeSpec project?

2. Split between init & compile

1. tsp init will place static files for each server project
2. Each service emitter will generated spec dependent mock files during compile flow. If mock file exists, prompting for overwrite or skip.

[timotheeguerin]

Feels like it will keep prompting for override which is not great.
But what about do generate it if it doesn't exist/some marker file doesn't exists and provide the script command to reset it.

I don't really like that we require the user to run this script after init again

[markcowl]

@timotheeguerin The reason for running the script after init is that some of the scaffolded components are dependent on the operations in the tsp file, and we anticipate that anyone using tsp init is going to want to regenerate after replacing the tsp in the init project, or developing their script.

The idea of the npm init-project script (or whatever we call it) in the package.json is to by default create the scaffolding in the appropriate directory based on the project layout and config.

In this scenario, we would also emit a readme.md with next steps, which would be altered based on the installed components, and we would also log next steps in the console, again, based on the emitters selected.

[timotheeguerin]

ok I guess I might not understand fully the folow and what is getting generated.
But is then the assumption that if for example I add a new operation I would run

tsp compile . # Generate the handling for that new operation
npx run init-project  # Generate the controller placeholder?

[markcowl]

@timotheeguerin Kind of.

If I have not done anythjng to implement the service, I would run these two commands, although I would probably run npx init-project before compiling, but the order doesn't really matter.

After I have implemented the service, I would just run tsp compile . to regenerate, and make any corrections to my implementation

[markcowl]

After discussion, here is the resolution:

• Use a static script init-server script in package.json which discovers if either of the server emitters are installed and calls their individual scaffolding commands
  • The script should not have any dependencies outside of typespec and existing package.json dependencies
• Since the script will not work in the nodeless scenario when using the tsp standalone executable
  • Add all scaffolding emission to http-server-csharp, which has, to enable the nodeless scenario
• Since http-server-js requires some kind of javascript runtime (node, yarn, bun) in the future we will need to test and account for these runtimes
• Long term solution across standalone exe and node scenarios will be a tsp scaffold command

[markcowl]

Completed