Migration of build + workflow scripts to typescript

[akshatnema]

Description

With the current package.json built for ES modules and packages, as we migrated the website to Next js 14 + Typescript, it has now started affecting the scripts folder like working of meetings workflow, automation of blog creation using npm script, etc. Since, to have uniformity across the website, we decided to migrate the existing Node js scripts to common js modules (in Typescript), enhancing the code maintenance, scalability, and readability for contributors.

Tasks need to be done

This issue comprises multiple tasks, that need to be completed to cover all the scripts in the project. Here is the list:

• Setup of ts-node in the website (without deflecting existing node js setup)
• Convert existing build scripts to typescripts, with the correct types
• Convert workflow scripts to use typescript and setup ts-node for GitHub workflows.
• Make all above changes without breaking the existing tests and add new tests, if left uncovered

[DhairyaMajmudar]

Hi @akshatnema, I would like to open a PR for migrating build scripts files into ts

[lucky29-git]

@DhairyaMajmudar I would like to work on this, please inform if you need any help.

[akshatnema]

This idea is currently part of AsyncAPI mentorship program (https://github.com/orgs/asyncapi/discussions/1361) - You have to apply from there itself.

[akshatnema]

Hey Everyone,
Here is the proposal submission form for project idea, Migration of build + workflow scripts to typescript. Please answer the questions in the form as well.

Form Link - https://forms.gle/ZjhDcfo55B7HDhUy6

[JeelRajodiya]

Project Status

Build Scripts Migration

(The order is subject to change)

• Environmental setup for ts-node to run the scripts, Migrate scripts/index.js and scripts/build-docs.js
• Migrate scripts/build-pages.js and scripts/build-post-list.js
• Migrate scripts/build-rss.js and scripts/compose.js
• Migrate scripts/utils/util.js, adopters/index.js, and casestudies/index.js
• Migrate scripts/finance/index.js
• Migrate scripts/tools/combine-tools.js and scripts/tools/extract-tools-github.js
• Migrate scripts/dashboard/build-dashboard.js, issue-queries.js, build-meetings.js
• Migrate remaining scripts (categorylist.js, tags-color.js)

List of Scripts (from the least dependent(on each other) to the most dependent script)

• utils.js
• compose.js
• build-rss.js
• build-pages.js
• build-newsroom-videos.js
• build-meetings.js
• build-docs.js
• build-post-list.js
• utils/readAndWriteJson.js
• /tools folder and build-tools.js
• /markdown folder
• finance/index.js
• /dashboard folder
• /casestudies folder
• /adopters folder
• index,js

Build script migration is expected to be finished by: 19th December, 2024

Workflow Script Migration

• Setup github-typescript and migrate Javascript code in workflows add-good-first-issue-labels.yml, automerge-for-humans-add-ready-to-merge-or-do-not-merge-label.yml
• Migrate Javascript code in automerge-for-humans-remove-ready-to-merge-label-on-edit.yml, automerge-orphans.yml, bounty-program-commands.yml
• Migrate Javascript code in help-command.yml, lighthouse-ci.yml, please-take-a-look-command.yml
• Migrate Javascript code in release-announcements.yml, update-pr.yml, validate-case-studies-structures.yaml

Unique workflows in the website repo

All Other workflows either are in the .github repo or does not have any javascript code to migrate

• lighthouse-ci.yml
• validate-case-studies-structures.yaml

Workflow Script Migration is expected to be finished by: 9th January, 2025

Test Scripts Migration (currently not under consideration for the project)

• Setup Typescript in jest and Migrate test scripts for /adopters, readAndWriteJson.test.js, and utils.test.js
• Migrate test scripts for /build-docs, /fixtures, and /casestudies
• Migrate test scripts for build-pages.test.js, index.test.js, and /finance
• Migrate test scripts for build-tools.test.js, build-meetings.test.js, and build-newsroom-videos.test.js

If Considered Test Scripts Migration is expected to be finished by: 20th February, 2025

[akshatnema]

@JeelRajodiya Regarding workflow scripts, migrate only those workflows which are not maintained from .github repo. Don't update the workflows which are maintained at .github repository.

Secondly, why there is a need of migrating tests in typescript?

[JeelRajodiya]

@JeelRajodiya Regarding workflow scripts, migrate only those workflows which are not maintained from .github repo. Don't update the workflows which are maintained at .github repository.

Sure, I will update the comment.

Secondly, why there is a need of migrating tests in typescript?

Isn't this in the scope of the project?, I have mentioned the test script migration in my proposal so I have included it here as well. Are we considering to migrate them?

[akshatnema]

Are we considering to migrate them?

Currently, @vishvamsinh28 is writing tests for the scripts. Let him finish that first. It's not necessary to migrate the tests in TS, if they works even if scripts are in typescript. If during of scripts, tests result out incompatible, we can then think of them to migrate them to typescript, else I don't find any reason to migrate them. WDYT?

[JeelRajodiya]

Currently, @vishvamsinh28 is writing tests for the scripts. Let him finish that first.

I was not aware about it. Thanks for informing.

It's not necessary to migrate the tests in TS, if they works even if scripts are in typescript. If during of scripts, tests result out incompatible, we can then think of them to migrate them to typescript, else I don't find any reason to migrate them. WDYT?

Sure, Let's drop the plan of migrating the tests. We can think of migrating them later on if needed.

[JeelRajodiya]

@akshatnema What do you think about using tsx instead of ts-node?

While running the .ts files with ts-node, I am having the following issues.

SyntaxError: Cannot use import statement outside a module

When I try to just use the require imports, typescript is not detecting the types of imported modules. So I have to use ES imports. To resolve this error, I added "type": "module" in the package.json file. This configuration requires me that in each file we should have ES imports, So I have may need to migrate all of the scripts at the same time.

However after adding the configuration, another error pop-up, please see below.

We can resolve this error as well, but It requires me to modify the tsconfig.json file.

On the other hand, when I run the scripts with tsx. We do not have any errors and it does not require me to make any modification in the package.json or tsconfig.json files.

Successful typescript run with tsx

[akshatnema]

SyntaxError: Cannot use import statement outside a module

@JeelRajodiya This is a very generic error, probably you have not setup the package.json with ts-node correctly. This should not happen.

When I try to just use the require imports, typescript is not detecting the types of imported modules. So I have to use ES imports. To resolve this error, I added "type": "module" in the package.json file. This configuration requires me that in each file we should have ES imports, So I have may need to migrate all of the scripts at the same time.

Yeah, that's perfectly fine because logically you should have all files is ts only. And it would be best if you thought in that approach to migrate all the script files.

We can resolve this error as well, but It requires me to modify the tsconfig.json file.

Yeah, feel free to do those changes. You have to make these changes to implement it in better. Just that we are getting some normal configuration issues, we can't simply move from ts-node to tsx.

[akshatnema]

@JeelRajodiya Also, during targeting all the above errors (from tsconfig file as well), if we find ourselves in a endless loop to resolve the errors or it is changing the environment for next js inside repo, then we can think to use tsx over here.