Tracking list of manual smoke tests before each release

[jessfraz]

Manual Tasks

Beta Give feedback

1. Mash keyboard in code editor
2. Electron tests: external links open an external browser #3373
   desktop-app tests +2
3. coredump will open github issue with stuff populated (from home and project page) there are different ways to trigger it as well (! button at top, bug at bottom, mod + shift + .)
4. running app.{app,exe} path/to/file opens the kcl file (all oses)
5. running app.{app,exe} path/to/dir opens the directory and any kcl file in it or creates a main.kcl if empty (all oses)
6. If someone at the office, test you can hit make from the app and it works
7. Test updating versions works
8. until we have a performance test, sanity check that time to open file directly from command line is not shit
9. Electron tests: ensure we can open files that in tauri we would not be able to #3455
   desktop-app tests +2
10. Electron test: upgrading app should not cause user to have to auth again #3469
    desktop-app tests +2
11. Electron: logging in and authing works #3465
    desktop-app tests +2
12. double click to open files from finder opens new window each time (mac/windows)
13. export from the app works and opens a file picker ux (needs to be manual since we override for playwright)
14. Ensure that a loading toast appears when testing the auto-updater, saying the download has begun
15. Token.txt should work #4038
    desktop-app tests +2
16. Move around and sketch rapidly: is interaction still snappy?

also relevant #3459

[lf94]

Can we put a process in place?

[lf94]

I was thinking maybe a spreadsheet with screenshot links that prove we have done the tests manually.

[jessfraz]

this is it haha, smoke test these for each os :)

[jessfraz]

oh that seems like a lot of work, I trust you all to actually do the thing, versus wasting time dealing w excel

[lf94]

We can save time by partitioning the tests amongst ourselves using a doc tho

[jessfraz]

fair fair set up whatever process you want :)

[franknoirot]

Maybe we auto-make a tasklist on "Cut release vXX.YY.ZZ" PRs

[nadr0]

Grab the out-ubuntu-22.04 AppImage from the build-publish-apps job in the CI CD pipeline of the PR that has the cut release. Example scroll to the bottom and see the artifacts sections to download the file.

Gotcha: I had to chmod +x the file and install Fuse to get the AppImage running

Use: updater-test-ubuntu-22.04 from the artifacts page for the update tests.

• Electron tests: external links open an external browser Electron tests: external links open an external browser #3373
• coredump will open github issue with stuff populated (from home and project page) there are different ways to trigger it as well (! button at top, bug at bottom, mod + shift + .)
  • !
  • Bug bottom folder page
  • Bug bottom model render page
  • ❗ (Does not work on linux) mod + shift + .
• running app.{app,exe} path/to/file opens the kcl file (all oses)

/home/kevin-nadro/Downloads/Zoo\ Modeling\ App-0.25.5-x86_64-linux.AppImage /home/kevin-nadro/Documents/zoo-modeling-app-projects/default/cube.kcl

• running app.{app,exe} path/to/dir opens the directory and any kcl file in it or creates a main.kcl if empty (all oses)
  • Opens directory
  • Creates main file if empty

/home/kevin-nadro/Downloads/Zoo\ Modeling\ App-0.25.5-x86_64-linux.AppImage /home/kevin-nadro/Documents/zoo-modeling-app-projects/default

rm /home/kevin-nadro/Documents/zoo-modeling-app-projects/empty/main.kcl

/home/kevin-nadro/Downloads/Zoo\ Modeling\ App-0.25.5-x86_64-linux.AppImage /home/kevin-nadro/Documents/zoo-modeling-app-projects/empty

• If someone at the office, test you can hit make from the app and it works
• Test updating versions works
• until we have a performance test, sanity check that time to open file directly from command line is not shit
  • xx.xx

/home/kevin-nadro/Downloads/Zoo\ Modeling\ App-0.25.5-x86_64-linux.AppImage /home/kevin-nadro/Documents/zoo-modeling-app-projects/default/chassis-with-rails.kcl

• Electron tests: ensure we can open files that in tauri we would not be able to Electron tests: ensure we can open files that in tauri we would not be able to #3455
  • Opening file
  • Editing file

/home/kevin-nadro/Downloads/Zoo\ Modeling\ App-0.25.5-x86_64-linux.AppImage /home/kevin-nadro/smoke-test-file-permissions-linux/main.kcl

• Electron test: upgrading app should not cause user to have to auth again Electron test: upgrading app should not cause user to have to auth again #3469 issues/3469
• Electron: logging in and authing works Electron: logging in and authing works #3465
• ❗ (Does not work on Linux) double click to open files from finder opens new window each time (mac/windows)
• export from the app works and opens a file picker ux (needs to be manual since we override for playwright)
• Move around and sketch rapidly: is interaction still snappy?

Notes

• Opening a path to a directory that doesn't exist, hangs, should we create the directory?

Loading project at startup: /home/kevin-nadro/Documents/zoo-modeling-app-projects/empty-2
Error: ENOENT: no such file or directory, stat '/home/kevin-nadro/Documents/zoo-modeling-app-projects/empty-2'
[42821:0930/142409.906556:ERROR:gl_surface_presentation_helper.cc(260)] GetVSyncParametersIfAvailable() failed for 1 times!
Checking for update
Update for version 0.25.5 is not available (latest version: 0.25.4, downgrade is disallowed).

• Cannot figure out mod + shift + . on linux for core dump
• Double clicking files on linux does not work.