fix!: Ensure all open() uses UTF-8 instead of default encoding.

[karthikbekalp]

What was the problem/requirement? (What/Why)

Cinema 4D submissions fail to run if filenames contain non-ASCII values. For example, a scene file named TestSceneñ.c4d would not render.

openjd_fail: Error encountered while starting adaptor: 
Cinema4D Encountered an Error: FileNotFoundError: The scene file 'C:\some_path\TestñSceneñ.c4d' does not exist

What was the solution? (How)

While loading the JSON/YAML files, the default open() uses system's encoding standard which is not UTF-8.
This causes issues in other DCCs when they try to read the data.

To ensure that we fix all the issues with encoding I also updated all the open() calls to also use UTF-8. Kudos to @mwiebe for pointing that out.

There was also an issue where the logger used to print out few errors unable to convert non-ASCII characters.

2024/12/18 15:48:43-06:00 UnicodeEncodeError: 'charmap' codec can't encode character '\u20bf' in position 182: character maps to <undefined>
2024/12/18 15:48:43-06:00 Call stack:
2024/12/18 15:48:43-06:00   File "<frozen runpy>", line 198, in _run_module_as_main
2024/12/18 15:48:43-06:00   File "<frozen runpy>", line 88, in _run_code
2024/12/18 15:48:43-06:00   File "C:\Program Files\Python312\Scripts\cinema4d-openjd.EXE\__main__.py", line 7, in <module>
2024/12/18 15:48:43-06:00     sys.exit(main())
2024/12/18 15:48:43-06:00   File "C:\Program Files\Python312\Lib\site-packages\deadline\cinema4d_adaptor\Cinema4DAdaptor\__main__.py", line 25, in main
2024/12/18 15:48:43-06:00     _EntryPoint(Cinema4DAdaptor).start(reentry_exe=reentry_exe)
2024/12/18 15:48:43-06:00   File "C:\Program Files\Python312\Lib\site-packages\openjd\adaptor_runtime\_entrypoint.py", line 309, in start
2024/12/18 15:48:43-06:00     return self._handle_daemon(
2024/12/18 15:48:43-06:00   File "C:\Program Files\Python312\Lib\site-packages\openjd\adaptor_runtime\_entrypoint.py", line 438, in _handle_daemon
2024/12/18 15:48:43-06:00     frontend.start()
2024/12/18 15:48:43-06:00   File "C:\Program Files\Python312\Lib\site-packages\openjd\adaptor_runtime\_background\frontend_runner.py", line 241, in start
2024/12/18 15:48:43-06:00     self._heartbeat_until_state_complete(AdaptorState.START)
2024/12/18 15:48:43-06:00   File "C:\Program Files\Python312\Lib\site-packages\openjd\adaptor_runtime\_background\frontend_runner.py", line 293, in _heartbeat_until_state_complete 

To fix this, I added sys.stdout.reconfigure(encoding="utf-8") to also force stdout to use UTF-8.

What is the impact of this change?

This fixes the issues with non-ASCII characters.

How was this change tested?

While renders now work for characters like

• Symbols like for Bitcoin ₿
• Latin character ę
• Greek character β
• Cyrillic Б
• and many more

They still fail for emoji and some mathematical operators. Cinema 4D's load document function returns None without details on why it fails.

• Have you run the unit tests?

Yes

Was this change documented?

Yes.

Is this a breaking change?

I discussed this with @AWS-Samuel and considering the packages are public, we have to operate under the assumption that customers are using the packages in their own, potentially private integrations. So even if we handle the change gracefully in all our integrations its still a breaking change.

If so, then please describe the changes that users of this package must make to update their scripts, or Python applications.

Users of this package must update their dependent packages to ensure that they use "UTF-8" encoding as well.

Does this change impact security?

• Does the change need to be threat modeled? For example, does it create or modify files/directories that must only be readable by the process owner?
  No
  • If so, then please label this pull request with the "security" label. We'll work with you to analyze the threats.

---

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

[sonarqubecloud]

Quality Gate passed

Issues
1 New issue
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

[crowecawcaw]

Looks good. Can we also add a test or linter rule that will require the encoding format to be specified for all open() calls? It seems very possible someone will forget it in the future, and we'd be unlikely to notice until we get a weird bug later (like the one you were digging into).

[karthikbekalp]

Looks good. Can we also add a test or linter rule that will require the encoding format to be specified for all open() calls? It seems very possible someone will forget it in the future, and we'd be unlikely to notice until we get a weird bug later (like the one you were digging into).

Hmm, interesting. I'll add it in an upcoming PR as we need to get this in ASAP so that we can solve customer's issue.