Persian (RTL) text renders incorrectly and inconsistently across operating systems in DeltaChat

[md4956]

• Description:
  DeltaChat does not display Persian phrases properly. The rendering behavior varies depending on the operating system. In some cases, characters appear disjointed or in the wrong order. In other cases, the text direction (right-to-left) is ignored, causing punctuation and spaces to appear at the wrong end of the string.

• Operating System (Linux/Mac/Windows/iOS/Android):
  Mac, iOS, Android
  (and probably Linux and Windows as well - wasn't able to test)

• Delta Chat Version:
  iOS: v2.48.0
  Mac: v2.48.0
  Android: v2.49.0

• Expected behavior:
  All Persian text should render correctly in right-to-left order. Mixed English-Persian sentences should maintain proper bidirectional (bidi) ordering according to Unicode standards. The same message should appear identically across all operating systems where DeltaChat is supported.

• Actual behavior:
  The rendering of Persian text is inconsistent. On some OS versions, words and phrases appear separated or reversed. On others, mixed English-Persian lines show incorrect text direction, with punctuation or spaces appearing at the left edge instead of the right.

• Steps to reproduce the problem:

1. Open DeltaChat on multiple different operating systems (e.g., Android and Windows, or iOS and Linux).
2. Send a message containing Persian+English text from either system to another.
3. Observe the display of the Persian+English phrase on both the sending and receiving devices.

• Screenshots:

• Logs:

• Additional notes:
  -- The same Persian text displays correctly in other messaging apps (e.g., Telegram, WhatsApp, Signal) on the same devices.
  -- The issue appears OS-dependent rather than solely DeltaChat version dependent.

• Sample combinations to test and compare:

1. English only
The quick brown fox jumps over the lazy dog near the riverbank.

2. Persian only
روباه قهوه‌ای سریع از روی سگ تنبل پرید و کنار رودخانه فرود آمد.

3. English with one Persian phrase in the middle
The design process requires توجه به جزئیات for a successful outcome.

4. Persian with one English phrase in the middle
فرآیند طراحی نیاز به attention to detail برای نتیجه موفق دارد.

5. English with multiple Persian phrases in the middle, begin and end with English
We believe that همکاری تیمی and ارتباط شفاف are essential, yet نوآوری مداوم remains the key driver.

6. Persian with multiple English phrases in the middle, begin and end with Persian
به نظر ما teamwork و clear communication ضروری هستند، اما continuous innovation همچنان محرک اصلی باقی می‌ماند.

7. Alternating words (Eng-Persian-Eng-Persian)
This این simple یک mixed جمله pattern الگو works.

8. Persian sentence starting with an English word
Hello به همکاران جدید خوش آمدید. امروز روی پروژه اصلی کار می‌کنیم.

9. English sentence starting with a Persian word
سلام everyone, welcome to the new team. Today we will work on the main project.

10. Fully mixed in short phrases (no clear dominant language)
کاربر باید click کند و then وارد صفحه new شود. بعد از آن، save را بزند.

11. Persian with English inside parentheses
مراحل اجرا (implementation steps) شامل سه فاز (design, testing, deployment) می‌باشد.

12. English with Persian inside parentheses
The final review (بررسی نهایی) will take place on Friday (جمعه) next week.

13. One language as subject, the other as rest of sentence
مدیریت پروژه is the most critical factor for success in agile environments.

14. Persian quote inside English sentence
He kept saying, «فردا کار را انجام می‌دهم», but nothing ever got done.

15. English quote inside Persian sentence
همیشه به خود می‌گفت "I'll do it tomorrow" اما هیچ‌وقت کاری انجام نمی‌شد.

[WofWca]

Is this only about the HTML viewer?

Which specific pieces of text are displayed incorrectly?

Is the text displayed correctly in the Web version of Telegram or WhatsApp on Chromium?

Either way I doubt that this is rooted in our code base.

[md4956]

1. It needs the fix on both HTML viewer and the chat message itself,
2. In the combined phrases, certain rules needs to be applied to the text direction and word placing in order to keep correct format,
3. Did a test on both web.whatsapp and web.telegram right now, and compared. Both are doing a better job, and correctly structuring the phrase with mixed languages. Telegram web does it the best by far. I've attached screenshot of the same examples on both.

[WofWca]

Related: chatmail/core#8227

[WofWca]

Let's keep open because the "truncated message" case is still not fixed.

[WofWca]

Also we haven't notified Android and iOS of this issue.

[md4956]

Just to have it here as well:

deltachat/deltachat-ios#3110 (comment)

[nicodh]

Closing as nothing is left to do for desktop