Add rich reply support

[9viz]

This PR fixes #57. I posted a preliminary patch in #57 and daily drove it ever since I posted it. Although I haven't used matrix much these past three--four weeks, I tested this current PR with both plain text and HTML body rendering. AFAIK I haven't broke anything, yet.

Hoping to hear from you,

[alphapapa]

Hi Visuwesh,

Thanks, this looks good. Please rebase the branch on current master. Then I'll plan to merge this for the next release.

[9viz]

Oops, forgot that you prefer rebase over merge, now done.

[alphapapa]

Hi @Vizs,

I pushed another commit to your branch that makes some minor changes. It also adds two FIXMEs that should probably be addressed before merging. Please review my changes and let me know what you think. Thanks.

[9viz]

Rest of the changes look fine to me, thanks for the cleanup!

[alphapapa]

@Vizs Thanks. Did you see the two FIXME comments? How do you think we should handle them?

Also, do you know of a good way to test this new code? e.g. could you set up a room with a few "rich reply" events that we could view in Ement to ensure that it works correctly?

[9viz]

(Did my "review comments" not go through?)

I say we leave the first FIXME in until people start to complain. I haven't encountered the situation you highlight in my testing period and I encounter rich-replies practically all the time when I use matrix since the t2bot's discord bridge uses rich replies.

As for the second FIXME, what do you think about the following function instead?

(defun ement-room--format-quotation-text (quoted-event)
  "Return text for QUOTED-EVENT."
  (pcase-let* (((cl-struct ement-event sender (content (map body))) quoted-event)
               ((cl-struct ement-user (id sender-id)) sender))
    (with-temp-buffer
      (insert "> " "<" sender-id ">" body)
      (goto-char (point-min))
      (forward-line 1)
      (while (not (eobp))
        (insert "> ")
        (forward-line 1))
      (insert "\n")
      (buffer-string))))

As for a room with rich replies, I invited you to one named "testytestytest". Please have a look at message events after "TEST FOR RICH REPLY". I used the following patch to make ement.el produce rich-replies,

diff --git a/ement-lib.el b/ement-lib.el
index 06c98ab..74b93fd 100644
--- a/ement-lib.el
+++ b/ement-lib.el
@@ -1092,7 +1092,10 @@ e.g. `ement-room-send-org-filter')."
     (when filter
       (setf content (funcall filter content room)))
     (when replying-to-event
-      (setf content (ement--add-reply content replying-to-event room)))
+      (setq content (cons `(m.relates_to (m.in_reply_to (event_id . ,(ement-event-id replying-to-event))))
+                          content))
+      ;;(setf content (ement--add-reply content replying-to-event room))
+      )
     (ement-api session endpoint :method 'put :data (json-encode content)
       :then (apply-partially then :room room :session session
                              ;; Data is added when calling back.

[alphapapa]

(Did my "review comments" not go through?)

I can't find any. Maybe you forgot to press the "Finish review" button. I've done that before, because the UI is non-obvious there.

[alphapapa]

Thanks, I agree with those solutions.

One more thing: I think we should have an option to make Ement send rich replies too. Would you be interested in adding that feature as well?

[9viz]

I pushed two more commits to quote prefix each line in the reply, and also a new user option to send rich replies.

I can't find any. Maybe you forgot to press the "Finish review" button. I've done that before, because the UI is non-obvious there.

Indeed, looks like I didn't do it. But anyway, all my comments are addressed now.

[alphapapa]

@Vizs I rebased this branch onto master and pushed some small changes on top, including linting with makem.sh again. Please review the new commits I pushed and the current diff against master and let me know what you think. If it looks good to you, I'll merge to master and we can start testing it "in anger" in preparation for the next version release. Thanks.

[9viz]

Thanks, everything looks good! As for the FIXME you added,

diff --git a/ement-room.el b/ement-room.el
index 08d79db..8060ee6 100644
--- a/ement-room.el
+++ b/ement-room.el
@@ -3457,6 +3457,7 @@ If FORMATTED-P, return the formatted body content, when available."
           :then (let ((room ement-room)
                       (session ement-session))
                   (lambda (fetched-event)
+                    ;; FIXME: Do we need to use `when'?  Shouldn't `fetched-event' always be present?
                     (when fetched-event
                       (pcase-let* ((new-event (ement--make-event fetched-event))
                                    ((cl-struct ement-room (local (map buffer))) room))

I simply cannot recall why I added the `when', if you want we can drop it and add it back when someone (TM) complains.

[alphapapa]

I simply cannot recall why I added the `when', if you want we can drop it and add it back when someone (TM) complains.

AFAIK, if this function is called, it means the request succeeded, which should mean that the value will be present; and if the value is nil, that would indicate a problem that we should probably allow to signal an error.

So if you remove the wrapping when form and the code still works, then I think we can safely remove it.

[9viz]

Very quick testing with the when removed tells me that everytrhing works fine, so we can remove it.

[alphapapa]

@Vizs Ok, thanks. I pushed another commit doing that.

However, before merging this, I looked at the spec again: https://spec.matrix.org/v1.3/client-server-api/#rich-replies IIUC, to support it correctly, we also need to:

• Include fallback quotes. Probably we need to also call ement--add-reply even when sending rich replies. (And maybe we should move the setting of the relationship metadata into that function, and do it uncondtionally, maybe even removing the option.)
• Strip fallback quotes from events that include rich replies. We already have the code to fetch the replies and re-render the event.
• Ensure that our rendered replies also handle message types other than m.text (as in 1, 2, and 3).

What do you think? Thanks.

[9viz]

Include fallback quotes. Probably we need to also call ement--add-reply even when sending rich replies. (And maybe we should move the setting of the relationship metadata into that function, and do it uncondtionally, maybe even removing the option.)

Oh yes, this was a misunderstanding on my part on how rich replies work. I misread/misunderstood the situation and thought fallback quotes need not be added when you're sending rich-replies since this PR was sparked by messages that did not include fallback quotes. You're right, we need to add fallback quotes unconditionally and only need to add the m.relates_to part if we are concerned with sending rich-replies.

Strip fallback quotes from events that include rich replies. We already have the code to fetch the replies and re-render the event.

As of now, well behaved clients always include the fallback quotes but I don't find it necessary to strip them except for rare cases when clients tend to truncate the reply's body however, stripping should be easy for HTML messages but something deep in me says that plain text bodies will not be fun to handle. I say we don't bother about plain text bodies for now and worry about it later since it is always a second-class citizen in the matrix world anyway. :(

Ensure that our rendered replies also handle message types other than m.text (as in 1, 2, and 3).

Hmm... if we are sure that we don't want to deal with stripping fallback quotes, I think we can live without having support for these just yet.

However, if you think stripping is necessary, I can implement it but unfortunately I am kind of busy these days with a whole load of quizzes and generally busy semester (+ prep for my masters project 🙃).

[phil-s]

Coming back to this after a long hiatus to review that diff...

Obviously we don't actually want the trailing whitespace here, but removing it wasn't part of the original branch. It was the merge commit 9viz@da037e5 which took it out.

modified   ement-lib.el
@@ -32,7 +32,7 @@
   (require 'ewoc)
   (require 'pcase)
   (require 'subr-x)
-
+  
   (require 'taxy-magit-section)
 
   (require 'ement-macros))

---

This next part was from commit 6ec7d31 and so I believe we want this change:

modified   ement-lib.el
@@ -1143,10 +1143,9 @@ ement-send-message
       (setf content (funcall filter content room)))
     (when replying-to-event
       (setf replying-to-event (ement--original-event-for replying-to-event session))
-      (when ement-room-send-rich-replies
-          ;; TODO: Submit a patch to Emacs to enable `map-nested-elt' to work as a generalized variable.
-          (setf (map-elt (map-elt (map-elt content 'm.relates_to) 'm.in_reply_to) 'event_id)
-                (ement-event-id replying-to-event)))
+      ;; TODO: Submit a patch to Emacs to enable `map-nested-elt' to work as a generalized variable.
+      (setf (map-elt (map-elt (map-elt content 'm.relates_to) 'm.in_reply_to) 'event_id)
+            (ement-event-id replying-to-event))
       (setf content (ement--add-reply content replying-to-event room)))
     (ement-api session endpoint :method 'put :data (json-encode content)
       :then (apply-partially then :room room :session session
modified   ement-room.el

---

The original commit for this next bit was 9viz@f67698a so it was a rebase accident that these lines were removed in the rebased commit 9viz@80a0bf3, and so we do want this change:

@@ -2256,14 +2256,16 @@ ement-room-write-reply
     (ement-room-with-highlighted-event-at (point)
       (pcase-let* ((room ement-room)
                    (session ement-session)
+                   (prompt (format "Send reply (%s): " (ement-room-display-name room)))
+                   (ement-room-read-string-setup-hook
                     (lambda ()
                       (setq-local ement-room-replying-to-event event)))
                    (body (ement-room-with-typing
                            (ement-room-read-string prompt nil 'ement-room-message-history

---

And I believe this last part is good too:

modified   ement-room.el
@@ -2256,14 +2256,16 @@ ement-room-write-reply
                       (setq-local ement-room-replying-to-event event)))
                    (body (ement-room-with-typing
                            (ement-room-read-string prompt nil 'ement-room-message-history
-                                                   nil 'inherit-input-method)))
-                   (replying-to-event (ement--original-event-for event ement-session)))
-        (ement-room-send-message room session :body body :replying-to-event replying-to-event
-                                 :rich-reply ement-room-send-rich-replies))))
+                                                   nil 'inherit-input-method))))
+        ;; NOTE: `ement-room-send-message' looks up the original event, so we pass `event'
+        ;; as :replying-to-event.
+        (ement-room-send-message room session :body body :replying-to-event event)))))
 
 (when (assoc "emoji" input-method-alist)
   (defun ement-room-use-emoji-input-method ()

The only two commits touching this code in the original rich-replies branch were commits 9viz@cc0c97d and 9viz@f67698a with the latter reverting the change from the former.

The initial rebase wasn't taking commit 3df2371 into account, and it looks correct for us to end up with that code given that the other two commits cancelled each other out (for this bit of code), so I believe this is correct too:

@@ -2262,9 +2262,10 @@ ement-room-write-reply
                       (setq-local ement-room-replying-to-event event)))
                    (body (ement-room-with-typing
                            (ement-room-read-string prompt nil 'ement-room-message-history
-                                                   nil 'inherit-input-method)))
-                   (replying-to-event (ement--original-event-for event ement-session)))
-        (ement-room-send-message room session :body body :replying-to-event replying-to-event)))))
+                                                   nil 'inherit-input-method))))
+        ;; NOTE: `ement-room-send-message' looks up the original event, so we pass `event'
+        ;; as :replying-to-event.
+        (ement-room-send-message room session :body body :replying-to-event event)))))

---

All in all, I think my rebase is good, so I suggest doing a hard-reset of 9viz/feature/rich-replies to that state, and pushing that back for this pull request:

git checkout feature/rich-replies
git remote add phil-s https://github.com/phil-s/ement.el.git --track phil-s/rich-replies --fetch
git reset --hard phil-s/phil-s/rich-replies
git push --force-with-lease origin feature/rich-replies

n.b. I've also rebased my branch over master again, which happened without conflicts.

[phil-s]

I've removed 9viz@787f537 from my branch, as that was reverting a whitespace change which, in the meantime, got fixed upstream; so we don't want to re-introduce that whitespace now.

[9viz]

Thank you so much for all the work! I never would have managed to clear up this mess.

I agree with all your suggestions. I will follow the instructions shortly and update my branch.

[phil-s]

Now... to figure out what all this actually does so that I can test it :)

I've merged it to the branch I'm using as my daily-driver, so I'll follow up if I encounter any problems with replies.

[9viz]

Off the top of mind: some matrix client cutoff the replied-to message so you don't get to see the full message anymore. This patch should show you the full message.

Quoting is a bit better when you use the plaintext format for messages. But this is rare though since I gave up on stripping off quotes from plain text messages since they are unreliable (plain text message format is anyone's game).

There is still work to do: Rich replies aren't reliable when the replied-to message is edited. I think I fixed it but there are some edge cases left that I didn't have the energy / time to work on this April. The leftover bits should be documented in this issue before the great rebase mess.

[9viz]

Followed your instructions, I hope it is all clear now.

[phil-s]

Confirming that this PR now matches my branch. Looks good.

[phil-s]

I note that if I reply to a message and then edit my reply, I still get the plain text quote injected into the text I'm editing. If I fail to remove it then, after submitting, I wind up seeing a message with the rich quote at the top, then the plain text quote, and then my reply.

Presumably we just want to inhibit the insertion of the plain text quote into the minibuffer/compose-buffer when editing replies?

[phil-s]

There is still work to do: Rich replies aren't reliable when the replied-to message is edited. I think I fixed it but there are some edge cases left that I didn't have the energy / time to work on this April.

I've noted that the quoted text in the replies to a message isn't updated when the original message is edited (whereas Element does update the quotes).

I'm honestly not a fan of what Element does there. Sure, it's nice if my typos aren't captured for all time because someone replied before I noticed; but by the same token there's scope for the sender of the original message to radically alter the apparent meaning of a reply by changing the quoted text to something very different to what it was at the time of the reply. If it's a choice between those two things, I prefer what I'm seeing in Ement.el right now.

[phil-s]

Quoting is a bit better when you use the plaintext format for messages. But this is rare though since I gave up on stripping off quotes from plain text messages since they are unreliable (plain text message format is anyone's game).

Because I use org-mode for composing by default, I actually never send plaintext messages (unless taking pains to specifically test that), so I'll not be exercising that functionality much. (Tangent: I think it would be nice if Ement noticed when the org-exported message was identical to the original source text, and sent it as plain text rather than as HTML.)

I'll naturally reply to plain text messages from other people from time to time, but I don't think that's what you were referring to?

[9viz]

I note that if I reply to a message and then edit my reply, I still get the plain text quote injected into the text I'm editing. If I fail to remove it then, after submitting, I wind up seeing a message with the rich quote at the top, then the plain text quote, and then my reply.

Presumably we just want to inhibit the insertion of the plain text quote into the minibuffer/compose-buffer when editing replies?

Hmm, I might have observed this but I have to look into it to be sure. I am not sure when I can get back to this though since I'm tight on time with the semester end nearing.

Re: 2 msg: While I agree with your statement, it might be better to follow what Element does. I am not sure if the spec touches on this topic though.

Quoting is a bit better when you use the plaintext format for messages. But this is rare though since I gave up on stripping off quotes from plain text messages since they are unreliable (plain text message format is anyone's game).

Because I use org-mode for composing by default, I actually never send plaintext messages (unless taking pains to specifically test that), so I'll not be exercising that functionality much. (Tangent: I think it would be nice if Ement noticed when the org-exported message was identical to the original source text, and sent it as plain text rather than as HTML.)

I'll naturally reply to plain text messages from other people from time to time, but I don't think that's what you were referring to?

No, I did not mean you replying to plain text messages. I meant having %b instead of %B in ement-room-message-format-spec. In this case, we currently make no effort to strip off the quoted message.

As for your tangent: when you send a message in org, ement.el currently sends the org source text as plain text body, and the exported HTML as the formatted body (HTML format) IIRC. I am not sure if there's anything more to do here.