-
Notifications
You must be signed in to change notification settings - Fork 14
Adding webviewer-ask-ai sample #75
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Adding webviewer-ask-ai sample #75
Conversation
A brand-new sample related to https://apryse.atlassian.net/browse/AS-196
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Overall it looks pretty good, Some changes are needed to match AC and allowing for the LLM to get the chat history on each call is important to conversing with it about the document by asking follow up questions.
| spinner.stop(); | ||
| }; | ||
|
|
||
| window.chatbot.getAllText(contentItem.promptType, callbackWrapper); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Everytime one of the selections is clicked the document is processed and either summarized or key words listed.
AC 2 & 3 state this should be done once on load and clicking the button will load the results instantly instead the llm is doing the work each time one of the buttons is clicked
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think the way we have it here is fine, we don't need call the LLM on document load, this isn't how a real world app would work
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Oh okay. Still might be good to cache that summary after the first click to use in follow up questions later if needed instead of passing in the entire document on each message (for long documents).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I agree with @lbittner-pdftron and @DavidEGutierrez.
| customUIConfig = customUIConfig.replaceAll("{{APP_SITE_NAME}}", APP_SITE_NAME); | ||
| customUIConfig = customUIConfig.replaceAll("{{ASK_WEB_SDK_ICO}}", ASK_WEB_SDK_ICO); | ||
|
|
||
| await instance.UI.importModularComponents(JSON.parse(customUIConfig), functionMap); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
All the chat state is lost when opening and closing the chat window. This is probably not ideal and perhaps an option to clear the chat would be better in case the user wants to minimize the chat temporarily without losing the conversation.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@Mohammed-AbdulRahman-Apryse this makes sense to implement. I experience the same issue when closing the panel and opening, it is not persistent and generates a brand-new instance.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
-
Concerning All the chat state is lost when opening and closing the chat window, this has been fixed in Mohammed-AbdulRahman-Apryse@09dbab4.
-
Concerning an option to clear the chat, I believe the clear the chatting is reasonable when opening a new document to be processed. This also is fixed.
| }, | ||
| // Handle selected text summary popup click | ||
| 'askWebSDKPopupClick': () => { | ||
| createBubble('Summarize the selected text.', 'user'); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
AC 4 & 5
The selection should be added to the chat and a default suggestion should be presented "summarize the document". I'm guessing the user should have the option to summarize when clicking that button or ask questions directly about the selection.
one option is to add a message like:
"please summarize the following text"
{inserted selection}
that way the chat history will give the llm better context
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@Myles-Hagen Thanks for the feedback.
Adding the selected text to the chat from the context-menu option makes sense in this case, @Mohammed-AbdulRahman-Apryse can add that the selected text.
For the other issue, I am not sure about adding another option after the selection to "summarize the document". The context menu is already set as the option to "summarize" the selected text. If the user has a free-text query they can type it in the input box at any time. Or they can use the 2 predefined links to summarize the document or extract keywords.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This has been fixed in Mohammed-AbdulRahman-Apryse@09dbab4.
| let pageNumber = 1; | ||
| // match to be turned into link | ||
| matches.forEach(match => { | ||
| pageNumber = match.match(/\d+/)[0]; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Sometimes citations are present but not clickable when asking for the document summary. When I highlighting text and get the summary there are no citations and also when I ask a question in the chat about the document there are no citations
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@Mohammed-AbdulRahman-Apryse review to make sure we have citations that are clickable, add citations on selected text, on summary link clicks, and free-text query from input.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The citation is clickable when the current viewable page is different than the citation page number. When clicking the citation and the webviewer is showing the same page number, it has no effect.
Concerning highlighting text and get the summary there are no citations, this has been fixed in Mohammed-AbdulRahman-Apryse@09dbab4.
webviewer-ask-ai/server/handler.js
Outdated
| // For other prompt types, use first chunk with warning | ||
| const chunks = chunkText(message, 12000); | ||
| const messages = [ | ||
| new SystemMessage(`${promptSettings.systemPrompt}\n\nNOTE: This document was too long, so only the first section is being processed.`), |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The document should be summarized on the initial chat load so if the entire document is too long then sending the summary for subsequent requests could be a good option to keep token count managable.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@Myles-Hagen I believe this is the same issue as the first point on this list and @lbittner-pdftron wrote a comment about it.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I agree with @DavidEGutierrez
| llm.temperature = promptSettings.temperature; | ||
| llm.maxTokens = promptSettings.maxTokens; | ||
|
|
||
| const response_data = await llm.invoke(messages); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@Mohammed-AbdulRahman-Apryse I thought keeping chat history was not required. If it is, then we need to enable it.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The chat history is enabled.
This has been fixed in Mohammed-AbdulRahman-Apryse@09dbab4.
- The chat history is added in: A. client\chatbot.js B. server\handler.js - Refactoring the page citation links in: client\chatbot.js - Maintaining the conversation sequence in: A. client\config\ui\functionMap.js B. client\globals.js - Option to summarize when clicking that button or ask questions directly about the selection in: client\config\ui\functionMap.js - Maintaining Artificial Intellegance terms by changing "System" into "Assistant" and "User" into "Human" in: A. client\chatbot.js B. client\config\ui\functionMap.js C. client\config\ui\styles.css D. server\handler.js - Added githubId value in mainsamplesource.json - Added sample name to lerna.js - Added the sample name to the generic webviewer-samples readme
|
The attached video is addressing the fixes. |
|
@Mohammed-AbdulRahman-Apryse Just a workflow thing to make sure we keep making progress - when you have addressed feedback, make sure to re-request a review. Thanks! |
The update allows the user to select text either from a single page or multiple pages. The affected file is webviewer-ask-ai\client\index.js
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
looks good, chat history is working, state preserved when modal closed and reopened and all links styled and clickable.

A brand-new sample related to https://apryse.atlassian.net/browse/AS-196
Description
Ask Web SDK - AI Sample
A WebViewer sample with custom panel added to UI to access a chat interface to ask questions about the loaded document. The chatbot panel will open immediately once the document loaded, with 2 canned default messages can be selected:
Summarize document
List keywords
If one of the suggestions is clicked, it is immediately added to the chat to be processed.
Also, select content in the document feature is available, with summarize selection button is added to the selection pop up menu to add the selection to the chat.
The chatbot results include an inline citation that links to the cited page.
The sample supports guardrails to only ask questions about the loaded document, and if I try to ask questions outside the document scope then a polite message is provided.
Resources
Available under "Additional information" from the task description.
Checklist
If you are adding a new sample
lerna.jsonwith the new sample nameIf you are removing an old sample
lerna.json