feat: parallelize text and multimodal processing in process_document_complete#227
feat: parallelize text and multimodal processing in process_document_complete#227ndcorder wants to merge 1 commit intoHKUDS:mainfrom
Conversation
These two steps are independent so there's no reason to wait for text insertion before starting multimodal. Uses asyncio.gather with return_exceptions so one failing doesn't kill the other.
eede248 to
ab1167c
Compare
|
Thanks for your contribution! I found one P1 issue. In |
Text insertion and multimodal processing in process_document_complete run sequentially right now but don't share any state. This runs them concurrently with asyncio.gather so total time is max(text, multimodal) instead of text + multimodal.
Uses return_exceptions=True so if one branch fails the other still completes.