Fix a crash in BackgroundTaskManager when background task initialization races with early call termination#1360
Fix a crash in BackgroundTaskManager when background task initialization races with early call termination#1360zscgeek wants to merge 1 commit intojambonz:mainfrom
Conversation
…ion races with early call termination (e.g., quick CANCEL). Add defensive guards and ensure partial tasks are not tracked. Repro (observed in production) Inbound INVITE followed by quick CANCEL Feature server attempts to initialize background transcribe; logs: “BackgroundTaskManager:initTranscribe - Error creating transcribe task” “Response#send: final response already sent” Teardown triggers BackgroundTaskManager.stopAll() Crash: TypeError: Cannot read properties of undefined (reading 'end') at background-task-manager.js:67:17 Root cause stop/_taskCompleted/_taskError unconditionally call task.span.end() Init methods return a task even if initialization fails before task.span assignment newTask stores tasks regardless of initialization completeness Changes Guard task.span.end() in stop, _taskCompleted, _taskError Only track tasks in newTask when task.span exists (fully initialized) In _initListen, _initBargeIn, _initTranscribe, _initTtsStream, return undefined in catch blocks Why this approach Keeps existing behavior intact for successful flows while making teardown idempotent and safe Eliminates class of crashes from partial initialization and teardown races Compatibility No API changes Logging preserved; spans end when present Tests/Follow-ups Suggest adding a test where an INVITE is canceled before answer and verify no crash, and that background tasks are not tracked if init fails. Consider deferring some background tasks until after upstream 200 OK/ACK to reduce early-race exposure. (cherry picked from commit 115dd70)
|
why closed @zscgeek ? Is another PR coming? |
|
closing while i complete more testing - i think it's good but had meant to open as draft while I validate 100% for sure |
|
btw i'm fine to reopen if you guys want to review in parallel |
|
@davehorton @sammachin - We've tested this more in our environment, and it seems to be working well, fixing the crashes we were seeing on inbound call race conditions. |
|
@xquanluu can you review? |
|
Thanks @zscgeek all of background tasks we currently have required to have media endpoint from freeswitch and should be enabled after call is answered. even in Config verb, if background transcribe is used, TaskPreconditions.Endpoint should be checked. We cannot have a case that the call initiated quickly got cancel, and transcribe task should be running. Do you have log for this call? could you send it to support@jambonz.org please. |
BackgroundTaskManagerwhen background task initialization races with early call termination (e.g., quick CANCEL). Add defensive guards and ensure partial tasks are not tracked.BackgroundTaskManager.stopAll()stop/_taskCompleted/_taskErrorunconditionally calltask.span.end()task.spanassignmentnewTaskstores tasks regardless of initialization completenesstask.span.end()instop,_taskCompleted,_taskErrornewTaskwhentask.spanexists (fully initialized)_initListen,_initBargeIn,_initTranscribe,_initTtsStream, returnundefinedin catch blocks