Fix crash in DX11 unmap if map was called before frame capture #3601
Conversation
The problem can be reproduced by calling map of a DX11 buffer and then keeping that open until a frame is captured in which unmap is called. The call to ```cpp bool found = FindDiffRange(MapWrittenData, record->GetShadowPtr(ctxMapID, 1), len, s, e); ``` would crash as `record->GetShadowPtr(ctxMapID, 1)` returns nullptr and `FindDiffRange` does not guard against this.
Sanakan8472
force-pushed
the
DX11UnmapCrash
branch
from
e18fa92 to
8f3076b
Compare
|
This change does not seem like the right fix and it should be handled differently - in general unmapping a buffer on DX11 during a frame capture that was not mapped during that frame should be detected and should cause the entire frame capture to fail and have to retry the next frame. Can you please open a bug report for this and include a reproducible test case so that this can be investigated? |
Thanks for the quick response @baldurk! As a bit of context, the project I'm working on has a bunch of mapped temp buffers that multiple background threads can copy into. This prevents the need for thread synchronization when accessing |
|
RenderDoc can't handle mapping a buffer prior to capturing and then unmapping it during capture because the map needs to be intercepted on D3D11, but the overhead of intercepting all maps is too high. If this is encountered RenderDoc will fail to capture the desired frame and instead retry on the next frame (up to 5 times). As long as I can reproduce the problem it doesn't have to be minimally sized. |
The problem can be reproduced by calling map of a DX11 buffer and then keeping that open until a frame is captured in which unmap is called. The call to
would crash as
record->GetShadowPtr(ctxMapID, 1)returns nullptr andFindDiffRangedoes not guard against this.