Fix SMC entering an infinite loop when writer and written are in the same page

[pmatos]

There was a case that happens in Crysis2, Resume Game, where it seems like the address writing code, writes to the same page. The flow of code protection ends up causing an infinite loop which ended up causing the game to hang.

[pmatos]

I hope someone with more experience in SMC can take a look. This is fixing Crysis2. Much logging showed that Crysis2 was entering an infinite loop of unprotecting a page and then reprotecting the same page, triggering a segfault and going on and on about this.

One thing that's bothering me is that I haven't managed to write a test case for this.

[bylaws]

I believe this is working around a bug somewhere else (and unprotecting is somewhat broken for actual inline SMC):

    if (CTX->IsAddressInCodeBuffer(Thread, ArchHelpers::Context::GetPc(ucontext)) && !CTX->IsCurrentBlockSingleInst(Thread) &&
        CTX->IsAddressInCurrentBlock(Thread, FaultAddress & FEXCore::Utils::FEX_PAGE_MASK, FEXCore::Utils::FEX_PAGE_SIZE)) {

this block is only intended to pass if the prior codeblock hadn't hit same-page SMC before, entering that block should trigger the memory instruction itself to become a single instruction codeblock and the memory to be protected when its recompiled. However after that, when the single instruction codeblock runs it will avoid this path and take the 'unprotect and mark as dirty but continue in the block' path, which is forward progress and wont infinite loop. I would refer to #4165 for the expected flow.

It would be extremely useful to get a dump of the ASM involved here, it would likely narrow down the issue immediately. Some sort of context reconstruction thing or similar etc.

[bylaws]

.

[pmatos]

I believe this is working around a bug somewhere else (and unprotecting is somewhat broken for actual inline SMC):

    if (CTX->IsAddressInCodeBuffer(Thread, ArchHelpers::Context::GetPc(ucontext)) && !CTX->IsCurrentBlockSingleInst(Thread) &&
        CTX->IsAddressInCurrentBlock(Thread, FaultAddress & FEXCore::Utils::FEX_PAGE_MASK, FEXCore::Utils::FEX_PAGE_SIZE)) {

this block is only intended to pass if the prior codeblock hadn't hit same-page SMC before, entering that block should trigger the memory instruction itself to become a single instruction codeblock and the memory to be protected when its recompiled. However after that, when the single instruction codeblock runs it will avoid this path and take the 'unprotect and mark as dirty but continue in the block' path, which is forward progress and wont infinite loop. I would refer to #4165 for the expected flow.

It would be extremely useful to get a dump of the ASM involved here, it would likely narrow down the issue immediately. Some sort of context reconstruction thing or similar etc.

That doesn't really match what I saw. I see SMC generating segfaults repeatedly on the same address. Unprotecting it, triggering single step, then protecting it again. and so on forever. I have a branch with lots of debugging. I will dig the details and share tomorrow.

[bylaws]

That doesn't really match what I saw. I see SMC generating segfaults repeatedly on the same address. Unprotecting it, triggering single step, then protecting it again. and so on forever. I have a branch with lots of debugging. I will dig the details and share tomorrow.

Yeah, if the first recompile after the single step trigger also triggers single step that's a bug in some core block size or single step logic. If not then it's probably context reconstruction causing e.g. an incorrect RIP that leads single step to step the wrong block

[pmatos]

That doesn't really match what I saw. I see SMC generating segfaults repeatedly on the same address. Unprotecting it, triggering single step, then protecting it again. and so on forever. I have a branch with lots of debugging. I will dig the details and share tomorrow.

Yeah, if the first recompile after the single step trigger also triggers single step that's a bug in some core block size or single step logic. If not then it's probably context reconstruction causing e.g. an incorrect RIP that leads single step to step the wrong block

Thanks. I am looking into this now.

[bylaws]

If you can grab the assembly itself that's failing can take a good guess

[pmatos]

If you can grab the assembly itself that's failing can take a good guess

the assembly itself seems to be generated at runtime. It's quite confusing because it feels (although I am way out of my depth at this point) related to DRM and I was not expecting this at this point (ie when pressing Resume Game). When I get some assembly, I will share but mostly I am working by adding debug prints, compile, running the game, adding more debug prints, compiling, re-running, etc which is pretty slow.

[bylaws]

If you can grab the assembly itself that's failing can take a good guess

the assembly itself seems to be generated at runtime. It's quite confusing because it feels (although I am way out of my depth at this point) related to DRM and I was not expecting this at this point (ie when pressing Resume Game). When I get some assembly, I will share but mostly I am working by adding debug prints, compile, running the game, adding more debug prints, compiling, re-running, etc which is pretty slow.

You should just be able to attach gdb and hexdump at the point it infloops.

Usually just add a print at compileblock for GuestRIP. Then check log and in gdb x/100c