[Windows] FAST_FAIL_INVALID_SET_OF_CONTEXT crash when running in CFG-enabled process

[Barrixar]

Unicorn emulator crashes with STATUS_STACK_BUFFER_OVERRUN (0xC0000409 / FAST_FAIL_INVALID_SET_OF_CONTEXT) when running in a process that has Windows Control Flow Guard (CFG) enabled. This affects PyInstaller-bundled applications on Windows 10/11.

Related (may need cross-reference):

• PyInstaller: https://github.com/pyinstaller/pyinstaller
• Speakeasy: https://github.com/mandiant/speakeasy

---

Environment

• OS: Windows 11 (Build 26200)
• Python: 3.14.2
• Unicorn: 2.1.4 (pip package)
• PyInstaller: 6.17.0
• Speakeasy: 1.6.1 (uses Unicorn internally)
• Architecture: x86_64

Windows Version Context

CFG enforcement varies by Windows version:

• Windows 10 1709+: CFG enforced by default on signed binaries
• Windows 11: CFG enforced more strictly
• Tested on: Windows 11 Build 26200

poc_unicorn_cfg.py

requirements.txt

This issue likely affects all Windows 10/11 systems where the PyInstaller bootloader is CFG-enabled.

Problem Description

When a Python application using Unicorn is bundled with PyInstaller and executed, it crashes immediately when Unicorn's emulation callbacks return. The crash occurs in VCRUNTIME140!longjmp due to Windows CFG validation failure.

Key Observation

Execution Method	CFG Status	Result
python script.py	CFG=disabled	✅ Works perfectly
PyInstaller.exe	CFG=enabled	❌ Crashes

The PyInstaller bootloader executable is compiled with CFG enabled (/GUARD:CF linker flag), which causes the crash when Unicorn's internal longjmp is invoked.

Root Cause Analysis

1. Unicorn uses setjmp/longjmp for internal state management (see qemu/util/setjmp-wrapper-win32.asm)
2. PyInstaller's bootloader has CFG enabled - the bundled EXE inherits CFG from the bootloader
3. Windows CFG validates longjmp targets - when longjmp is called, CFG checks if the target is a valid call target
4. Unicorn's callback return path fails CFG validation - the longjmp target isn't registered as a valid CFG target

Evidence from Unicorn Source

Unicorn already has a custom setjmp wrapper (qemu/util/setjmp-wrapper-win32.asm):

; Why do we need this wrapper?
; Short answer: Windows default implementation of setjmp/longjmp is incompatible with generated code.

And the .NET bindings explicitly check for CFG (bindings/dotnet/UnicornEngine/WinNativeImport.fs):

if (Flags &&& 0x1u) <> 0u then
    raise <| ApplicationException("Control Flow Guard (CFG) is enabled. Unicorn cannot run with CFG enabled.")

This shows the Unicorn team is aware of CFG incompatibility, but only the .NET bindings have a runtime check.

Crash Details

Error Code (Verified)

Exit Code: -1073740791 (decimal)
          0xC0000409 (hex)
          STATUS_STACK_BUFFER_OVERRUN
          Subcode: FAST_FAIL_INVALID_SET_OF_CONTEXT (10)

PowerShell verification:

# After running the crashing EXE:
$LASTEXITCODE                           # Shows: -1073740791
'0x{0:X}' -f [uint32]$LASTEXITCODE      # Shows: 0xC0000409

Approximate Call Stack (reconstructed from error analysis)

VCRUNTIME140!longjmp
  -> CFG validation fails when returning from Unicorn callback
  -> FAST_FAIL triggered

Note: Full WinDbg crash dump not captured. The crash happens after the hook callback executes but before control returns to emulation loop.

CFG Policy Check

import ctypes
from ctypes import wintypes, sizeof, byref

kernel32 = ctypes.windll.kernel32

class CFG_POLICY(ctypes.Structure):
    _fields_ = [("Flags", wintypes.DWORD)]

policy = CFG_POLICY()
kernel32.GetProcessMitigationPolicy(
    ctypes.c_void_p(-1),  # Current process
    7,  # ProcessControlFlowGuardPolicy
    byref(policy),
    sizeof(policy)
)
print(f"CFG enabled: {bool(policy.Flags & 1)}")

Results:

• Python interpreter directly: CFG enabled: False
• PyInstaller EXE: CFG enabled: True

Minimal Reproduction (Verified)

This PoC has been tested and confirmed to reproduce the issue.

poc_unicorn_cfg.py

#!/usr/bin/env python3
"""
Proof of Concept: Unicorn + CFG crash
When bundled with PyInstaller, this crashes due to CFG + longjmp incompatibility.
Run directly with Python: works
Run as PyInstaller EXE: crashes
"""
import sys

def check_cfg():
    """Check if CFG is enabled for current process"""
    if sys.platform != 'win32':
        return False, "Not Windows"
    try:
        import ctypes
        from ctypes import wintypes, sizeof, byref
        
        class CFG_POLICY(ctypes.Structure):
            _fields_ = [("Flags", wintypes.DWORD)]
        
        policy = CFG_POLICY()
        result = ctypes.windll.kernel32.GetProcessMitigationPolicy(
            ctypes.c_void_p(-1), 7, byref(policy), sizeof(policy)
        )
        if result:
            return bool(policy.Flags & 1), f"Flags=0x{policy.Flags:08X}"
        return False, "Query failed"
    except Exception as e:
        return False, str(e)

def main():
    is_frozen = getattr(sys, 'frozen', False)
    cfg_enabled, cfg_info = check_cfg()
    
    print(f"Frozen (PyInstaller): {is_frozen}")
    print(f"CFG enabled: {cfg_enabled} ({cfg_info})")
    print()
    
    if cfg_enabled:
        print("[!] WARNING: CFG is enabled - Unicorn will likely crash!")
        print("[!] This is a known incompatibility.")
        print()
    
    # Import Unicorn
    from unicorn import Uc, UC_ARCH_X86, UC_MODE_32, UC_HOOK_CODE
    
    # Simple x86 code: INC EAX; INC EAX; HLT
    CODE = b'\x40\x40\xf4'
    ADDRESS = 0x1000
    
    print("Creating Unicorn instance...")
    uc = Uc(UC_ARCH_X86, UC_MODE_32)
    
    print("Mapping memory...")
    uc.mem_map(ADDRESS, 0x1000)
    uc.mem_write(ADDRESS, CODE)
    
    # Add a code hook - this is where the crash happens when returning
    hook_count = [0]
    def hook_code(uc, address, size, user_data):
        hook_count[0] += 1
        print(f"  Hook #{hook_count[0]}: addr=0x{address:x}, size={size}")
    
    print("Adding code hook...")
    uc.hook_add(UC_HOOK_CODE, hook_code)
    
    print("Starting emulation...")
    print("-" * 40)
    
    try:
        # The crash happens when the hook callback returns
        # longjmp is called internally and fails CFG validation
        uc.emu_start(ADDRESS, ADDRESS + len(CODE))
        print("-" * 40)
        print(f"Emulation completed! Hook called {hook_count[0]} times.")
        print("SUCCESS: No CFG crash occurred.")
    except Exception as e:
        print(f"Exception: {e}")
    
    input("\nPress Enter to exit...")

if __name__ == "__main__":
    main()

Build and Test Steps

# 1. Install dependencies (see requirements.txt)
pip install -r requirements.txt
# Or manually:
# pip install unicorn pyinstaller

# 2. Test with Python directly (should work)
python poc_unicorn_cfg.py

# 3. Build with PyInstaller (must collect unicorn binaries)
pyinstaller --onefile --console --collect-all unicorn poc_unicorn_cfg.py

# 4. Run the EXE (will crash after hooks execute)
dist\poc_unicorn_cfg.exe

# 5. Verify exit code
echo $LASTEXITCODE  # PowerShell: shows -1073740791
# Convert to hex: 0xC0000409

Expected Output (Python direct)

Frozen (PyInstaller): False
CFG enabled: False (Flags=0x00000000)

Creating Unicorn instance...
Mapping memory...
Adding code hook...
Starting emulation...
----------------------------------------
  Hook #1: addr=0x1000, size=1
  Hook #2: addr=0x1001, size=1
  Hook #3: addr=0x1002, size=1
----------------------------------------
Emulation completed! Hook called 3 times.
SUCCESS: No CFG crash occurred.

Actual Output (PyInstaller EXE) - VERIFIED

Frozen (PyInstaller): True
CFG enabled: True (Flags=0x00000001)

[!] WARNING: CFG is enabled - Unicorn will likely crash!
[!] This is a known incompatibility.

Creating Unicorn instance...
Mapping memory...
Adding code hook...
Starting emulation...
----------------------------------------
  Hook #1: addr=0x1000, size=1
  Hook #2: addr=0x1001, size=1
  Hook #3: addr=0x1002, size=1
<process crashes - no further output>

Exit code: -1073740791 (0xC0000409)

IMPORTANT: The hooks execute successfully — all three Hook #N lines print before the crash. The crash occurs after each callback function returns, during Unicorn's internal longjmp call that transfers control back to the emulation loop. This is the CFG validation failure in action: the Python callback runs fine, but when Unicorn tries to longjmp back to its emulation code, Windows CFG rejects the jump target as invalid.

Impact Scope

Binding	CFG Behavior	Notes
.NET	Checks CFG at startup, throws exception with helpful message	Best UX
Python	No check — crashes with cryptic exit code	No warning
Other bindings	Unknown — likely no check	Untested

The .NET bindings demonstrate that the Unicorn team is aware of this issue and chose to handle it gracefully. The Python bindings would benefit from similar protection, even if just a warning message.

---

Suggested Solutions

Option 1: Add CFG check to Python bindings (like .NET)

Add a runtime check in bindings/python/unicorn/unicorn.py:

import sys
if sys.platform == 'win32':
    try:
        import ctypes
        from ctypes import wintypes
        
        class CFG_POLICY(ctypes.Structure):
            _fields_ = [("Flags", wintypes.DWORD)]
        
        policy = CFG_POLICY()
        if ctypes.windll.kernel32.GetProcessMitigationPolicy(
            ctypes.c_void_p(-1), 7, ctypes.byref(policy), ctypes.sizeof(policy)
        ):
            if policy.Flags & 1:
                import warnings
                warnings.warn(
                    "Control Flow Guard (CFG) is enabled. Unicorn may crash when "
                    "callbacks return. This commonly occurs in PyInstaller builds. "
                    "Run with Python directly to avoid this issue.",
                    RuntimeWarning
                )
    except Exception:
        pass

Option 2: Register longjmp targets with CFG

Use SetProcessValidCallTargets to register Unicorn's internal longjmp targets as valid CFG targets. This would require:

1. Identifying the longjmp target addresses at runtime
2. Calling SetProcessValidCallTargets before emulation starts

Option 3: Build Unicorn DLL without CFG

Compile unicorn.dll without the /GUARD:CF linker flag. This would require changes to the CMake build system.

Option 4: Request PyInstaller to provide non-CFG bootloader

PyInstaller could provide an option to build without CFG for applications that use Unicorn or similar libraries.

Workarounds (Current)

1. Run Python script directly instead of PyInstaller EXE (verified working)
2. Disable CFG system-wide (not recommended for security reasons)

Untested Alternatives

• Nuitka - may produce executables without CFG (untested)
• cx_Freeze - may produce executables without CFG (untested)
• Building PyInstaller bootloader without CFG - requires custom build

Additional Context

This issue affects any application that:

1. Uses Unicorn emulator with callbacks
2. Is bundled with PyInstaller (or any tool that produces CFG-enabled EXE)
3. Runs on Windows 10/11 with default security settings

Common affected use cases:

• Malware analysis tools (Speakeasy, etc.)
• Binary analysis frameworks
• Emulation-based unpacking tools
• CTF challenge solvers

References

• Unicorn setjmp wrapper: qemu/util/setjmp-wrapper-win32.asm
• Unicorn .NET CFG check: bindings/dotnet/UnicornEngine/WinNativeImport.fs
• Windows CFG documentation: https://learn.microsoft.com/en-us/windows/win32/secbp/control-flow-guard
• PyInstaller bootloader source: https://github.com/pyinstaller/pyinstaller/tree/develop/bootloader

Checklist for Bug Report

• Minimal reproduction case provided
• Environment details documented
• Root cause analysis included
• Evidence from Unicorn source showing awareness of issue
• Suggested solutions provided
• Workarounds documented

---

Cross-Project Collaboration Note

Related PyInstaller Issue: pyinstaller/pyinstaller#9352

This is an ecosystem compatibility issue that spans multiple projects. I've filed companion issues with both Unicorn and PyInstaller because the solution space exists in both codebases, and neither project is solely "at fault."

How the projects interact:

• Unicorn uses a custom setjmp/longjmp implementation that bypasses Windows stack unwinding (necessary for emulation)
• PyInstaller produces executables with CFG enabled (a reasonable security default)
• Windows CFG validates longjmp targets at runtime, which conflicts with Unicorn's approach

Why collaboration helps:

• Unicorn could add a Python binding warning (like .NET bindings already have) — this helps users understand the limitation
• Unicorn could potentially register longjmp targets with CFG via SetProcessValidCallTargets — this would be a proper fix
• PyInstaller could offer a non-CFG bootloader option — this would help all CFG-incompatible libraries, not just Unicorn

Neither fix alone is "the right answer" — ideally both projects make improvements:

1. Short-term: Unicorn warns users, PyInstaller documents the CFG implication
2. Long-term: Unicorn becomes CFG-compatible, OR PyInstaller offers non-CFG builds for specialized use cases

I'm not asking either project to "own" this issue entirely. I'm hoping both teams can acknowledge the interaction and make incremental improvements that benefit the broader ecosystem of security/emulation tools.

---

Note: This bug report was prepared after extensive investigation. The PoC has been tested and the crash verified with exit code 0xC0000409. The existing CFG check in .NET bindings confirms this is a known limitation that should be documented/addressed in other language bindings as well.

[wtdcode]

#1999

Unicorn needs to take over the control flow and jump between .Net runtime and native runtime, which is not compatible with CFG, as far as I could remember.