feat: Major architecture enhancement with advanced analysis capabilities

[anhed0nic]

                RE-ARCHITECT: COMPREHENSIVE FEATURE UPDATE

═

This commit introduces a complete overhaul of the RE-Architect framework with advanced reverse engineering capabilities, including symbolic execution unpacking, comprehensive intermediate representation, dynamic analysis with Frida integration, and sophisticated obfuscation optimization.

                        NEW MODULES ADDED

src/unpacking/ Symbolic execution unpacker using Angr
src/ir/ Two-level Intermediate Representation (IR)
src/optimization/ Obfuscation optimization with iterative passes
tests/unit/ Comprehensive unit test coverage
tests/integration/ End-to-end integration test suites
─

─
CORE FUNCTIONALITY ENHANCEMENTS
─
Dynamic Analysis Frida-based function tracing, memory analysis
syscall collection, execution path analysis

Internal IR Decompiler Capstone-based disassembly to custom IR
Ground-level (machine) + Sky-level (C-like)
─┤
Symbolic Unpacking Angr-powered unpacking for UPX, PECompact, etc.
Automatic packer detection and payload extraction
─
Obfuscation Optimization Iterative passes for junk removal, opaque
predicate solving, MBA simplification, VM detection

─
TECHNICAL SPECIFICATIONS
─
Dependencies Added:
frida>=16.0.0 Dynamic instrumentation and analysis
angr>=9.2.0 Symbolic execution and binary analysis
capstone>=5.0.0 Multi-architecture disassembly engine
─
IR Architecture:
Ground Level Machine instruction representation
Sky Level C/C++-like abstract syntax tree
x86/AMD64 Support Complete instruction set including AVX, x87
Verbose Naming Self-documenting operation categories

Analysis Capabilities:
Function Tracing Real-time execution monitoring
Memory Access Analysis Read/write pattern detection
System Call Collection API usage profiling
Execution Path Analysis Control flow coverage analysis

                        PACKER SUPPORT

─
UPX Universal Packer for eXecutables
PECompact PE compression and protection
ASPack aPLib-based compression
FSG Fast Small Good packer
Mew Mew11 packer variant
Themida Advanced protection suite
VMProtect Virtual machine protection
Armadillo Software protection system
Enigma Executable packer and protector

                    OBFUSCATION OPTIMIZATION PASSES

Junk Code Removal Dead block elimination and no-op sequence removal
Opaque Predicate Always-true/false branch resolution
Mixed Boolean Arithmetic MBA expression simplification using claripy
VM Handler Detection Virtualization engine handler identification

─
TEST COVERAGE

Unit Tests (4 new) Individual component testing
Integration Tests (3) End-to-end pipeline validation
Mock Support Comprehensive mocking for external dependencies
Error Handling Graceful degradation and fallback mechanisms

                        PERFORMANCE METRICS

Pipeline Integration Seamless integration with existing analysis flow
Resource Management Proper cleanup and memory management
Timeout Handling Configurable execution limits
Fallback Mechanisms Graceful degradation when dependencies unavailable

BREAKING CHANGES:
BinaryLoader.load() now accepts auto_unpack parameter (default: True)
Pipeline results now include unpacking and optimization metadata
New IR structure requires updates to analysis components

MIGRATION GUIDE:
Update binary loading calls to specify auto_unpack=False if needed
Access new analysis results via results['unpacking'] and results['obfuscation_optimization']
Use InternalIRDecompiler for custom IR analysis

FILES CHANGED: 20 files modified, 9 new modules added
TESTS ADDED: 7 new test files with comprehensive coverage
DEPENDENCIES: 2 new major dependencies (frida, angr)

This update transforms RE-Architect into a comprehensive reverse engineering platform capable of handling modern malware, packed binaries, and obfuscated code with state-of-the-art analysis techniques.

[Copilot]

Pull Request Overview

This PR implements a comprehensive architecture enhancement to RE-Architect with advanced reverse engineering capabilities including symbolic unpacking, dual-level IR decompilation, obfuscation optimization, and Frida-based dynamic analysis.

Key changes:

• Adds symbolic unpacking system using Angr for common packers (UPX, PECompact, ASPack, etc.)
• Introduces two-level IR framework (Ground-Level and Sky-Level) with Internal IR Decompiler
• Implements iterative obfuscation optimization with passes for junk removal, opaque predicates, and MBA simplification

Reviewed Changes

Copilot reviewed 19 out of 20 changed files in this pull request and generated 4 comments.

Show a summary per file

File	Description
tests/unit/test_symbolic_unpacker.py	Unit tests for symbolic unpacker with packer detection and unpacking scenarios
tests/unit/test_obfuscation_optimizer.py	Unit tests for obfuscation optimizer iterations and convergence
tests/unit/test_dynamic_analyzer.py	Unit tests for dynamic analyzer trace/memory/syscall normalization
tests/unit/test_comparison_routes.py	Unit tests for comparison API routes with mock authentication
tests/integration/test_unpacking_pipeline.py	Integration tests for unpacking in the analysis pipeline
tests/integration/test_obfuscation_pipeline.py	Integration tests for obfuscation optimization pipeline
tests/integration/test_dynamic_disabled_pipeline.py	Integration tests for pipeline with dynamic analysis disabled
src/unpacking/symbolic_unpacker.py	Symbolic execution unpacker using Angr for packer detection and unpacking
src/unpacking/init.py	Unpacking module initialization
src/optimization/optimizer.py	Obfuscation optimizer with iterative passes for code cleaning
src/optimization/init.py	Optimization module initialization
src/llm/function_summarizer.py	LLM function summarizer with Internal IR context support
src/ir/ir_core.py	Core IR data structures for ground-level and sky-level representations
src/ir/init.py	IR framework initialization
src/decompilers/internal_ir_decompiler.py	Internal IR decompiler with Capstone-based lifting
src/decompilers/decompiler_factory.py	Updated decompiler factory to include Internal IR decompiler
src/core/pipeline.py	Pipeline enhancements for unpacking and obfuscation optimization
src/core/binary_loader.py	Binary loader with automatic unpacking support
src/analysis/dynamic_analyzer.py	Enhanced dynamic analyzer with Frida integration

Comments suppressed due to low confidence (3)

src/decompilers/internal_ir_decompiler.py:1

• String replacement of %NAMES% with str(targets) will produce invalid JavaScript syntax. The str() of a Python list includes square brackets and single quotes, but JavaScript expects an array literal. Should use json.dumps(targets) instead.

"""

tests/unit/test_symbolic_unpacker.py:1

• The entropy calculation formula is incorrect. Shannon entropy uses logarithms, not bit_length. The correct formula should be entropy -= probability * math.log2(probability). The current implementation will produce mathematically incorrect entropy values.

"""

src/decompilers/internal_ir_decompiler.py:1

• The conditional expression is redundant. If binary_info.path is not a string or Path, str() is still called on it, making the conditional check unnecessary. Simply use target_path = str(binary_info.path).

"""

---

_{Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.}

[Copilot]

The claripy import is missing at the module level but used on line 291. The code checks for claripy.ast.BV but claripy is only imported inside the unpack method on line 178.