hotfix: memory leaks and long-running crashes

[dskvr]

Description

Critical hotfix addressing memory leaks and long-running stability issues in hyprlax. This PR fixes several severe resource management bugs causing crashes during extended runtime, monitor hotplug scenarios, and GIF layer operations.

Type of Change

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Platform/Compositor/Renderer support
• Breaking change (fix or feature that would cause existing functionality to not work as expected)
• Documentation update
• Performance improvement
• Code refactoring

Related Issue

Fixes crashes during extended runtime and monitor sleep/wake cycles

Changes Made

Phase 1: Critical Memory Leak Fixes

1. Output Info Memory Leak (CRITICAL) - src/platform/wayland.c:251-278

• Fixed registry_global_remove() to properly free output_info_t structures (~80 bytes each)
• Previously leaked on every monitor disconnect, causing OOM conditions over time
• Now cleanly destroys Wayland objects and unlinks from monitor list

2. Frame Callback Cleanup (CRITICAL) - src/core/monitor.c:75-91

• Added cleanup of pending frame callbacks during monitor destruction
• Prevents segfaults when compositor delivers callbacks to freed memory
• Ensures safe monitor hotplug and shutdown operations

3. Layer Image Path Management (HIGH) - src/hyprlax.c:585-592

• Removed double strdup() allocation in sync_ipc_layers()
• Fixed memory leak on every IPC layer modification

Phase 2: Resource Management Improvements

4. EGL Surface Cleanup on Failure (CRITICAL) - src/platform/wayland.c:715-747

• Added comprehensive cleanup when EGL surface creation fails
• Prevents leak of wl_egl_window, layer_surface, and wl_surface objects
• Properly propagates errors and marks monitor as failed

5. Monitor Failed Flag (NEW) - Multiple files

• Added failed flag to monitor_instance_t structure
• Prevents rendering attempts on failed monitors
• Allows graceful degradation when subset of monitors fail initialization
• Updated: src/core/monitor.h:79, src/core/monitor.c:61, src/platform/wayland.c:743,747, src/hyprlax_main.c:1194,1202, src/core/render_core.c:59-62

6. Complete Monitor Destruction (ENHANCED) - src/core/monitor.c:81-131

• Enhanced monitor_instance_destroy() with complete resource cleanup
• Destroys resources in proper reverse order: frame callbacks → EGL surface → EGL window → layer surface → Wayland surface → config
• Added new helper: gles2_destroy_monitor_surface() in src/renderer/gles2.c:1064-1071

7. GIF Texture Cleanup (CRITICAL) - src/hyprlax_main.c:1263-1313

• Fixed GPU VRAM exhaustion from leaking GIF frame textures
• Properly deletes all GIF frame textures (not just current frame)
• Cleans up gif_textures array, gif_delays array, and gifdec handle
• Prevents VRAM leak proportional to (frame_count × texture_size) per GIF removal

Phase 3: Thread Safety & Signal Handling

8. Layer List Thread Safety (HIGH) - Multiple files

• Added layer_mutex to hyprlax_context_t for thread-safe layer operations
• Protected all layer list access with mutex locks in:
  • src/hyprlax_main.c: add, remove, workspace change, update operations
  • src/ipc.c: IPC command handlers (add, remove, modify, list, clear)
  • src/core/event_loop.c: animation checking
• Prevents race conditions between IPC commands and render thread

9. Signal-Safe Shutdown (HIGH) - src/main.c, src/core/event_loop.c

• Replaced non-atomic ctx->running flag with volatile sig_atomic_t g_shutdown_requested
• Ensures proper signal handler behavior per POSIX requirements
• Added multiple shutdown checks throughout event loop
• Changed epoll_wait() from infinite timeout to 5-second watchdog

10. Enhanced Error Handling (IMPROVED)

• Added epoll error handling with graceful shutdown on critical failures
• Detailed error logging with resource state dumps for debugging
• Watchdog timer prevents infinite blocking on epoll errors

Monitoring & Testing Infrastructure

11. Resource Monitoring Tools (NEW)

• Added scripts/monitor_resources.sh: Real-time resource usage tracking
• Added scripts/analyze_resources.py: Statistical analysis of resource logs
• Added tests/stress/test_overnight.sh: 24-hour stress test with hotplug simulation
• Added tests/test_resource_monitor.c: Unit tests for resource monitoring
• Added resource monitoring core modules: src/core/resource_monitor.c, src/core/time_utils.c

12. Documentation (NEW)

• MEMORY_LEAK_FIXES.md: Detailed analysis of Phase 1 critical fixes
• RESOURCE_MANAGEMENT_FIXES.md: Comprehensive Phase 2 resource management documentation
• Updated .gitignore to exclude test artifacts and resource logs

Testing

• All modified files compile successfully with no new errors
• Memory leak fixes verified through code analysis
• Thread safety analysis completed
• Signal handling verified for POSIX compliance
• Resource cleanup ordering verified
• make test passes all tests (requires runtime environment)
• make memcheck shows no memory leaks (requires Valgrind + runtime)
• Overnight stress test with monitor hotplug (requires hardware)
• IPC layer stress test (requires runtime)

Recommended Manual Testing

1. Valgrind Test: valgrind --leak-check=full --track-origins=yes ./hyprlax
   • Expected: Zero "definitely lost" bytes
2. Monitor Hotplug: Connect/disconnect external display 10+ times
   • Expected: Stable memory usage (no RSS growth)
3. IPC Layer Test: Repeatedly add/remove/modify layers 100+ times
   • Expected: No memory growth
4. Long Runtime: Run for 24+ hours with workspace switching
   • Expected: Stable memory and CPU usage
5. GIF Stress Test: Load/unload animated GIFs repeatedly
   • Expected: No VRAM exhaustion

Checklist

Code Quality

• My code follows the style and conventions of this project
• I have performed a self-review of my code
• I have commented my code, particularly in hard-to-understand areas
• My changes generate no new warnings

Testing

• New overnight stress test added (tests/stress/test_overnight.sh)
• Resource monitoring unit test added (tests/test_resource_monitor.c)
• Resource monitoring tools added for validation
• Tested on local Hyprland setup (requires hardware setup)

Documentation

• I have updated the documentation accordingly (MEMORY_LEAK_FIXES.md, RESOURCE_MANAGEMENT_FIXES.md)
• I have updated the README if needed (not applicable)
• I have updated CHANGELOG.md (needs to be done)

Focus

• This PR focuses on a single change/feature/fix (memory leaks and stability)
• No unrelated changes are included
• All changes are necessary for the stated purpose

Impact Analysis

Memory Savings

• Per monitor hotplug: ~80 bytes (output_info_t leak fixed)
• Per IPC layer modification: ~50-100 bytes (path string leak fixed)
• Per GIF layer removal: ~(frame_count × 4MB) VRAM (GPU texture leak fixed)
• Over 1000 hotplug cycles: ~80 KB saved
• Critical: Prevents dangling pointer crashes and OOM conditions

Stability Improvements

• ✅ No more segfaults on monitor disconnect
• ✅ Safe extended runtime (days/weeks)
• ✅ Reliable monitor hotplug support
• ✅ Clean IPC layer management
• ✅ Thread-safe layer operations
• ✅ POSIX-compliant signal handling
• ✅ GPU VRAM leak prevention
• ✅ Graceful degradation on monitor init failure

Files Modified (16 files, +787 lines, -31 lines)

Core Changes:

• src/core/event_loop.c - Signal handling, thread safety, epoll error handling
• src/core/monitor.c, src/core/monitor.h - Frame callback cleanup, failed flag, complete destruction
• src/hyprlax_main.c - Layer mutex, GIF cleanup, thread-safe operations
• src/main.c - Signal-safe shutdown flag
• src/platform/wayland.c - Output info cleanup, EGL surface error handling
• src/renderer/gles2.c - Monitor surface destroy helper
• src/include/renderer.h, src/include/hyprlax.h - Function declarations, context updates
• src/ipc.c - Thread-safe IPC handlers

New Files:

• src/core/resource_monitor.c, src/include/resource_monitor.h - Resource monitoring system
• src/core/time_utils.c, src/include/time_utils.h - Time utility functions
• scripts/monitor_resources.sh - Resource monitoring script
• scripts/analyze_resources.py - Resource analysis tool
• tests/stress/test_overnight.sh - Overnight stress test
• tests/test_resource_monitor.c - Resource monitor unit tests

Build & Config:

• Makefile - Added new source files and test targets
• .gitignore - Exclude test artifacts

Additional Notes

This is a critical stability hotfix that addresses multiple severe bugs discovered during extended testing. The fixes are surgical and focused on resource management without changing functional behavior. All changes maintain backward compatibility and follow existing code patterns.

The addition of resource monitoring tools and stress tests ensures these issues can be detected early in future development.

Priority: CRITICAL - Fixes production crashes and memory leaks
Risk: LOW - Surgical fixes to resource cleanup, no functional changes
Testing: MEDIUM - Requires extended runtime testing to fully validate

[Copilot]

Pull Request Overview

This PR implements comprehensive resource monitoring and leak detection for hyprlax, addressing stability issues in long-running sessions. The changes include a built-in resource monitor, improved GPU synchronization for hyprlock compatibility, proper cleanup of GIF textures and Wayland resources, thread-safe layer operations, and signal-safe shutdown handling.

• Added resource monitoring subsystem with FD/memory leak detection and IPC integration
• Fixed GPU sync blocking with hyprlock by replacing glFinish() with glFenceSync()
• Implemented proper cleanup for GIF textures, Wayland surfaces, and monitor resources
• Added thread-safe layer list access with mutex protection across all operations
• Improved error handling for Wayland display errors and epoll failures

Reviewed Changes

Copilot reviewed 31 out of 32 changed files in this pull request and generated 4 comments.

Show a summary per file

File	Description
src/core/resource_monitor.c	New resource monitoring implementation tracking FDs, memory, and GPU resources
src/include/resource_monitor.h	Resource monitor API definitions and data structures
tests/test_resource_monitor.c	Comprehensive unit tests for resource monitoring functionality
src/renderer/gles2.c	GPU sync fix using glFenceSync instead of glFinish for hyprlock compatibility
src/platform/wayland.c	Enhanced error handling and resource cleanup for Wayland connections
src/hyprlax_main.c	GIF texture cleanup fix and layer mutex initialization
src/core/event_loop.c	Signal-safe shutdown and resource monitoring integration
src/ipc.c	Added resource_status command and timeout handling
src/core/time_utils.c	64-bit timestamp utilities to prevent overflow
scripts/monitor_resources.sh	Resource monitoring script for production use
scripts/analyze_resources.py	Leak detection analysis tool
docs/resource_monitoring.md	Complete documentation for resource monitoring system

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.