Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adds Windows support #53

Open
wants to merge 3 commits into
base: master
Choose a base branch
from
Open

Conversation

ramkrishna2910
Copy link

Windows Support for Human-Eval

This PR adds comprehensive Windows support to the code evaluation pipeline. Here are the key changes:

Process Management

  • Replaced Unix-specific Process handling with Windows-compatible Pool implementation
  • Added proper process cleanup using pool.terminate() and pool.join() to prevent zombie processes on timeout
  • Switched to 'spawn' context instead of 'fork' for Windows compatibility: multiprocessing.get_context('spawn')

Timeout Implementation

  • Added Windows-specific timeout mechanism using threading since SIGALRM is not available on Windows
  • Implemented thread-based timeout using PyThreadState_SetAsyncExc to inject exceptions into running threads
  • Added proper thread state cleanup to prevent memory leaks and ensure thread state consistency
  • Introduced done and raised events to handle race conditions in timeout scenarios

Memory Management

  • Added Windows-specific memory limits using Windows Job Objects
  • Integrated pywin32 package for Job Object API access
  • Added conditional dependency: pywin32>=306; platform_system == "Windows"
  • Implemented memory limits through JOB_OBJECT_LIMIT_PROCESS_MEMORY flag

Platform Detection

  • Added IS_WINDOWS flag using platform.system() for conditional code paths
  • Separated Unix and Windows-specific implementations into distinct functions
  • Made all path handling Windows-compatible

These changes ensure that the human-eval framework runs reliably on Windows while maintaining feature parity with Unix systems. The implementation handles Windows-specific quirks around process management, timeouts, and memory limits while preserving the security and isolation guarantees of the original implementation.

Following is the sanity check using the example results done on Windows:

image

Closes #51 #45
CC #50

@ramkrishna2910 ramkrishna2910 changed the title Add Windows support Adds Windows support Dec 27, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

I have the windows problem
1 participant