[Bug]: combine_crsp_comp_sf() runs out of memory

[FernandoRDLL]

Which script were you running?

main.py (data generation)

Error Category

Memory / OOM error

Bug Description

This function has OOM events.

Steps to Reproduce

Run the function in main alone. It runs out of memory. Not when it's done in the full main.

Expected Behavior

Finish the function and write outputs

Error Output / Stack Trace

[I will assign this issue to myself]

Operating System

Linux in SOM HPC

Python Version

3.13.11

Polars Version

Default version in uv

uv Version

0.10.9

Available RAM

No response

WRDS Authentication Method

Stored credentials (keyring)

WRDS Two-Factor Authentication

• Yes, I approved the 2FA request
• No 2FA prompt appeared
• 2FA failed or timed out
• Not applicable

Additional Context

No response

Pre-submission Checklist

• I have searched existing issues to ensure this bug has not already been reported
• I am using the latest version of the code from the main branch
• I have read the README and followed the setup instructions