Difficult management of resources

[OgnjenMilicevic]

I have a problem running a large number of CircDBG instances in parallel on a cluster because I cannot control resource usage. Config parameter ThreadsNum doesn't seem to work for me, as I use your examplefrom README with 4 threads, and some processes easily take 25 cores on the cluster. Memory sometimes goes above 65GB with no increase in CPU usage. As a result, the cluster is brought into heavy swapping and remains unresponsive for hours.

Am I doing something wrong? Is there a way to predict resource usage so I can pan execution with SLURM?
If I were to put a hard limit on the CircDBG container such as 40GB, what would happen to that process?

Thanks!

[MarjanGolshani]

Hi dear
I have the same problem, can you say how you solved it?

[OgnjenMilicevic]

Hi! I implemented cgroups resource limitations. I didn't find any solutions in the program.

[lxwgcool]

I will check it out this weekend. Sorry for the delay. Best Xin

…

On Sat, Feb 27, 2021 at 5:57 AM OgnjenMilicevic ***@***.***> wrote: Hi! I implemented cgroups resource limitations. I didn't find any solutions in the program. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#3 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ACUKW3J5CTFBZJUFXR7HTF3TBDF2HANCNFSM4UH2QAGQ> .

[MarjanGolshani]

thanks for your reply. I think that problem comes from this part, but when I change it I get another error: cannot find the file"./Detection_Result/Brief_sum.txt" but had created that file.