The diff server has a very complicated mechanism around managing a pool of worker processes to do diffs that could be cleaned up and made easier to understand and manage by separating it out into its own class and module:
|
# TODO: we should split out all the management of the executor and diffing |
|
# (so this, get_diff_executor, caller, etc.) into a separate object owned |
|
# by the server so we don't need weird bits checking the server's |
|
# `terminating` state and so that all the parts are grouped together. |
|
async def diff(self, func, a, b, params, tries=2): |
It’s also gotten more complicated since #210 landed with the new MAX_DIFFS_PER_WORKER environment variable.
This should generally encapsulate the logic in server.server.DiffHandler.diff() and the other methods it depends on (e.g. get_diff_executor()). You should be able to set up a diff pool with a set number of workers, a max number of diffs per worker, etc. and then run async tasks in the pool.
This is a sub-issue of #28.
The diff server has a very complicated mechanism around managing a pool of worker processes to do diffs that could be cleaned up and made easier to understand and manage by separating it out into its own class and module:
web-monitoring-diff/web_monitoring_diff/server/server.py
Lines 478 to 482 in 8e0c499
It’s also gotten more complicated since #210 landed with the new
MAX_DIFFS_PER_WORKERenvironment variable.This should generally encapsulate the logic in
server.server.DiffHandler.diff()and the other methods it depends on (e.g.get_diff_executor()). You should be able to set up a diff pool with a set number of workers, a max number of diffs per worker, etc. and then run async tasks in the pool.This is a sub-issue of #28.