Thanks for sharing that framework I think it has a really unique edge and potential to serve as a basis of a universal standard for python data ops.
I am using it to structure government data for one of my projects and once the data starts to grow the need for parallel/distributed computation becomes clearly evident.
I was planning to extend it to support thread pool/ ray distributed runners but I figure you may have already developed something similar given that in the local runner I already see facilities accommodating that functionality such as exceptions on remote calls and concurrency support.
If you are actively developing this project still do you have any plans to release such runners and if not do you have any general guidance of how such can be built?
Thanks for sharing that framework I think it has a really unique edge and potential to serve as a basis of a universal standard for python data ops.
I am using it to structure government data for one of my projects and once the data starts to grow the need for parallel/distributed computation becomes clearly evident.
I was planning to extend it to support thread pool/ ray distributed runners but I figure you may have already developed something similar given that in the local runner I already see facilities accommodating that functionality such as exceptions on remote calls and concurrency support.
If you are actively developing this project still do you have any plans to release such runners and if not do you have any general guidance of how such can be built?