Redistribute intervals fix

[keskitalo]

Currently, integration tests run on just two processes. When they are launched with four processes, the problem size and data distribution changes and new unit test failures come to light.

This PR fixes an error in the interval redistribution, making necessary adjustments to the spt3g import/export facilities.

It also fixes a corner case issue in automatic WCS bounds finding.

[review-notebook-app]

Check out this pull request on 

See visual diffs & provide feedback on Jupyter Notebooks.

---

Powered by ReviewNB

[keskitalo]

@tskisner I cannot complete the unit tests on my system because op_sim_tod_atm fails with Process 1: MPIShared_5029865228910440920 failed to attach buffer of 288 bytes (36 elements of 8 bytes each): [Errno 24] Too many open files: '/MPIShared_5029865228910440920' Any ideas?

[tskisner]

@keskitalo Some systems have a small number of allowed open file handles, and shared memory segments look like a "file" to the operating system. In the unit tests, we are repeatedly creating and destroying these. It may be that there is a leak somewhere that does not cleanly shut those down. My hunch is that if you look at the test which hit that error and try to run it separately (with toast.tests.run("name_of_test_file")), then it would complete.

[tskisner]

I just pushed another change based on our side-channel communication. With this final fix all tests run locally for me on 4 processes. I'll look into implement MPI oversubscription on github CI so that we can improve our test coverage.

[tskisner]

Glad we got these fixes in place!