Add flip_conjugation method and user info if it would reduce the uvw discrepancy

[bhazelton]

Description

Pulled the conjugation flip code out as a user-callable method. Added info to the uvw check warning message to inform a user when a conjugation flip might reduce uvw discrepancies.

Motivation and Context

fixes #1632

Types of changes

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to change)
• Documentation change (documentation changes only)
• Version change
• Build or continuous integration change
• Other

Checklist:

• I have read the contribution guide.
• My code follows the code style of this project.

New feature checklist:

• I have added or updated the docstrings associated with my feature using the numpy docstring format.
• I have updated the tutorial to highlight my new feature (if appropriate).
• I have added/updated tests to cover my new feature.
• I have updated the CHANGELOG.

[bhazelton]

It turns out this is causing a lot of test errors. Digging in, there are flips happening when the improvement to the uvw matching to the expected value is quite marginal e.g. 44m max diff instead of 50m. This is giving me bad feelings because it feels quite stochastic -- e.g whether or not it wants to flip can depend on which set of baselines is read in on a partial read.

I think we need a different approach here. Options I can think of:

• Add a method to flip the conjugation and allow the user to call it directly. Update the current warning to tell users what the max uvw difference is if they flip the conjugation and point them to the method if they want to call it themselves.
• Add a parameter to set the tolerance to use when checking for flipped conjugation that defaults to 1m.
• Require that the improvement is large (maybe an order of magnitude?). In some ways this feels better -- the improvement has to be substantial to get the flipping to happen. I tested requiring an order of magnitude improvement and that fixed all the test errors. But it still has a bit of a "hardcoded" feel, so I'm not sure I love it vs making users explicitly choose to do the flipping.

Thoughts? @jpober @mkolopanis @adeliegorce

[adeliegorce]

Hi! Thank you for having a look at this. I do agree point #3 implies a hardcoded upper limit which can be sub-optimal. Personally I think option #1 is best - or a variation: the flip is an additional input to the read_ms method such that it can be done serially in a heavy job importing a lot of datafiles.

[jpober]

Ah, yeah, I was worried something like that might happen. I think I agree that presenting the user with an option to flip conjugation is good, perhaps also editing the warning about uvws not matching in the ms reader to suggest that sometimes a conjugation flip can be the issue.

[bhazelton]

Ok, I took a stab at it. It now has the old behavior, but adds some info to the warning message to inform users when a conjugation flip might help and points to the method that can do the flip.

edit: I realized that I didn't actually add it as an option to the read method to do it as part of the read step. I can do that if we want, but the read method already has a lot of options and I'm not sure how often this will come up (since flips already happen if they meet the existing threshold).

Let me know what you think @adeliegorce @jpober

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 99.93%. Comparing base (c3fb3d6) to head (7f11bba).
⚠️ Report is 19 commits behind head on main.

Additional details and impacted files

@@           Coverage Diff           @@
##             main    #1634   +/-   ##
=======================================
  Coverage   99.93%   99.93%           
=======================================
  Files          67       67           
  Lines       22627    22634    +7     
=======================================
+ Hits        22613    22620    +7     
  Misses         14       14           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[adeliegorce]

Thank you for solving my issue!