Unified Interface: Absolute Tolerance

[PaulJonasJost]

Add tolerance to the unified interface. Some optimizers not supported, especially Dlib with epsilon, though epsilon is not strictly a tolerance!

[codecov-commenter]

⚠️ Please install the to ensure uploads and comments are reliably processed by Codecov.

Codecov Report

❌ Patch coverage is 88.23529% with 4 lines in your changes missing coverage. Please review.
✅ Project coverage is 84.35%. Comparing base (bc7d89a) to head (e5ab895).

Files with missing lines	Patch %	Lines
pypesto/optimize/optimizer.py	88.23%	4 Missing ⚠️
❗ Your organization needs to install the Codecov GitHub app to enable full functionality.

Additional details and impacted files

@@             Coverage Diff             @@
##           develop    #1660      +/-   ##
===========================================
- Coverage    84.37%   84.35%   -0.02%     
===========================================
  Files          164      164              
  Lines        14320    14354      +34     
===========================================
+ Hits         12082    12109      +27     
- Misses        2238     2245       +7     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[PaulJonasJost]

closes #1646

[dweindl]

Thanks, @PaulJonasJost. Good feature to add, but I think this requires a few changes:

It's currently unclear

• whether it's an absolute or relative tolerance. (included in the docstring, but not in the method name. we might want to support both in the future, so the method name should be unambiguous.)
• what the tolerance applies to. many gradient-based optimizers will support tolerances on the objective as well as its gradient, so it needs to become clear which one is specified here.

[dweindl]

From what I remember, ipopt has quite complex termination criteria. While various tolerances are supported, I think just hitting this single value is insufficient for termination, so it might be a bit confusing. Not completely sure whether it should be added here or not.