Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Inconsistent piff log message levels #132

Open
parejkoj opened this issue May 13, 2022 · 2 comments
Open

Inconsistent piff log message levels #132

parejkoj opened this issue May 13, 2022 · 2 comments

Comments

@parejkoj
Copy link

Under seemingly normal processing, piff outputs log messages that appear to have incorrect log levels. warning for the iteration numbers and total chi2, and info for statements about ill-conditioned matrices (which we seem to get all the time, so maybe there is a threshold that is set to tight somewhere?). Can you please change those WARNINGs that are really just informative to "INFO", and either make the "ill-conditioned matrix" messages WARNING, or do something to surpress them when they are likely not a real problem?

WARNING: Iteration 1: Fitting 63 stars
INFO: Beginning solution of matrix size (3750, 3750)
INFO: Ill-conditioned matrix (rcond=3.64162e-17): result may not be accurate.
WARNING:              Total chisq = 9645.68 / 38425 dof
WARNING: Iteration 2: Fitting 63 stars
INFO: Beginning solution of matrix size (3750, 3750)
INFO: Ill-conditioned matrix (rcond=7.03794e-17): result may not be accurate.
WARNING:              Total chisq = 9099.78 / 38425 dof
@rmjarvis
Copy link
Owner

FWIW, I think of the different logging levels as levels of verbosity. Not specifically warnings/info/debugging. So you can run Piff with verbosity= 0, 1, 2, or 3.

0 = No logging messages unless there is actually an error.
1 = Not much logging. Just some high level information about how the fit proceeds like the overall chisq/dof, how many outliers were excluded, etc.
2 = More information to help figure out where things might be going wrong if something doesn't look right. Simple configuration or data handling mistakes are usually apparent with this level of verbosity.
3 = A huge dump of debugging information to really dig in to finding a less obvious problem.

So you can think of the "warning" label as "high level information". And "info" as "more information". Really, nothing in there is a warning by normal English usage. But the logging package doesn't have that distinction, so we use their warning label for the former set.

@rmjarvis
Copy link
Owner

rmjarvis commented Sep 26, 2022

@timj pointed me to lsst-utils (https://pypi.org/project/lsst-utils/) which has logger with a verbose level between info and debug. So probably the best solution to this is to use that for our logging, and push info -> verbose and (at least most) warning -> info.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants