postgres.CopyToTable: Robustize resources cleanup #3072
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Instead of a generic exception, raise a specific ValueError. The former is considered an anti-pattern because generic exceptions are harder to handle.
Make sure to rollback and close connections as well as to release cursors when an error occurs during CopyToTable.copy(). This can happen, for example, when rows have an invalid format or the method is overriden in a subclass. Tests are added, too (see DailyCopyToTableTest.test_cleanup_on_error()).
LinqLover
commented
Apr 15, 2021
lallea
reviewed
Apr 15, 2021
lallea
reviewed
Apr 15, 2021
|
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. If closed, you may revisit when your time allows and reopen! Thank you for your contributions. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR aims to make connections using the
contrib.postgresmodule more robust. If an error occurs during copying the rows, psycopg2 objects previously could be left opened without a proper clean-up, causing further connection attempts to the database to fail as well (psycopg2.errors.ObjectInUse).By ensuring proper use of try...catch blocks and context managers, issues of this kind should be fixed now. Also, this PR includes regression tests.
For more information about releasing psycopg2 objects, visit: https://www.psycopg.org/docs/usage.html#with-statement
LBNL, a more precise exception class is used in the default
copy()implementation ofCopyToTable(f66534c).Kind of replaces #2934, which I unfortunately never had managed to complete.
Preferably please squash when merging.