Skip to content

Conversation

Avatat
Copy link

@Avatat Avatat commented Aug 24, 2024

Hello everyone!

First and foremost, I would like to thank you for creating this script - it's incredibly helpful in my daily work!

A few months ago, I started having issues with the upmap-remapped.py script (on Ubuntus 16.04 - 20.04, Cephs 14-16).

I quickly discovered that the issue occurred due to the recently added jq filtering. According to the jq manual, it requires a filter, and the basic/empty filter is just a dot .. The fix was trivial: adding a dot . filter to all jq commands resolved the issue, and the script started working again.

A few tries/weeks later, I encountered another issue - Python was complaining that the JSON string wasn't correct and couldn't be decoded. I was surprised because the string had gone through jq, so it should have been correct. This issue occurred some time ago, so I can't provide all the details, but it was related to some special characters or sequence in the JSON string from ceph pg ls remapped that was parsed by Python. I'm certain it was related to the issue described here: https://discuss.python.org/t/help-json-loads-cannot-parse-valid-json/12605
But getting back to the point I fixed the Python string/JSON issue by converting the subprocess output to a "raw string" which was then correctly decoded by the JSON interpreter/decoder.

@Avatat Avatat force-pushed the fix-jq-json-error branch from 9d5ec8d to 10257ce Compare August 24, 2024 14:37
@drakonstein
Copy link
Contributor

drakonstein commented Sep 4, 2024

I was running into these jq issues as well, but instead of investigating what was wrong with the current implementation I converted this tool to use the python rados library. #42
I like your fix as well and added it to my pull request as to avoid merge conflicts.

davidturner-sie added a commit to drakonstein/ceph-scripts that referenced this pull request Sep 4, 2024
@dvanders
Copy link
Collaborator

dvanders commented Sep 4, 2024

Thanks I'll test these changes next week when back from Ceph Day Korea.

@dvanders
Copy link
Collaborator

@Avatat could you please check if you still need this after we merged #42 ?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants