Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

reject files not ending in .root when fetching #31

Open
chosila opened this issue Mar 27, 2023 · 1 comment
Open

reject files not ending in .root when fetching #31

chosila opened this issue Mar 27, 2023 · 1 comment
Assignees
Labels
enhancement New feature or request

Comments

@chosila
Copy link
Contributor

chosila commented Mar 27, 2023

Currently run 2022 seems to have many files that end in .root.dqminfo, which when loaded by datafetcher throws and error such as

WARNING  [DataFetcher : load_data] Problem loading file                                                                                                                                        data_fetcher.py:265
     'root://eoscms.cern.ch//eos/cms/store/group/comm_dqm/DQMGUI_data/Run2022/SingleMuon/0003541xx/DQM_V0001_R000354113__SingleMuon__Run2022A-PromptReco-v1__DQMIO.root.dqminfo', it might                    
     be corrupted. We will just skip this file.                                                                                                                                                               

It does not halt the fetching as the fetcher just skips the file, but it may prevent user from knowing if one of the files really was corrupted and needs to be reported.

Perhaps we can make sure that the file ends in .root before loading the files into fetcher.

@chosila chosila added the enhancement New feature or request label Mar 27, 2023
@chosila chosila self-assigned this Mar 27, 2023
@GluonicPenguin
Copy link
Contributor

I've added a few lines in the data fetcher here to handle these (you might have done sth similar): #34

Specifically these lines:

files = [i for i in files if "PromptReco" not in i]
files = [i for i in files if "pilot" not in i]
files = [i for i in files if "Backfill" not in i]

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants