-
Notifications
You must be signed in to change notification settings - Fork 4
Mongo fails ( current dependancies broken ? ) or dumps changed ? #1
Description
when trying to run the mongo tool processing fails with the following output on all 4 targets. I think it is probably a dependency problem or maybe a change in the actual xml files from the dumps. I have spent a few hours trying to pinpoint it but i am not a java script guy so do not really know where to start. Also i am not clear if after the dumps are done if there is some preprocessing that must be done before trying to import into the mongo db. So far on my end; updated npm and node.js to most recent versions installed discogs-data-tools and any dependancies asked for, ran the fetcher to retrieve the latest discogs dumps , lastly try to import into mongo with this command :
discogs-data-tools mongo --latest --connection mongodb://USER:PW@127.0.0.1:27017 --data-dir F:/DataBase/Discogs-Data_flut1_tools/data/ .... with some variations for testing ..the error returned is consistent for all 4 dumps ( same error )
*************** COMMAND OUTPUT *********************
Latest version is "20211201"
- Fetching file listings...(node:1704) [MONGODB DRIVER] Warning: Current Server Discovery and Monitoring engine is deprecated, and will be removed in a future version. To use the new Server Discover and Monitoring engine, pass option { useUnifiedTopology: true } to the MongoClient constructor.
(Usenode --trace-warnings ...to show where the warning was created)
√ Connected to MongoDB server
√ Indexes created on artists
√ Indexes created on labels
√ Indexes created on masters
√ Indexes created on releases
i Processing F:\DataBase\Discogs-Data_flut1_tools\data\20211201\discogs_20211201_artists.xml.gz...
× error during processing
TypeError: Cannot destructure property 'entry' of 'documents[e.index]' as it is undefined.
at processEntries (C:\Users\USER\AppData\Roaming\npm\node_modules\discogs-data-tools\cli\mongo.js:255:15)
at processTicksAndRejections (node:internal/process/task_queues:96:5)
***************** END OUTPUT *****************************
so again i am not sure what is failing here but this would be a handy as heck tool if it was working ...
or if there is some preprocessing or unpacking that needs done the documentation needs updated to reflect that .
I should mention also there seems to be some issues in npm with this tool too here is the output from npm install -g discogs-data-tools
npm WARN deprecated request-promise-native@1.0.9: request-promise-native has been deprecated because it extends the now deprecated request package, see request/request#3142
npm WARN deprecated har-validator@5.1.5: this library is no longer supported
npm WARN deprecated uuid@3.4.0: Please upgrade to version 7 or higher. Older versions may use Math.random() in certain circumstances, which is known to be problematic. See https://v8.dev/blog/math-random for details.
npm WARN deprecated request@2.88.2: request has been deprecated, see request/request#3142
changed 192 packages, and audited 193 packages in 17s
11 packages are looking for funding
run npm fund for details
8 moderate severity vulnerabilities
To address all issues (including breaking changes), run:
npm audit fix --force
Run npm audit for details.