Update tools

[jeffmcaffer]

It's been a while since the tools being used for harvesting have been updated. ScanCode is up to 3.2.3 (last October) and IIRC ClearlyDefined is on 3.0.x? Not sure about FOSSology. Pretty user that Licensee has newer versions.

ScanCode in particular may have some new data in the results that we can use to get better definitions

• ScanCode
• FOSSology
• Licensee

[nellshamrell]

Discussed at this week's community meeting and this is definitely something we should do!

• Need to update the tools
• Do we need to re-run them on everything? Yes, most likely. It's what we've done when updating these tools in the past.
• How big a difference will it make?
• Talk to the Google and Amazon folks running our crawlers to make them aware when we’re re-running things

[sschuberth]

Do we need to re-run them on everything?

I'd propose a scan-as-you-go approach: Keep scan results from older tool (ScanCode) versions, but schedule harvesting with new tool versions as soon as results are requested.

How big a difference will it make?

As @pombredanne points out, esp. the improvements in ScanCode have been huge. So, please, yes, upgrade ScanCode! 😸