[Feature] Folder Watcher - Monitor directory or directories and upload sboms

[riteshnoronha]

Based on the discussion DependencyTrack/dependency-track#4256, it would be great to support watching specific folders and upload sboms as they become available.

Things to keep in mind.

• The output adapters should be able to handle duplicate sboms. e.g discard them or make them alternatives as interlynk does.
• sbommv needs to be always running.

[viveksahu26]

@riteshnoronha , this is the design docs before proceeding towards implemented part.

Things to keep in mind

• Duplicate Handling:
  • The output adapters should be able to handle duplicate sboms. e.g discard them or make them alternatives as interlynk does.
• Running in the background as a daemon process
  • sbommv needs to be always running.

Detailed Understanding

• Folder Watching:
  • monitor specific folders (and optionally their subdirectories) for any new or updated SBOM files.
• Automated Uploads:
  • upload new sboms as they detected
• Duplicate Prevention
  • ensure that the same SBOM is not uploaded multiple times

Current sbommv feature

• One-Time Execution
  • Currently, sbommv performs one-time fetching/scanning and uploading of SBOMs from a specified folder.

What's missing in sbommv

• continous running
  • from a one-off run to a continuously running background process (daemon mode).
• Filesystem Watching
  • to watch file system changes continuously so that new or modified SBOMs are detected in real time.
• Duplicate Handling
  • to detect and avoid duplicate SBOM uploads

How Can We Achieve This in sbommv

To implement the Folder Watcher feature, sbommv needs to be extended into a continuously running tool that monitors directories, detects new SBOMs, and uploads them while avoiding duplicates. Below is a comprehensive plan to achieve this.

Proposed solution

1. Daemon Mode Implementation

• Objective: Enable sbommv to run continuously in the background.
• Implementation:
  • should we introduced a new command daemon that starts sbommv in continous mode ?
  • The daemon mode keeps the process running indefinitely, continuously monitoring the filesystem.

2. Filesystem Watching

• Objective: Continuously monitor specified directories for new SBOM files.
• Implementation:
  • Golang’s fsnotify Library: It's a cross-platform filesystem watching library
  • Allow users to specify one or more directories to watch, with optional recursive monitoring of subdirectories.
  • Process new SBOM files as they appear by triggering an upload.

3.Duplicate SBOM Handling

• Objective: Prevent uploading duplicate SBOMs.
• Implementation:
  • Hash-Based Identification:
    • Calculate a hash (e.g., SHA-256) of each SBOM file to uniquely identify it.
  • Output Adapter Logic for DTrack
    • Modify or extend output adapter logic (especially for DTrack) to either discard duplicate SBOMs.

4. CLI Structure & Sub-Commands:

• Current Command

   $ sbommv transfer --input-adapter=folder --in-folder-path="temp" --output-adapter=interlynk --out-interlynk-url="https://api.interlynk.io/lynkapi"

• Proposed Daemon Command

   $ sbommv daemon --input-adapter=folder --in-folder-path="temp" --in-folder-recursive=true --output-adapter=interlynk --out-interlynk-url="https://api.interlynk.io/lynkapi"

• Additional Sub-Commands

  • Like systemd which is a daemond process runs in the background. To check the status of the background process, it has seperate command called status. Similarly command like start and stop are common for daemon process. So do we need to add some similar command for sbommv, like:
    • status, start, stop

[viveksahu26]

After discussion the the proposed daemon command would be like:

 $ sbommv transfer  -d --input-adapter=folder --in-folder-path="temp" --output-adapter=interlynk --out-interlynk-url="https://api.interlynk.io/lynkapi"

Where, -d is to enable daemonD mode for sbommv, i.e running in the background. All input adapters should have their own monitoring function to watch out their respective sboms. For now we have to focus on folders monitoring and watching. And rest of the input adapters to add a NOTE that currently doesn't support it.