Should we move external pipelines from KFP-Tekton into the katalog?

[ckadner]

The catalog_upload.json file (MLX bootstrapper) pulls in some external assets.

$ curl -s https://raw.githubusercontent.com/machine-learning-exchange/mlx/main/bootstrapper/catalog_upload.json | \
    grep -E '^  "|"url":' |  grep -v "/katalog/"

  "components": 
      "url": "https://raw.githubusercontent.com/Trusted-AI/AIF360/master/mlops/kubeflow/bias_detector_pytorch/component.yaml"
      "url": "https://raw.githubusercontent.com/Trusted-AI/adversarial-robustness-toolbox/main/utils/mlops/kubeflow/robustness_evaluation_fgsm_pytorch/component.yaml"

  "pipelines": 
      "url": "https://github.com/kubeflow/kfp-tekton/blob/master/sdk/python/tests/compiler/testdata/parallel_join.yaml"
      "url": "https://github.com/kubeflow/kfp-tekton/blob/master/sdk/python/tests/compiler/testdata/sequential.yaml"
      "url": "https://github.com/kubeflow/kfp-tekton/blob/master/sdk/python/tests/compiler/testdata/resourceop_basic.yaml"

The 2 Trusted-AI components we probably want to keep in their place. The KFP-Tekton pipelines however we may want to move them into the katalog repo, especially since they are only meant to be compiler test data in KFP-Tekton.

One of the impacts the current situation, is the script that updates the README cannot get a proper name (and description) for these pipelines and it cannot (re)generate the true listing of katalog assets. So whoever runs the script needs to heed the warning which the scripts prints out and exercise some judgment before committing the changes or risk that the external asset links may have been removed unintentionally.

A better way, albeit more work, might be to have the script read the existing README file, find absolute YAML links which point outside the katalog repo, and re-inject them into the newly generated README. But that introduces more additional complexity.

/cc @animeshsingh
/cc @Tomcli

Originally posted by @ckadner in #69 (comment)