Remote component template file could be in a directory

[jrd]

Minimal .gitlab-ci.yml illustrating the issue

include:
  - component: $CI_SERVER_FQDN/snake_coders/tools/ci/python-tests@0.4

But the python-tests component is defined in such a structuer:

$ find templates/security
templates/python-tests
templates/python-tests/template.yml
templates/python-tests/README.md

Then
python-tests/template.yml includes another component:

spec:
  component: [sha]
  …
---
include:
  - component: $CI_SERVER_FQDN/snake_coders/tools/ci/python-base@$[[ component.sha ]]

The bug only appear when component A includes component B.

The component include is transformed into a project/file component (parser-includes.js file, init function) in the following code:

if (!fs.existsSync(localFileName) && !(await Utils.remoteFileExist(cwd, f, ref, domain, projectPath, gitData.remote.schema, gitData.remote.port))) {
  continue;
}
const fileDoc = {
  include: {
    project: projectPath,
    file: f,
    ref: ref,
    inputs: value.inputs || {},
  },
};

[jrd]

It seems that the $[[ component.sh ]] is not resolved.
When constructed the fileDoc, ref is not resolved so the recursive lookup cannot find it.

[jrd]

I made it work with the following corrections:

In parser-includes.js:

• In init, while parsing value["component"]:

const fileDoc = {
  include: {
    project: projectPath,
    file: f,
    ref: ref,
    component: {
      name: componentName,
      sha: ref,
      version: ref,
      reference: ref,
    },
    inputs: value.inputs || {},
  },
};

• Then in init, I add another argument for expandInclude call:

# start of init
const include = this.expandInclude(gitlabData?.include, opts.variables, {});
# value["project"] parsing
fileDoc["include"] = this.expandInclude(fileDoc["include"], opts.variables, value["component"] || {});

• Then I added a component value to expandInclude and added anothe expandWithComporent function:

    static expandInclude(i, variables, component) {
        let include = i || [];
        if (include && include.length == null) {
            include = [i];
        }
        if (typeof include === "string") {
            include = [include];
        }
        for (const [index, entry] of Object.entries(include)) {
            if (typeof entry === "string" && (entry.startsWith("https:") || entry.startsWith("http:"))) {
                include[index] = { "remote": entry };
            }
            else if (typeof entry === "string") {
                include[index] = { "local": entry };
            }
            else {
                include[index] = entry;
            }
        }
        for (const entry of include) {
            for (const [key, value] of Object.entries(entry)) {
                if (Array.isArray(value)) {
                    entry[key] = value.map((v) => Utils.expandText(this.expandWithComponent(v, component), variables));
                }
                else {
                    entry[key] = Utils.expandText(this.expandWithComponent(value, component), variables);
                }
            }
        }
        return include;
    }
    static expandWithComponent(text, component) {
        if (typeof text !== "string") {
            return text;
        }
        return text.replace(/\$\[\[ *component.([a-z]+) *\]\]/g,
        (_match, name) => {
            return component[name] ?? "";
        });
    }