Make execution more debugable / understandable

[JoshuaBehrens]

I have a bitbucket pipeline, that wants to host Shopware in the docker-in-docker from a docker-compose, in there the shopware/docker-base is used. In there is an entrypoint script (that one needs to lookup). This script runs the deployment-helper. The deployment-helper downloads shopware-cli, shopware-cli runs PHP commands like composer install and bin/console plugin:install. Now the binaries starting other binaries look like this:

• docker compose
  • docker
    • bash
      • php deployment-helper
        • shopware-cli project ci
          • php composer install
            • composer run post-install-cmd
              • php bin/console asset:install ?
          • php bin/console asset:install ?

Now I am just looking at this:

Container init-1  service "init" didn't complete successfully: exit 255

I have to extrapolate/guesstimate the execution hierarchy/stacktrace from own research or assumptions. This is not easy to grasp.

I am unsure about the last two as I am not sure which command output belongs to which command. Or I do not even know which command is tried to be executed. Is it from an entrypoint? Is it from shopware-cli? Is it the deployment-helper? Which process is now trying a command, that maybe did not yet have any identifying output?

Docker at least said it is the init service. But after that? I have no idea. We need something to do about this script nesting. Make can be more verbose. composer scripts can be more verbose. bash, make and composer can say what command they would execute next. How can I achieve the same here?

Why did we have to write these tools, that also just delete directories and execute other commands, in yet another "management tool"? With a debatable configuration language, that just has autocompletion because on some server in the internet is a definition file. Nothing else is guiding. I could run all these tools with more verbosity and can better understand the script. What can this tool do?

I am sore about the yaml rant. I am not sorry about the "why not use make, bash or composer scripts rant"

[shyim]

shopware-cli and deployment helper has nothing together.

shopware-cli will build the project, and the deployment helper runs the required commands during deployment.
shopware-cli = on ci/cd/image build
deployment-helper = during deployment on the host machine

They don't require also each other.

Deployment Helper is already very verbose. It prints everything that it executes; I see no problem there. shopware-cli could be improved. I am there with you.

The docker-compose example in the README is just an example, and if you get errors like:

Container init-1  service "init" didn't complete successfully: exit 255

you can just do docker compose logs init and see the reason, I don't see any value wrapping stuff, or adding anything helper around. When you don't know what to do there, you maybe should learn first docker and docker compose. The container is for more advanced users who do the hosting on their own or use platforms like AWS/GCP/Azure to run their containers. So we can expect some knowledge here to be honest.

I believe you didn't understand how the projects can work together and like to blame :)

[JoshuaBehrens]

At first I wanted to react to your take on my docker knowledge. I do not see myself as an expert but longtime user and I had my struggles, that I was able to overcome with docker and especially in combination with Bitbucket docker-in-docker limitations, and AWS fargate. Glad I had support there by experts :D Tools like k8s still feel utterly complex for "something one just tries out" and are out of my knowledge.

I had a look at the docker logs and only saw a successful output of a plugin installation:

init-1  | Shopware Plugin Lifecycle Service
init-1  | =================================
init-1  | 
init-1  |  Install 1 plugin(s):
init-1  |  * Shopware Publisher (v3.2.1)
init-1  | 
init-1  |  Plugin "SwagPublisher" has been installed and activated successfully.
init-1  | 
init-1  |  [OK] Installed 1 plugin(s).                                                    
init-1  | 
init-1  |  ! [NOTE] You may want to clear the cache after activating plugin(s). To do so  
init-1  |  !        run the cache:clear command                                           
init-1  | 
init-1  | =================================================
init-1  | End: php bin/console -n plugin:install SwagPublisher --activate
init-1  | > Time: 7254.27ms
init-1  | =================================================
init-1  | 
init-1  | 

So I assumed there is a script running, that I cannot properly follow as I do not know where it is started, why and what would come next. By occam's razor I assumed, that something, that I do not know yet is likely the source of an error. So I am now here as I assumed the deployment helper to be an issue as it likely was the last command orchestrating commands and one new thing in my project I fiddle with.

Eventually you were right about me not knowing enough about the docker commands I tried out but at a completely different part. So here are some more insights:

The issue seems indeed to be some missing information on how I used docker compose up --detach --wait

I run this in CD

docker compose -f=some-path/docker-compose.yml up --build --no-start
docker compose -f=some-path/docker-compose.yml up --detach --wait shopware || true
docker compose -f=some-path/docker-compose.yml logs init

as I expected to have to name the service I have to wait for but also have a detached start-up. (The || true is just for log extraction).

This is the command I run to see the logs

docker compose -f=some-path/docker-compose.yml up --build --no-start
docker compose -f=some-path/docker-compose.yml up

But this seems to be different execution style. Although docker output says it starts mysql:

[+] Running 6/6
 ✔ Network default         Created
 ✔ Container init-perm-1   Exited
 ✔ Container opensearch-1  Started
 ✔ Container mysql-1       Started
 ✘ Container init-1        service "init" didn't complete successfully: exit 255
 ✔ Container shopware-1    Created

I seem to get an issue with the database connection and stops there with a 255. So docker was behaving unexpected to me.

Eventually I likely solved it with no further argument after the --wait option.

docker compose -f=.dev-ops/e2e/cypress-env/docker-compose.yml up --build --no-start
if ! docker compose -f=.dev-ops/e2e/cypress-env/docker-compose.yml up --detach --wait; then
    docker compose -f=.dev-ops/e2e/cypress-env/docker-compose.yml logs
    exit 1
fi

---

They don't require also each other

I once heard, that deployment helper installs shopware-cli but looking at the source code I see no reference to shopware-cli. You are right. But this feels weird to have documentation on shopware-cli, that says is can be configured in .shopware-project.yml, and deployment helper uses the same file? This might lead to the confusion, that these tools belong together as well. Was it intended to have one configuration file for multiple tools? Other tools in this ecosystem seem to just have one configuration each.

I believe you didn't understand how the projects can work together

Apparently this is an issue. I did not understand why I see plugin:install output. Which tool is doing what part. So this initial criticism still stands. "I had to extrapolite what happens because the logs are not enough". Maybe we need an overall documentation how all the different tools you expect users to use in combination. Or I did not stumble upon it.

[shyim]

It's intended to configure everything in one file. So, we don't have a clustering of multiple config files.

I think we can add more docs, and maybe prefix the output of deployment helper, with [deployment-helper]: