Neo loses content when the plugin gets removed via projectConfig

[hiasl]

Bug Description

@krafttoast and I found out the following:

• When you uninstall Neo locally its fields get migrated to matrix and the content stays.
• When you deploy this removal to production where Neo gets removed via projectConfig, the fields are migrated but the content gets lost

Steps to reproduce

1.) Prepare the initial system

• Take a fresh installation of Craft 5 with Neo 5
• environment should be dev and allowAdminChanges = true
• Create a very simple Neo field with one block only
• Prepare one sample entry with this Neo field having some content
• Create a DB backup and commit a GIT branch "initial"

2.) Manually remove NEO and validate ex-NEO content

• Remove NEO via the console: ddev plugin/uninstall neo
• Check your content: the NEO field was migrated to matrix, the content is still there
• commit your projectConfig to a new GIT branch "noneo"

3.) Remove NEO via projectConfig

• Switch the GIT back to initial and restore the DB backup from above
• Change .env to environment production and make sure allowAdminChanges = false
• When you log into the CP the entry with the sample NEO content should be there, no projectConfig diffs.
• Now switch to the "noneo" branch and do a craft up
• Neo gets uninstalled, the field is migrated to matrix, but the content is lost
• Interestingly also project.yaml gets a new modification date (although allowAdminChanges is false) and some minor changes are made in the field's yaml and project.yaml regarding UID comments.

We tried this on our full project as well as on a local, naked sample installation.

Expected behaviour

The content should survice NEOs removal.

Neo version

5.5.12

Craft CMS version

5.10.12

DB

PostgreSQL: 16

What is the affected Neo field's propagation method?

No response

Does this issue involve templating, and if so, is eager-loading used?

This is not a templating issue

[hiasl]

We just found out, that Craft changed the order how plugins are installed / uninstalled via projectConfig in Craft 5.8 and later, see here: craftcms/cms@876d09f

We believe, that this might be causing the problem, as migrating the content seem to be handled by Neo as a hook after the plugin deinstallation.

A possible solution could be, that Neo provides a console command to create a content migration, that migrates the content to matrix, which could be done before uninstalling Neo.

[hiasl]

@brandonkelly @angrybrad @craftcms this might be interesting for you, since Neo is quite a popular plugin

[ttempleton]

Neo fields can be converted to Matrix through the field settings page - change the field type to Matrix, and you should get a convert button. A console command could be more convenient, though, so will probably do that at some stage.

[krafttoast]

@ttempleton if I remember correctly thats the same conversion that happens if you uninstall the plugin, so it would make pc changes correct? So this cant be done in prod where writing to pc is prohibited, also it would not solve the issue with dataloss when deploying to prod correct?

[hiasl]

@ttempleton
1.) Thanks for the info, I did not know that.
2.) How does the convert button process the change? will it be executed by the webserver? Because we have a HUGE database, it will run for hours (it did when uninstalling Neo in a local environment).
3.) And, as @krafttoast points out, it might have the same problem when deployed via pC

[ttempleton]

Fair point, it would still make project config changes and would probably not have a different result in terms of your data loss. And yeah, it would be executed by the web server, so won't work for you. I'll prioritise adding a console command, and looking into what's causing the data loss.