Existing META.yml is given too much priority when generating MYMETA.yml and etc, resulting in lost License data

[demerphq]

I was working on a BBC ticket for perl, Perl/perl5#20864 and I noticed that it complained that the LICENSE value from the Makefile.PL was invalid. The value is/was "perl_5", which as far as I can tell is a valid value.

So I started digging into it. Eventually I found my way to ExtUtils::MM_Any::_fix_metadata_before_conversion(). I instrumented the function to dump out its arguments and cluck a stacktrace when it was called, and it seems to get called twice. Once from metafile_target(), and once from write_mymeta().

In the first call the license value is recognized as correct, and things look (to me) to be correct. On the second call however things look much different, actually downright strange, and the license is not recognized and turns into "unknown". The second call seems to be responsible for generating the MYMETA.yaml and MYMETA.json files and both say the license is "unknown". Furthermore they both say this:

  "generated_by" : "ExtUtils::MakeMaker version 6.44, CPAN::Meta::Converter version 2.150010",

note that this is from a perl that was built from blead a day ago, with ExtUtils::MakeMaker 7.66.

Here is the instrumented output:

~/git_tree/Slay-Makefile$ perl Makefile.PL 
$VAR1 = {
          'abstract' => 'Wrapper to Slay::Maker that reads the rules from a file',
          'author' => [
                        'Slaven Rezic <[email protected]>',
                        'Mark Nodine <[email protected]>'
                      ],
          'dynamic_config' => 1,
          'generated_by' => 'ExtUtils::MakeMaker version 7.66',
          'license' => [
                         'perl_5'
                       ],
          'meta-spec' => {
                           'url' => 'https://metacpan.org/pod/CPAN::Meta::Spec',
                           'version' => 2
                         },
          'name' => 'Slay-Makefile',
          'no_index' => {
                          'directory' => [
                                           't',
                                           'inc'
                                         ]
                        },
          'prereqs' => {
                         'build' => {
                                      'requires' => {
                                                      'ExtUtils::MakeMaker' => 0
                                                    }
                                    },
                         'configure' => {
                                          'requires' => {
                                                          'ExtUtils::MakeMaker' => 0
                                                        }
                                        },
                         'runtime' => {
                                        'requires' => {
                                                        'File::Copy::Recursive' => '0',
                                                        'File::Path' => '0',
                                                        'Slay::Maker' => '0.04'
                                                      }
                                      }
                       },
          'release_status' => 'stable',
          'resources' => {
                           'repository' => {
                                             'url' => 'git://github.com/eserte/Slay-Makefile.git'
                                           }
                         },
          'version' => '0.14'
        };
in _fix_metadata_before_conversion at /home/yorton/perl5/perlbrew/perls/latest_blead/lib/5.37.10/ExtUtils/MM_Any.pm line 1189.
	ExtUtils::MM_Any::_fix_metadata_before_conversion(PACK001=HASH(0x5574837f78a0), HASH(0x557483ef1b88)) called at /home/yorton/perl5/perlbrew/perls/latest_blead/lib/5.37.10/ExtUtils/MM_Any.pm line 1142
	ExtUtils::MM_Any::metafile_target(PACK001=HASH(0x5574837f78a0)) called at /home/yorton/perl5/perlbrew/perls/latest_blead/lib/5.37.10/ExtUtils/MakeMaker.pm line 837
	ExtUtils::MakeMaker::new("MM", PACK001=HASH(0x5574837f78a0)) called at /home/yorton/perl5/perlbrew/perls/latest_blead/lib/5.37.10/ExtUtils/MakeMaker.pm line 78
	ExtUtils::MakeMaker::WriteMakefile("NAME", "Slay::Makefile", "AUTHOR", "Slaven Rezic <srezic\@cpan.org>", "VERSION_FROM", "lib/Slay/Makefile.pm", "ABSTRACT", "Wrapper to Slay::Maker that reads the rules from a file", ...) called at Makefile.PL line 20
$VAR1 = [
          'perl_5'
        ];
Generating a Unix-style Makefile
Writing Makefile for Slay::Makefile
$VAR1 = {
          'abstract' => 'Wrapper to Slay::Maker that reads the rules from a file',
          'author' => [
                        'Mark Nodine <[email protected]>'
                      ],
          'dynamic_config' => 0,
          'generated_by' => 'ExtUtils::MakeMaker version 6.44, CPAN::Meta::Converter version 2.150010',
          'license' => [
                         'unknown'
                       ],
          'meta-spec' => {
                           'url' => 'http://search.cpan.org/perldoc?CPAN::Meta::Spec',
                           'version' => 2
                         },
          'name' => 'Slay-Makefile',
          'prereqs' => {
                         'build' => {
                                      'requires' => {
                                                      'ExtUtils::MakeMaker' => 0
                                                    }
                                    },
                         'configure' => {
                                          'requires' => {
                                                          'ExtUtils::MakeMaker' => 0
                                                        }
                                        },
                         'runtime' => {
                                        'requires' => {
                                                        'File::Copy::Recursive' => '0',
                                                        'File::Path' => '0',
                                                        'Slay::Maker' => '0.04'
                                                      }
                                      }
                       },
          'release_status' => 'stable',
          'version' => '0.12'
        };
in _fix_metadata_before_conversion at /home/yorton/perl5/perlbrew/perls/latest_blead/lib/5.37.10/ExtUtils/MM_Any.pm line 1189.
	ExtUtils::MM_Any::_fix_metadata_before_conversion(PACK001=HASH(0x5574837f78a0), HASH(0x5574844dead0)) called at /home/yorton/perl5/perlbrew/perls/latest_blead/lib/5.37.10/ExtUtils/MM_Any.pm line 1703
	ExtUtils::MM_Any::write_mymeta(PACK001=HASH(0x5574837f78a0), HASH(0x5574844dead0)) called at /home/yorton/perl5/perlbrew/perls/latest_blead/lib/5.37.10/ExtUtils/MakeMaker.pm line 1246
	ExtUtils::MakeMaker::flush(PACK001=HASH(0x5574837f78a0)) called at /home/yorton/perl5/perlbrew/perls/latest_blead/lib/5.37.10/ExtUtils/MakeMaker.pm line 79
	ExtUtils::MakeMaker::WriteMakefile("NAME", "Slay::Makefile", "AUTHOR", "Slaven Rezic <srezic\@cpan.org>", "VERSION_FROM", "lib/Slay/Makefile.pm", "ABSTRACT", "Wrapper to Slay::Maker that reads the rules from a file", ...) called at Makefile.PL line 20
$VAR1 = [
          'unknown'
        ];
Invalid LICENSE value 'perl_5' ignored
Writing MYMETA.yml and MYMETA.json

Notice the first output says

'generated_by' => 'ExtUtils::MakeMaker version 7.66',

LICENSE field no longer honored properly.
but the second says

'generated_by' => 'ExtUtils::MakeMaker version 6.44, CPAN::Meta::Converter version 2.150010',

as was found in the MYMETA.yaml files on disk. If I delete the META.yml file from disk before I run Makefile.PL then I no longer get the complaint about an unknown version, and the generated files now look "correct".

It seems to me that we end up keeping way too much of the META.yml on disk than we should, with the result that we end up ignoring things that were passed into WriteMakefile.

I am a bit lost on what to do to fix this, but it sure feels like we give the old META.yml too much priority, and that we produce really inscrutable errors when this happens. Now that i understand things a bit it seems I should just get the META.yml removed from the repo for this module. But frankly this was really confusing to debug and understand. I think we should warn that the META.yaml is really out of date in a situation like this.

FWIW some of my current understanding of this bug came about while I was writing this report. Sorry if it is a bit stream-of-consciousness.

[Grinnz]

I don't entirely follow what's going on but the steps described (generating MYMETA from META) are only relevant on installation, where the metadata is no longer useful.

[Grinnz]

(that is to say, the only relevant fields of MYMETA as opposed to META are the dynamically generated prerequisites if any.)

[Leont]

I don't know where your 'license' => [ 'perl_5' ], comes from but I can't reproduce it. It's 'unknown' all along for me