Stem: Make defines contain :suite.rc

[wxtim]

Closes #246

Ensure that a :suite.rc is added to the template language when creating defines.

Check List

• I have read CONTRIBUTING.md and added my name as a Code Contributor.
• Contains logically grouped changes (else tidy your branch by rebase).
• Does not contain off-topic changes (use other PRs for other changes).
• Applied any dependency changes to both setup.cfg (and conda-environment.yml if present).
• Tests are included (or explain why tests are not needed).
• CHANGES.md entry included if this is a change that can affect users
• Cylc-Doc pull request opened if required at cylc/cylc-doc/pull/XXXX.
• This is a bug fix but there are no features on master requiring a minor release?

[oliver-sanders]

Currently the code breaks if [jinja2:suite.rc] or [empy:suite.rc] are specified in the rose-suite.conf file, but works if [template vars] is used instead.

These changes switch this the other way around so that [jinja2:suite.rc] and [empy:suite.rc] work but [template vars] is broken.

[MetRonnie]

LGTM, but haven't tested

[oliver-sanders]

The original error can still be reproduced on this branch:

$ rose stem --source=mysource=$PWD -S QUESTION='"7 x 6"'

$ cat runN/rose-suite.conf && echo -e '\n\n-----------------\n\n' && cat runN/opt/rose-suite-cylc-install.conf
ROSE_STEM_VERSION=1
# Config Options ' (cylc-install)' from CLI appended to options already in `rose-suite.conf`.
opts=(cylc-install)

[jinja2:suite.rc]
ANSWER=42


-----------------


# This file records CLI Options.

!opts=

[env]
# ROSE_ORIG_HOST set by cylc install.
ROSE_ORIG_HOST=<host>

[jinja2]
...
SITE="meto"
SOURCE_MYSOURCE_MIRROR=""
SOURCE_MYSOURCE_REV=""

[jinja2:suite.rc]
QUESTION="7 x 6"
# ROSE_ORIG_HOST set by cylc install.
ROSE_ORIG_HOST=<host>

[wxtim]

@oliver-sanders - I'm terribly sorry that I asked you to review something which was very clearly broken.

[MetRonnie]

Code looks good, still haven't tested

[oliver-sanders]

Tested. Got conflicts. Please squash.

[wxtim]

I think this requires a 5 minute check from @MetRonnie or @oliver-sanders that I haven't messed up the rebase, then merge.

[MetRonnie]

Rebase is fine. I'm unclear on how to test this. I haven't understood the example from the original issue.

[oliver-sanders]

set -eu                                          
                                                 
# create an example workflow                     
cd $(mktemp -d)                                  
mkdir rose-stem                                  
cat > rose-stem/rose-suite.conf <<__HERE__       
ROSE_STEM_VERSION=1                              
                                                 
[jinja2:suite.rc]                                                                                                                  
ANSWER=42            
__HERE__             
touch rose-stem/flow.cylc    
                     
# install it         
rose stem --source=mysource=$PWD --workflow-name=foo    
                     
# inspect the installed config    
echo -e '\n\n# rose-suite.conf'    
cat ~/cylc-run/foo/runN/rose-suite.conf    
echo -e '\n\n# cylc-install opt conf'    
cat ~/cylc-run/foo/runN/opt/*    

Would produce:

# rose-suite.conf
ROSE_STEM_VERSION=1
# Config Options ' (cylc-install)' from CLI appended to options already in `rose-suite.conf`.
opts=(cylc-install)

[jinja2:suite.rc]
ANSWER=42


# cylc-install opt conf
# This file records CLI Options.

!opts=

[env]
# ROSE_ORIG_HOST set by cylc install.
ROSE_ORIG_HOST=vld601.cmpd1.metoffice.gov.uk

[jinja2]
HOST_SOURCE_MYSOURCE="vld601:/var/tmp/tmp.AImdjdV0Zm"
HOST_SOURCE_MYSOURCE_BASE="vld601:/var/tmp/tmp.AImdjdV0Zm"
SITE="meto"
SOURCE_MYSOURCE="/var/tmp/tmp.AImdjdV0Zm"
SOURCE_MYSOURCE_BASE="/var/tmp/tmp.AImdjdV0Zm"
SOURCE_MYSOURCE_MIRROR=""
SOURCE_MYSOURCE_REV=""

[jinja2:suite.rc]
# ROSE_ORIG_HOST set by cylc install.
ROSE_ORIG_HOST=vld601.cmpd1.metoffice.gov.uk

Note, the correct [jinja2:suite.rc] and incorrect [jinja2] sections.

To test, try this with each of:

• template variables
• jinja2:suite.rc
• empy:suite.rc

And ensure that the resulting sections match the input name.

[MetRonnie]

Ok, think I've got it. With this branch, if I have in rose-suite.conf

[jinja2:suite.rc]
ANSWER=42

then in opt/rose-suite-cylc-install.conf I get everything in [jinja2:suite.rc], (unlike on 1.3.x where I get some things in [jinja2]).

And if I have

[template variables]
ANSWER=42

then in opt/rose-suite-cylc-install.conf I get everything in [template variables]

[wxtim]

Any reason not to hit the Big Green Button™