Protection against complexity

[WilliamsTravis]

I am not sure how best to handle this, but I think we should start considering ways to protect against potential hidden errors due to increasing model complexity.

I just had issue where the resource dataset grid changed with an update and it affected the techmap linking it to the land-use grid. In this case (a NARIS run), reV caught the error. When the new resource grid is larger than the old one and the user forgets to update the project points file, reV will fail at generation because of project points will cause an out-of-bounds indexing error, so no problem there. However, if the new grid is smaller than the old there's a few cases where potential hidden errors could arise:

1. The user remembers to update the project points but forgets to update the techmap:

• No errors are raised and the user might assume that everything worked. In this case, the project points will be valid through generation, but later in the pipeline existing techmaps will point the land use grid to the wrong generation values and they'll have incorrect supply-curve outputs.

2. The user also forgets to update the project points:

• They'll have incorrect generation/econ values as well.

We'll fix this on our side by labeling techmap keys with resource dataset name & versions and making sure we update the project points with new resource versions. However, this issue could have, very easily, been missed. I think it would be obvious for CONUS-scale runs (the mismatching gids should create an obvious pattern), but for smaller areas (like Rhode Island) it might not be so clear. Also, it wouldn't be a problem in the first few runs for a new user unless they inherited reV exclusion files from somewhere else. For long-term users modeling small regions, I can imagine them accidentally reporting faulty results for some time and doing some damage.

The best ways to handle this issue that I've thought of are to:

1. Stop writing the techmap to the exclusion file, remove the techmap_dset parameter from the supply-curve-aggregation methods, and recreate the techmap every run.

• This would be pretty foolproof I think, but would take longer to run.

2. Keep reusing techmaps, but remove the parameter from the aggregation methods and try to generate a future proof key internally.

• My thought is that the key could be inferred from the resource file name and version number. This would have the same runtimes, but assumes the attributes are correct and files aren't overwritten.
• Would remove some flexibility.
• How else could we do this?

3. Keep the user option to set the techmap_dset, but make the default set the key automatically. Also, update the documentation to warn about this issue.

• Assumes users will read documentation and be careful.

So, that's what made me think about this and now I'm wondering how many other potential hiddens errors could exist or be introduced. I think it's important enough to brainstorm solutions/protections, but perhaps this sort of thing is a common/expected issue as model complexity grows and I just haven't come across it yet?

It's not that urgent for my group, but as more people/groups start using reV I think it will become more and more important.