Adds a copy action

[baywet]

fixes #146
fixes #121
closes #140

Hi everyone,
I've tried to get to the simplest common denominator, and incorporate some of the feedback that was already provided:

• We want the ability to use what's already in the document to update the document.
• We want single purpose actions, that are mutually exclusive, atomic, and can be combined like lego bricks.
• We want our "action keywords" to be verbs.

I'm proposing to add a copy field to the action, that's mutually exclusive with remove or update. It's functionally equivalent to the update field except that it uses a JSON path to source the JSON Node instead of defining it inline.

I know there was some additional thinking about templating expressions, sourcing from env variables etc... But I think those can be considered orthogonal and be added in a separate PR by their champions. I'd like to get the smallest possible unit that brings a meaningful contribution through.

As an image is better than a thousand words, here are a couple of examples.

Copying a node to a node that already exists

overlay: 1.1.0
info:
  title: Merge an existing path item with another one
  version: 1.0.0
actions:
  - target: '$.paths["/bar"]'
    copy:  '$.paths["/foo"]'

Here the existing foo path item properties are being copied to the bar path item.

Copying a node to a node that DOES NOT already exists

overlay: 1.1.0
info:
  title: Merge an existing path item with another one, ensuring the target exists
  version: 1.0.0
actions:
  - target: '$.paths'
    update: { "bar": {} }
  - target: '$.paths["/bar"]'
    copy:  '$.paths["/foo"]'

Moving an item

overlay: 1.1.0
info:
  title: Move an existing path item to another one
  version: 1.0.0
actions:
   # optionally add a remove action to ensure "a clean slate" for the bar path item
  - target: '$.paths'
    update: { "bar": {} }
  - target: '$.paths["/bar"]'
    copy:  '$.paths["/foo"]'
  - target: '$.paths["/foo"]'
    remove: true

A couple of todos if that gets consensus:

• We need to stand up a v1.1-dev branch, and this needs to be rebased
• We need to add an example in the spec
• We need to update the schema
• We need to add some tests
• Editorial information (version, table, etc...) need to be applied

[baywet]

CC @ThomasRooney @lornajane @mikeschinkel

[baywet]

Here is an experimental implementation, it was fairly straightforward :)
BinkyLabs/openapi-overlays-dotnet#105

[ThomasRooney]

I personally really like this. It's simple nomenclature and implementation.

Annecdotally to me the alternative approach of using runtime expressions is more powerful in some ways, but also limiting in others. We've been using them at Speakeasy to represent complex concepts in Arazzo but unfortunately we've had to build escape hatches for places where they don't work (specifically the lack of conditionals). I'd propose we accept this as a minor version bump, and do that as an independent addition to this change as part of future work given a set of differing use-cases.

[mikekistler]

cc: @RobertCraigie @yjp20

Would be great to get your feedback on this one.

[ralfhandl]

What happens if the target does not exist?

[baywet]

In case the target does not exist, same as remove/update, NO-OP
In case the copy does not exist, we could define this as NO-OP, or error? I'm happy either way.

[RobertCraigie]

What's the reasoning for making it a no-op/error instead of adding the node for you? Even if that just works for the direct target, and doesn't create parent nodes. (sorry if this is in the spec, I couldn't find it)

I'm asking as for our versions of "copy" transformations, we create the target node for you. Although admittedly our version is special cased for copying paths and schema objects, so creating the target node is very simple as we don't have to even define what the behaviour would be for recursively creating parent nodes.

I do think that creating the node if it didn't already exists would be pretty helpful as currently you would need to define two different actions with a different path syntax, instead of just one action, or even two actions with the same path.

Although this seems like a fundamental aspect of the Overlays design, so it probably does not make sense to try and figure this out just for copy.

[baywet]

For the target: I'm happy either way, I just want the behavior to be consistent with other kinds of actions. As far as I understand the current spec (v1.0), if the target yields 0 matches, the behavior is currently implementation specific (could be a NO-OP, could be a warning, could be an error). And changing that would probably represent a breaking change from a behavior standpoint. Please correct me if I missed anything.

As for the copy, as written right now, if the cardinality does not match what target found, it's an error. I think that's probably the safest behavior for implementers. There's no point erroring out when copy finds nothing, if at the same time the target found nothing. We might want to rephrase things to we don't need to error when copy finds more items than target.

[ralfhandl]

What if the target selects nothing?

[baywet]

(let's keep the conversation in the other thread until we come to a decision, and I'll revert back here to update the text once we have)

[lornajane]

I'd put this field in between update and remove in the table. All actions need target and description, so it's a bit weird to split them.

[baywet]

updated.

[lornajane]

Perhaps "used in sequence with" rather than "combined with" since the fields are mutually exclusive and can't be combined - so this could be confusing.

[baywet]

updated

[lornajane]

I would like to see a sentence before and after each code snippet explaining what is being illustrated in the snippet, and the outcome of it.

[baywet]

added sections, and resulting descriptions to all examples.

[baywet]

I'll stand up a new PR for the 1.1-dev setup changes soon, and once we merge it, this one should get easier to review and I'll address the other comments. Sorry about the "mess" I iterated my way through the changes to get this PR in an as advanced state as possible.

[baywet]

There, I just pushed #180

[baywet]

from the meeting, compatible is not explicitly defined, but this should be done as a separate PR, let's create a separate issue for that.

[mikeschinkel]

@baywet — #justfyi I am working on an updated overlay.md to illustrate what I mentioned on the call today. I am adding some examples and changing some headings for consistency because with a table of examples the inconsistency stands out more. I also changed some text from passive to active voice because the ghost of my technical writing professor in university is always sitting on my shoulder commenting on my passive voice.

Hope to have this available very soon, though I am not 100% sure how to submit a PR to a PR. Should I submit a PR to baywet/Overlay-Specification/feat/copy-node and then mention that I did so here?

[baywet]

For anybody following along, thanks for bearing with us while we clean up the branches/versions story. I've just "temporary aligned" things before retargeting to main (pending on #183).
This now should be much easier to review for "just the copy stuff", we're down to 3 modified files in the diff view :) (from almost 40 this morning)

[baywet]

@baywet — #justfyi I am working on an updated overlay.md to illustrate what I mentioned on the call today. I am adding some examples and changing some headings for consistency because with a table of examples the inconsistency stands out more. I also changed some text from passive to active voice because the ghost of my technical writing professor in university is always sitting on my shoulder commenting on my passive voice.

Hope to have this available very soon, though I am not 100% sure how to submit a PR to a PR. Should I submit a PR to baywet/Overlay-Specification/feat/copy-node and then mention that I did so here?

@mikeschinkel Thanks for doing this, yet a PR to my feature branch should be fine if it's a large set of changes, I just pushed a few new things since the call FYI. If the changes are smaller, suggestions (comments with suggestion) in this PR can work as well.

[mikeschinkel]

@baywet — I am getting my IDE flagging x-oai-traits: ['paged'] as not valid and as I am not familiar with the syntax I asked ChatGPT and it said this, so you might want to look at it more closely?

The selector $.paths.*.get[[email protected]] for the x-oai-traits example currently defines ['paged'] (array), but the JSONPath selector checks for .paged, implying an object. It might not match as written. You may want to verify whether the intended shape was an object ({ paged: true }) or adjust the JSONPath for array membership.

[mikeschinkel]

@baywet — Here is the PR to your PR: baywet#1

I added a table listing the examples, and the rest of the changes followed from that to improve structure, clarity, and consistency (with no normative changes).

Summary of changeds

• Added a quick-reference table summarizing all examples
• Promoted Examples to a top-level non-normative section
• Added Array Move and Array Replace examples
• Standardized headings and ordering (Update, Copy, Move, Array)
• Adjusted anchors, grammar, and appendix order (File Naming → A, Revision History → B)

All content above ## Examples remains unchanged except to fix broken links.

[ralfhandl]

I am getting my IDE flagging x-oai-traits: ['paged'] as not valid

@mikeschinkel PR #118 contains a fix for that example.