Support for conditional requests/Strategy for not missing updates?

[cchndl]

Hi!

I was wondering how to make sure in SciCat that updates don't get lost when multiple clients access the same resource?
What I wanted to do uses the origdatablocks endpoint, but probably extends to the other PATCH endpoints.

I noticed the Etag header was set, and I tried supplying If-Match and If-Unmodified-Since, but as far as I now understand after having a look, the default behaviour in the etag library is to only hash the response value for caching, and you have to implement the logic for these headers yourself (or I'm doing something wrong).

Is there another way that I have missed? The search didn't turn up issues/pr's.

Best
cchndl

Here is a python test script, using the instruments endpoint since it's more compact than origdatablocks/datasets:

import requests
# setup
url = ""
instrument = {"uniqueName": "test-instrument", "name": "TBD"}
credentials = {"username": "", "password": ""}

r = requests.post(f"{url}/api/v3/auth/login", json=credentials)

headers = {"Authorization": "Bearer " + r.json().get("access_token")}

r = requests.post(f"{url}/api/v3/instruments/", headers=headers, json=instrument)
instr_id = r.json()["pid"]

# get the initial instrument state
r = requests.get(f"{url}/api/v3/instruments/{instr_id}", headers=headers)

not_modified_headers = headers.copy()
not_modified_headers["If-Match"] = r.headers.get("etag")
not_modified_headers["If-Unmodified-Since"] = r.headers.get("date")

intermediate_update = {"name": "intermediate"}
conditional_update = {"name": "conditional"}

# do an unconditional update
requests.patch(
    f"{url}/api/v3/instruments/{instr_id}",
    headers=headers,
    json=intermediate_update,
)
# this should only happen if the resource has not changed since the GET request, no?
r = requests.patch(
    f"{url}/api/v3/instruments/{instr_id}",
    headers=not_modified_headers,
    json=conditional_update,
)

r = requests.get(f"{url}/api/v3/instruments/{instr_id}", headers=headers)
print(r.json().get("name"))  # will print "conditional"

[sbliven]

Would using the recently-added application/merge-patch+json mime type with PATCH help with this? See #1891.

[cchndl]

Thanks for the suggestion, but the problem would still persist. I can use it to make the updates nicer, but the issue here is between multiple write requests.

If I have two PATCH's from state A: A->B and A->C, the changes from the first patch would get overridden by the second one, like shown above. The merge-patch+json would help if they touch different keys (but this is already the case with the json mime-type if you only specify the changing fields), but if its the same field, it will still be overridden.

The RFC for merge-patch also says explicitly that it replaces arrays in full. For the problem that I have, the list of Files in the OrigDatablock, we would need a merging instead of this replacing.

Between requests, the conditional headers that I linked above are the "http-way" that I know.