New Error Code for Product Data Requests where the Data Holder does not hold the required Product Reference Data

[NeilSansomeMBL]

Description

Currently, there are four endpoints for Product Reference Data defined in the CDR Standards which Data Holders are obligated to support:

• For publicly offered Banking Products: GET Product & GET Product Details
• For Energy Plans (subject to Clause 4.2, Schedule 4 CDR Rules): GET Generic Plan and GET Generic Plan Detail.

However, there is no defined error response under the CDR Standards – Error Codes for scenarios where a Data Holder does not hold the Product Reference Data being requested. This includes scenarios such as:

• where a Data Holder is not required to disclose Product Reference Data for the product because the product is not publicly offered (e.g. a white label book in run-down), and
• where a Data Holder is not required to disclose Product Reference Data for a product because the Data Holder is in a separate industry (e.g. Banks being requested to provide Generic Plan information).

In these scenarios, a Data Holder may still be required to:

• build the appropriate parameter validation and error handling for each Product Reference Data endpoint, and
• present a ‘HTTP 200 success’ response with an empty payload in accordance with each endpoint’s response definition to convey to the requestor that there are no relevant products/data available.

Intention and Value of Change

It is unclear whether the above approach would achieve a better consumer outcome than an appropriately labelled ‘404 error’ response that similarly conveys to the requestor that there are no relevant products/data available, and it may be costly and complex to build (and maintain) the required error handling for each Product Reference Data endpoint without significant consumer benefit.

As a result, we propose to introduce the below new error response to:

• better inform the requestor that there are no relevant products available (and that Product Reference Data is not available because it is not required), and
• remove the need for Data Holders to maintain more costly and complex error handling for each Product Reference Data endpoint.

Area Affected

High Level Standards - Error Codes
Banking APIs - GET Products
Energy APIs - GET Generic Plans

Change Proposed

A new response is added to each Product Reference Data Endpoint:

Status	Meaning	Description
404	Not Found	Resource not required

With a new 404 (Not Found) Error Code added under the Consumer Data Standards, Error Codes, such as:

Error Title	Error Code	Description
Resource Not Required	urn:au-cds:error:cds-all:Resource/NotRequired	The requested resource URL is a valid API endpoint defined by the Consumer Data Standards, but it is not required to be supported as the Data Holder does not hold the required data.

[markskript]

Note that we have been speaking to another DH who is currently returning 404 and they agree that this is incorrect and will be updating their implementation to match the current standards by returning a HTTP 200 and an empty JSON array.

As such, I do not believe a change in standards is needed here as there appears to be a solution accepted by the majority of DHs already.

[nils-work]

This issue was discussed in the MI call today. Participants noted:

• A preference to avoid receiving error responses from these endpoints
• Consistency across Data Holders is preferable
• A consistent pattern across endpoints is preferable
• Data Holders have a requirement to provide a product data request service which implies it should provide a 'success' response even to indicate that there are no products being offered
• That future potential changes to the Register may provide alternatives relating to Product Reference endpoints in general. e.g.: #561 - Update Register API to return separate PRD and status/outage endpoint.

[perlboy]

Can further information on use case be provided here please?

From my current understanding of the use case, providing a 404 response specifically to "known but no data available" requests implies that, in fact, there is data available and if this is the case and the product is in market it's mandated anyway.

Going per sector:

• Banking: Get Banking Products is expected to never 404 for anyone in the Register, although voluntary data holders can do whatever they want, and as @markskript indicated an empty array is not only expected but is the result. Get Product Detail should never result in a 404 because the productId is sourced from Get Banking Products, if it's listed but then not available the Holder is non-compliant. Any recipient arbitrarily calling product identifiers not listed in Get Banking Products deserves the undocumented behaviour.
• Energy: Energy endpoints are not required to be delivered by Holders at any time. Forcing a Holder to implement these endpoints solely for the purpose of returning 404s is not only ridiculous it is of dubious value relative to cost. To be clear there is no Rules instrument that requires these endpoints to be implemented in any capacity, even Status/Outages has dubious Rules enablement although arguably that's for operational reasons.

As @nils-work has highlighted, this request seems to be working around a Register design flaw that's been highlighted for years. Fix the source of the problem rather than hack in workarounds.