Question - Threshold limits - Data driven or Expert driven

[Ak784]

Hello Everyone,

We see most of our DQD checks for lab threshold failed. While I understand the threshold values are customizable, can I know whether the threshold values that are configured are based on expert inputs or any data-driven approach (meaning gathering statistics about certain measurements from multiple sites and choosing the most common one, etc).

I ask this because, if we have to adapt the DQD threshold checks based on our region characteristics, we would like to follow a similar procedure.

So, few questions

a) DQD rule checks are expert-driven or data-driven? Is there any way I can read about how certain rules and thresholds were configured?

b) If it's expert-driven, Is it possible to know how it was done? Like 10-20 experts sat down and arrived at a threshold value for these measurements?

c) I understand there is an overlap between Achilles Heel (less DQ rules) and DQD (more DQ rules). Is there any reason as to why we have a new tool called DQD instead of extending the existing Achilles Heel? Am trying to understand the background info, so I can suggest an appropriate tool to our stakeholders

d) Is there any other advantage to using DQD over Achilles (except exhaustive rules coverage). Is it the dashboard? Can sites communicate the quality of their dataset to other sites by saying that their data has a quality of 97% or 95% etc. (which is not possible in Achilles)

[fdefalco]

The DQD plausibility checks were expert driven through a working group activity. There was a two day meeting where they attempted to establish plausible values for all measurements. Where they stand today, they are flawed in many cases. There is a branch in the DQD repository called measurement-palooza where we are tracking progress on improving the plausibility thresholds. We are employing a process of learning from our available data as well as combining research on the individual labs, it is a tedious and slow process but we are making progress.

The Achilles Heel report has been retired and should no longer be used. DQD is a superset of data quality checks that adopted a data quality framework. DQD is the recommendation going forward where there are thousands of checks that both evaluate structure and content of a CDM database.

I would not recommend using the percentage of passing rules as a metric and instead recommend the number of data quality checks that fail.