Z Score vs SDI

You work hard to implement top class quality control in all areas of your laboratory. The success of your labours is reported to you through your External Quality Assessment (EQA) results. It can be frustrating when your report is returned, only for you to find that you’ve been assigned a poor performance score due to other laboratories in your participation group.

At RIQAS, we want your EQA results to reflect your performance, not that of everyone else, to truly illustrate the efficacy of your quality control procedures. This is why, instead of Z scores, we report your performance in terms of a Standard Deviation Index (SDI). However, we know that in some countries, you’re required to report a Z score. Don’t fret. You can still find this result in the .csv file provided with your report.

A Z score is a statistical measurement that describes a value’s relationship to the mean of a group of values. In other words, it’s a value calculated to tell us how many standard deviations (SDs) a result is from the expected mean. Z score is reported in terms of SD’s, therefore a Z score of 0 shows the result is identical to the mean.

While useful in many cases, when used in EQA, a Z score can give a false perception of performance. We want RIQAS participant performance assessment to be based on their individual performance, rather than being impacted by how well, or poorly, the other laboratories in the group performed for a sample.

Z score is calculated using a variable SD. This means that as results are added, the mean and SD can change. For example, if overall performance for a sample improves, the CV associated with the data will decrease, causing an increase in Z score. Let’s take a quick look at how RIQAS performance assessment works, and then we can get into SDI.

RIQAS Performance Assessment.

Our target scoring system has been developed to provide a simple interpretation of your laboratory’s performance. To calculate a target score, your result is calculated as a percentage deviation (V) from the Mean for Comparison. This deviation is then compared to a Target Deviation for Performance Assessment (TDPA) to calculate the Target Score.

The difference between your result and the mean for comparison is expressed as a Target Score (TS) using the following mathematical formulae:

The better your percentage deviation compared to the TDPA, the higher your Target Score will be.

TDPA are set to encourage participants to achieve and maintain acceptable performance. Target Deviations are assigned to be fit-for-purpose and take all possible sources of variation into account, including sample homogeneity and stability as per ISO/IEC17043, ISO13528 and IUPAC.

In general, the TDPA is set so that ~10% laboratories achieve Target Scores less than 50. However, depending on homogeneity and stability, the TDPAs may be adjusted, so that participants’ performance is not adversely affected by sample variability. If your % deviation (V) is equal to the Target Deviation for Performance Assessment (TDPA) then a target score of 50 is achieved.

RIQAS reviews TDPAs annually and the methods used to assign them have been agreed by the RIQAS Advisory Panel.

Standard Deviation Index (SDI)

To provide a more accurate assessment of performance, we use SDI instead of Z score. SDI is a score which compares the participant’s difference from the assigned value (mean for comparison) with an evaluation interval called the Standard Deviation for Performance Assessment (SDPA).

The SDPA calculation involves a series of steps. First, we calculate a CV for Performance assessment (CVPA) as shown below:

As mentioned, the TPDA is normally set so that ~10% of laboratories achieve a TS less than 50. In such cases, the t-value used to convert TDPA to CVPA is ~1.645. However, depending on homogeneity and stability, the TDPA may need be increased, so that participants’ performance is not adversely affected by sample variability. In such cases less than 10% of laboratories will have poor performance, and a larger t-value will be chosen to convert TDPA to CVPA

We then convert CVPA to SDPA:

Using this equation, an initial SDPA is calculated for every mean for comparison (i.e. for all methods, method, and instrument statistics). However, for new parameters or those which have small participation numbers, it’s not always possible to assign a target deviation, TDPA or SDPA. In such cases, the SDPA will be the SD calculated when the mean for comparisons is generated.

According to ISO/IEC17043, when the assigned value is based on consensus (mean for comparison), the uncertainty of the assigned value must be calculated and combined with the SDPA when it is considered to be significant. This forms an adjusted SDPA, which is used to calculate the participant’s performance in terms of SDI.

Using the SDPA_adjusted we can calculate SDI using the formula below:

On your RIQAS report, you’ll find the SDI associated with the current sample in the text section of each report page. We also provide your last 20 SDIs, plotted on a Levey-Jennings chart, along with an indication of the mean for comparison for each sample (I = Instrument group, M = Method group, or A = All Methods group). Acceptable performance is an SDI of less than ± 2.

RIQAS EQA

RIQAS is the world’s largest EQA scheme with more than 75,000 laboratory participants spanning over 138 countries. Choosing an EQA provider is no easy task. That’s why we’ve produce a guide to help you find the right one for you. You can download it here.

At RIQAS, we’re always coming up with new ways to make your performance assessment and result interpretation even easier. We’re also proud of our new programmes and pilot schemes. This year, we’re running pilot programmes for Anti-psychotic drugs, Chagas and Blood Typing.

If you’d like to find out more about the range of programmes we provide, visit our website or download our brochure. Alternatively, you can get in touch with us at marketing@randox.com.