ROC Curve Analysis

What is it? A Receiver’s Operating Characteristic (ROC) curve plots every value of a continuous measurement by its specificity and sensitivity to distinguish health status in a population of subjects. The area under the curve (AUC) reflects the measurement’s potential to be a diagnosis tool. At a specific sensitivity, the specificity can be determined, or vice versa.

When is it used? This analysis is used to identify the appropriate classifying thresholds to diagnose a patient with expected sensitivity and specificity.

How does it work?

ROC curve analysis: Example

We’ve identified a potential biomarker, Protein “A”, of Alzheimer’s disease that is elevated in the plasma of Alzheimer’s patients compared to healthy patients (Figure 1). We now need to determine the lower cut-off value of “Protein A” levels that will identify a patient with Alzheimer’s disease. We don’t want to have the threshold too low like concentration X in Figure 1 or else a lot of healthy patients will be wrongly diagnosed. However we also don’t want to have a threshold that is really high like concentration Y in Figure 1 because a lot of Alzheimer’s patients won’t get diagnosed. A ROC curve can help identify the “sweet spot” (i.e., optimum sensitivity-specificity balance).

Figure 2 explains what sensitivity and specificity are. Ideally, the sensitivity and specificity would be 100%. In reality, virtually all biomarkers do not have perfect sensitivity and specificity.

A ROC curve is generated across all values and the AUC is determined (Figure 3). Higher AUC values represent a better biomarker. A point along the ROC curve is chosen with the desired trade-off between sensitivity and specificity. With known sensitivity and specificity, the cut-off value can be ascertained.

For this example, let’s assume the black dashed line is the ROC curve for our data. We would likely choose the X-Y coordinate of 0.1, 0.8, such that the biomarker would have a specificity of 90% and a sensitivity of 80%.

Figure 1. Overlapping histogram plots for concentrations of Protein A in different populations. A cut-off of concentration "X" will have high sensitivity, but low specificity. A cut-off of concentration "Y" will have low sensitivity, but high specificity.
Figure 2. Calculation of senstivity and specificty.
Figure 3. Comparison of ROC curves across three potential biomarkers. The higher the AUC value, the higher predictive value of the biomarker. Biomarker 3 has very poor predictive power (AUC ~0.5) as it cannot differentiate between healthy and diseased patients at all.

What does the data look like? ROC curve analyses are usually portrayed as a plot like Figure 3.

Leave a Reply

Your email address will not be published. Required fields are marked *