Evaluation of diagnostic tests

Diagnostic tests aim to correctly assess whether or not an animal has a disease. The use of diagnostic tests can have several objectives:

• To assess whether an animal has a disease
• To assess whether an animal has recovered from disease following an intervention
• To prove an animal does not have a disease

Evaluating diagnostic tests

Some common concepts when evaluating diagnostic tests are as follows:

• Accuracy: whether the test accurately measures the variable of interest
• Precision or repeatability: the consistency of the test i.e. whether it repeatedly produces the same result
• Sensitivity: the probability that an infected animal is correctly classified as positive by the diagnostic test
• Specificity: the probability that an uninfected animal is correctly classified as negative by the diagnostic test
• Positive predictive value: the probability of obtaining a positive result in a diseased compared with non-diseased animal
• Negative predictive value: the probability of obtaining a negative result in a non-diseased compared with diseased animal

	Disease status
	Diseased	Non-Diseaed
Positive	A	B
Negative	C	D
Total	A + C	B + D

Where A is the number of animals that are correctly identified as positive (true positives) and D is the number of non-diseased animals that are correctly identified as negative (true negatives). C is the number of diseased animals that incorrectly produce a negative result (false negatives) and B is the number of non-diseased animals that give a positive test result (false positives).

Sensitivity and specificity are the likelihood that animal’s are correctly classified as positive or negative, respectively. As such, sensitivity is calculated by dividing the number of infected animal that correctly tested positive during testing by the number of diseased animals. Specificity is calculated by dividing the number of non-disease animal’s that tested negative during testing by the total number of non-diseased animals. Calculations of sensitivity and specificity require that the true disease status of the animal is known i.e. a gold standard is available. A gold standard is a test with 100% sensitivity and specificity and is very rare therefore the true disease status of the animal often has to be estimated using a test with high sensitivity/specificity.

Sensitivity = A/(A+C)

Specificity = D/(B+D)

The proportion of false negative results that the test is expected to give can be calculated by 1 minus the sensitivity and the proportion of false positive results expected using the test can be calculated by 1 minus the specificity.

Predictive values are the probability that an individual’s test result reflect their true disease status. A positive predictive value (PV+) is the probability that an animal with a positive test result truly has the disease and negative predictive value (PV-) is the probability that an animal with a negative test result is truly free from disease. Predictive values are not only dependent on the characteristics of the test but also on the prevalence of the disease in the population.

PV+ = A/(A+B)

PV- = D/(C+D)

As the prevalence of the disease in the population increases animals are more likely to have the disease. Therefore the probability that an animal which tests positive truly has the disease increases i.e. PV+ increases and the probability that an animal which tests negative has the disease decreases i.e. PV- decreases. If the disease is rare and there is a low prevalence in the population animal's are less likely to have the disease, therefore the likelihood that an animal which tests positive is truly positive is low i.e. PV+ is low and the likelihood that an animal which tests negative does not have the disease is high i.e. PV- is high.

Selecting a cut-off for diagnostic tests

Decisions made following diagnostic testing are usually dichotomous e.g. treat or do not treat the animal, therefore diagnostic tests are usually interpreted as dichotomous outcomes (diseased or non-diseased). In this case, if a diagnostic test is measuring a continuous outcome e.g. antibody titre then a cut-off for classifying animal’s as positive or negative must be selected.

Method	Summary	Advantages	Disadvantages
Gaussian (normal) distribution	Previously the most common method of selecting a cut-off	Easy to use and understand	Assumes test results are normally distributed Does not take the prevalence of disease into account No scientific basis for cut-off
Percentile		Does not assume normality Simple	Does not take prevalence into account No scientific basis for cut-off
Risk factor		Practical if risk factor is easy to measure	Requires knowledge of risk factors PV+ may be low if risk factor is common in population
Therapeutic method		Requires up to date knowledge of treatment methods
Predictive value method	Currently the most common method of selecting a cut-off	Scientific basis for selecting a cut-off Takes charachteristics of the test into account Can be altered depending on objective of testing	Most complex method