Evaluation of diagnostic tests

Diagnostic tests can be used by veterinarians when determining the likelihood that an animal has a particular disease. The use of diagnostic tests can have several objectives:

• To assess whether an animal exhibiting symptoms of a disease has the disease
• To detect infection in asymptomatic animals
• To assess whether an animal has recovered from disease following an intervention
• To prove an animal is free from disease disease

Evaluating diagnostic tests

Some common concepts when evaluating diagnostic tests are as follows:

• Accuracy: whether the test accurately measures the variable of interest
• Precision or repeatability: the consistency of the test i.e. whether it repeatedly produces the same result
• Sensitivity: the probability that an infected animal is correctly classified as positive by the diagnostic test
• Specificity: the probability that an uninfected animal is correctly classified as negative by the diagnostic test
• Positive predictive value: the probability that an animal that produces a positive test result is truly diseased
• Negative predictive value: the probability that an animal that produces a positive test result is truly diseased
• Likelihood ratio positive test (LR+): likelihood of obtaining a positive test in a disease compared with non-diseased animal
• Likelihood ratio negative test (LR-): likelihood of obtaining a negative test in a diseased compared with non-diseased animal

Sensitivity and Specificity

	Disease status
	Diseased	Non-Diseaed
Positive	A	B
Negative	C	D
Total	A + C	B + D

Where A is the number of animals that are correctly identified as positive (true positives) and D is the number of non-diseased animals that are correctly identified as negative (true negatives). C is the number of diseased animals that incorrectly produce a negative result (false negatives) and B is the number of non-diseased animals that give a positive test result (false positives).

Sensitivity and specificity are the likelihood that animal’s are correctly classified as positive or negative, respectively. As such, sensitivity is calculated by dividing the number of infected animal that correctly tested positive during testing by the number of diseased animals. Specificity is calculated by dividing the number of non-disease animal’s that tested negative during testing by the total number of non-diseased animals. Calculations of sensitivity and specificity require that the true disease status of the animal is known i.e. a gold standard is available. A gold standard is a test with 100% sensitivity and specificity and is very rare therefore the true disease status of the animal often has to be estimated using a test with high sensitivity/specificity.

Sensitivity = A/(A+C)

Specificity = D/(B+D)

The proportion of false negative results that the test is expected to give can be calculated by 1 minus the sensitivity and the proportion of false positive results expected using the test can be calculated by 1 minus the specificity.

Predictive values

Predictive values are the probability that an individual’s test result reflect their true disease status. A positive predictive value (PV+) is the probability that an animal with a positive test result truly has the disease and negative predictive value (PV-) is the probability that an animal with a negative test result is truly free from disease. Predictive values are not only dependent on the characteristics of the test but also on the prevalence of the disease in the population.

PV+ = A/(A+B)

PV- = D/(C+D)

As the prevalence of the disease in the population increases animals are more likely to have the disease. Therefore the probability that an animal which tests positive truly has the disease increases i.e. PV+ increases and the probability that an animal which tests negative has the disease decreases i.e. PV- decreases. If the disease is rare and there is a low prevalence in the population animal's are less likely to have the disease, therefore the likelihood that an animal which tests positive is truly positive is low i.e. PV+ is low and the likelihood that an animal which tests negative does not have the disease is high i.e. PV- is high.

Post-test probability

Before performing a diagnostic test a veterinarian usually has an idea of the likelihood that an animal has a disease, usually based on the levels of the disease in the population. The probability that an animal has a disease before a diagnostic test is performed is termed the pre-test probability and is usually the prevalence of the disease in the population, but can be modified depending on other factors e.g. whether the animal is showing symptoms of disease, whether certain risk factors for disease are present. Once a diagnostic test has been performed this probability can be modified to incorporate the results of the diagnostic tests to give an overall probability that an animal has the disease i.e. post-test probability. This is carried out as follows:

Pre-test probability = prevalence(P)
Pre-test odds = p/(1-p)
Post-test odds = pre-test odds X LR
Post-test probability = post-test odds/(1+post-test odds)

Selecting a cut-off for diagnostic tests

Decisions made following diagnostic testing are usually dichotomous e.g. treat or do not treat the animal, therefore diagnostic tests are usually interpreted as dichotomous outcomes (diseased or non-diseased). In this case, if a diagnostic test is measuring a continuous outcome e.g. antibody titre then a cut-off for classifying animal’s as positive or negative must be selected. At whichever point the cut-off is selected there is usually some overlap between results i.e. some diseased animals will have the same value as non-diseased animals and hence, there will be false-positive and false-negative results.

Method	Summary	Advantages	Disadvantages
Gaussian (normal) distribution	Previously the most common method of selecting a cut-off	Easy to use and understand	Assumes test results are normally distributed Does not take the prevalence of disease into account No scientific basis for cut-off
Percentile		Does not assume normality Simple	Does not take prevalence into account No scientific basis for cut-off
Risk factor		Practical if risk factor is easy to measure	Requires knowledge of risk factors PV+ may be low if risk factor is common in population
Therapeutic method	An animal is classified as positive at the point where treatment would be administered		Requires up to date knowledge of treatment methods
Predictive value method	Currently the most common method of selecting a cut-off	Scientific basis for selecting a cut-off Takes charachteristics of the test into account Can be altered depending on objective of testing	Most complex method