Difference between revisions of "Evaluation of diagnostic tests"

Revision as of 15:23, 19 January 2011

Diagnostic tests can be used by veterinarians when determining the likelihood that an animal has a particular disease. The use of diagnostic tests can have several objectives:

To assess whether an animal exhibiting symptoms of a disease has the disease
To detect infection in asymptomatic animals
To assess whether an animal has recovered from disease following an intervention
To prove an animal is free from disease disease

Evaluating diagnostic tests

Some common concepts when evaluating diagnostic tests are as follows:

Accuracy: whether the test accurately measures the variable of interest
Precision or repeatability: the consistency of the test i.e. whether it repeatedly produces the same result
Sensitivity: the probability that an infected animal is correctly classified as positive by the diagnostic test
Specificity: the probability that an uninfected animal is correctly classified as negative by the diagnostic test
Positive predictive value: the probability that an animal that produces a positive test result is truly diseased
Negative predictive value: the probability that an animal that produces a positive test result is truly diseased
Likelihood ratio positive test (LR+): likelihood of obtaining a positive test in a disease compared with non-diseased animal
Likelihood ratio negative test (LR-): likelihood of obtaining a negative test in a diseased compared with non-diseased animal

Sensitivity and Specificity

	Disease status
	Diseased	Non-Diseaed
Positive	A	B
Negative	C	D
Total	A + C	B + D

Where A is the number of animals that are correctly identified as positive (true positives) and D is the number of non-diseased animals that are correctly identified as negative (true negatives). C is the number of diseased animals that incorrectly produce a negative result (false negatives) and B is the number of non-diseased animals that give a positive test result (false positives).

Sensitivity and specificity are the likelihood that animal’s are correctly classified as positive or negative, respectively. As such, sensitivity is calculated by dividing the number of infected animal that correctly tested positive during testing by the number of diseased animals. Specificity is calculated by dividing the number of non-disease animal’s that tested negative during testing by the total number of non-diseased animals. Calculations of sensitivity and specificity require that the true disease status of the animal is known i.e. a gold standard is available. A gold standard is a test with 100% sensitivity and specificity and is very rare therefore the true disease status of the animal often has to be estimated using a test with high sensitivity/specificity.

Sensitivity = A/(A+C)

Specificity = D/(B+D)

The proportion of false negative results that the test is expected to give can be calculated by 1 minus the sensitivity and the proportion of false positive results expected using the test can be calculated by 1 minus the specificity.

Predictive values

Predictive values are the probability that an individual’s test result reflect their true disease status. A positive predictive value (PV+) is the probability that an animal with a positive test result truly has the disease and negative predictive value (PV-) is the probability that an animal with a negative test result is truly free from disease. Predictive values are not only dependent on the characteristics of the test but also on the prevalence of the disease in the population.

PV+ = A/(A+B)

PV- = D/(C+D)

As the prevalence of the disease in the population increases animals are more likely to have the disease. Therefore the probability that an animal which tests positive truly has the disease increases i.e. PV+ increases and the probability that an animal which tests negative has the disease decreases i.e. PV- decreases. If the disease is rare and there is a low prevalence in the population animal's are less likely to have the disease, therefore the likelihood that an animal which tests positive is truly positive is low i.e. PV+ is low and the likelihood that an animal which tests negative does not have the disease is high i.e. PV- is high.

Post-test probability

Before performing a diagnostic test a veterinarian usually has an idea of the likelihood that an animal has a disease, usually based on the levels of the disease in the population. The probability that an animal has a disease before a diagnostic test is performed is termed the ===pre-test probability=== and is usually the prevalence of the disease in the population, but can be modified depending on other factors e.g. whether the animal is showing symptoms of disease, whether certain risk factors for disease are present. Once a diagnostic test has been performed this probability can be modified to incorporate the results of the diagnostic tests to give an overall probability that an animal has the disease i.e. post-test probability. This is carried out as follows:

Pre-test probability = prevalence(P)
Pre-test odds = p/(1-p)
Post-test odds = pre-test odds X LR
Post-test probability = post-test odds/(1+post-test odds)

Selecting a cut-off for diagnostic tests

Decisions made following diagnostic testing are usually dichotomous e.g. treat or do not treat the animal, therefore diagnostic tests are usually interpreted as dichotomous outcomes (diseased or non-diseased). In this case, if a diagnostic test is measuring a continuous outcome e.g. antibody titre then a cut-off for classifying animal’s as positive or negative must be selected. At whichever point the cut-off is selected there is usually some overlap between results i.e. some diseased animals will have the same value as non-diseased animals and hence, there will be false-positive and false-negative results.

Method	Summary	Advantages	Disadvantages
Gaussian (normal) distribution	Previously the most common method of selecting a cut-off	Easy to use and understand	Assumes test results are normally distributed Does not take the prevalence of disease into account No scientific basis for cut-off
Percentile		Does not assume normality Simple	Does not take prevalence into account No scientific basis for cut-off
Risk factor		Practical if risk factor is easy to measure	Requires knowledge of risk factors PV+ may be low if risk factor is common in population
Therapeutic method	An animal is classified as positive at the point where treatment would be administered		Requires up to date knowledge of treatment methods
Predictive value method	Currently the most common method of selecting a cut-off	Scientific basis for selecting a cut-off Takes charachteristics of the test into account Can be altered depending on objective of testing	Most complex method

@@ Line 1: / Line 1: @@
-==Introduction==
+[[Category:Veterinary Epidemiology - General Concepts|K]]
-<br />
+Diagnostic tests can be used by veterinarians when determining the likelihood that an animal has a particular disease. The use of diagnostic tests can have several objectives:
-A diagnostic test is an objective method of deciding whether an animal has a disease, or not. Decisions made following diagnostic testing are usually dichotomous e.g. treat or do not treat the animal, therefore diagnostic tests are usually interpreted as dichotomous outcomes (diseased or non-diseased). In this case, if a diagnostic test is measuring a continuous outcome e.g. antibody titre then a cut-off for classifying animal’s as positive or negative must be selected. The figure below shows that whereever the cut-off is selected there is usually some overlap between results i.e. some diseased animals will have the same value as non-diseased animals and resulting in some false-positive and false-negative results.
+* To assess whether an animal exhibiting symptoms of a disease has the disease
+* To detect infection in asymptomatic animals
+* To assess whether an animal has recovered from disease following an intervention
+* To prove an animal is free from disease disease
+==Evaluating diagnostic tests==
-[[File:Diagnostic test results.jpg]]
+Some common concepts when evaluating diagnostic tests are as follows:
+* '''Accuracy:''' whether the test accurately measures the variable of interest
+* '''Precision''' or '''repeatability:''' the consistency of the test i.e. whether it repeatedly produces the same result
+* '''Sensitivity:''' the probability that an infected animal is correctly classified as positive by the diagnostic test
+* '''Specificity:''' the probability that an uninfected animal is correctly classified as negative by the diagnostic test
+* '''Positive predictive value:''' the probability that an animal that produces a positive test result is truly diseased
+* '''Negative predictive value:'''  the probability that an animal that produces a positive test result is truly diseased
+* '''Likelihood ratio positive test (LR+):''' likelihood of obtaining a positive test in a disease compared with non-diseased animal
+* '''Likelihood ratio negative test (LR-):''' likelihood of obtaining a negative test in a diseased compared with non-diseased animal
-==Interpretation of diagnostic tests==
+<br />
-Several statistics can aid clinicians in the interpretation of diagnostic tests these include '''sensitivity''' and '''specificity''', '''predictive values''', '''likelihood ratios''' and '''pre-''' and '''post-test probabilities'''. The following 2 x 2 table is often used by epidemiologists to calculate these statistics:
+===Sensitivity and Specificity===
-[[File:table.jpg]]
+{| class="wikitable" border="1"
+|-
+!
+! Disease status
+|-
+!
+! Diseased
+! Non-Diseaed
+|-
+! Positive
+| A
+| B
+|-
+! Negative
+| C
+| D
+|-
+! Total
+| A + C
+| B + D
+|}
+Where A is the number of animals that are correctly identified as positive (true positives) and D is the number of non-diseased animals that are correctly identified as negative (true negatives). C is the number of diseased animals that incorrectly produce a negative result (false negatives) and B is the number of non-diseased animals that give a positive test result (false positives). <br />
-===Sensitivity and specificity===
+Sensitivity and specificity are the likelihood that animal’s are correctly classified as positive or negative, respectively.  As such, sensitivity is calculated by dividing the number of infected animal that correctly tested positive during testing by the number of diseased animals. Specificity is calculated by dividing the number of non-disease animal’s that tested negative during testing by the total number of non-diseased animals. Calculations of sensitivity and specificity require that the true disease status of the animal is known i.e. a gold standard is available. A gold standard is a test with 100% sensitivity and specificity and is very rare therefore the true disease status of the animal often has to be estimated using a test with high sensitivity/specificity. <br /><br />
-'''Sensitivity (''Se'')''' is the probability that a positive animal is correctly identified as positive and '''specificity (''Sp'')''' is the probability a negative animal is correctly identified as negative.
+:''Sensitivity = A/(A+C)''<br />
-[[File:specificity.jpg]]
+:''Specificity = D/(B+D)<br /><br />''
+The proportion of false negative results that the test is expected to give can be calculated by 1 minus the sensitivity and the proportion of false positive results expected using the test can be calculated by 1 minus the specificity. <br />
+<br />
 ===Predictive values===
-There are two types of predictive value; '''positive predictive value (PV+)''' is defined as the probability that an animal which tests positive actually has the disease and '''negative predictive value (PV-)''' is the probability and animal that tests negative does not have the disease. PV+ and PV- are calculated as follows:
+'''Predictive values''' are the probability that an individual’s test result reflect their true disease status. A '''positive predictive value (PV+)''' is the probability that an animal with a positive test result truly has the disease and '''negative predictive value (PV-)''' is the probability that an animal with a negative test result is truly free from disease. Predictive values are not only dependent on the characteristics of the test but also on the prevalence of the disease in the population.<br /><br />
-[[File:predictive value.jpg]]
+:''PV+ = A/(A+B)''<br />
-===Likelihood ratio===
+:''PV- = D/(C+D)''<br /><br />
-Likelihood ratios (LR) give clinicians an idea of how likely a test result could have been produced by a diseased compared with a non-diseased animals and can be calculated for both positive and negative test results. The LR for a positive test result (LR+) is interpreted as how likely it is to find a positive test result in diseased compared with non-diseased individuals and LR- is interpreted as how likely it is to find a negative test result in diseased compared with non-diseased animals. Likelihood ratios are a characteristic of the test and do not depend on prevalence, they are calculated as follows:
+As the prevalence of the disease in the population increases animals are more likely to have the disease. Therefore the probability that an animal which tests positive truly has the disease increases i.e. PV+ increases and the probability that an animal which tests negative has the disease decreases i.e. PV- decreases. If the disease is rare and there is a low prevalence in the population animal's are less likely to have the disease, therefore the likelihood that an animal which tests positive is truly positive is low i.e. PV+ is low and the likelihood that an animal which tests negative does not have the disease is high i.e. PV- is high.
+==Post-test probability==
-[[File:likelihood ratio.jpg]]
+Before performing a diagnostic test a veterinarian usually has an idea of the likelihood that an animal has a disease, usually based on the levels of the disease in the population.  The probability that an animal has a disease before a diagnostic test is performed is termed the ===pre-test probability=== and is usually the prevalence of the disease in the population, but can be modified depending on other factors e.g. whether the animal is showing symptoms of disease, whether certain risk factors for disease are present. Once a diagnostic test has been performed this probability can be modified to incorporate the results of the diagnostic tests to give an overall probability that an animal has the disease i.e. '''post-test probability.''' This is carried out as follows:
-===Post-test probability===
+''Pre-test probability  = prevalence(P)''<br />
-Before performing a diagnostic test a veterinarian usually has an idea of the likelihood that an animal has a disease, usually based on the levels of the disease in the population.  The probability that an animal has a disease before a diagnostic test is performed is termed the '''pre-test probability''' and is usually the prevalence of the disease in the population, but can be modified depending on other factors e.g. whether the animal is showing symptoms of disease, whether certain risk factors for disease are present. Once a diagnostic test has been performed this probability can be modified to incorporate the results of the diagnostic tests to give an overall probability that an animal has the disease i.e. '''post-test probability.''' This is carried out as follows:
+''Pre-test odds         = p/(1-p)''<br />
+''Post-test odds        = pre-test odds X LR'' <br />
+''Post-test probability = post-test odds/(1+post-test odds)''
-[[File:post-test probability.jpg]]
+==Selecting a cut-off for diagnostic tests==
+Decisions made following diagnostic testing are usually dichotomous e.g. treat or do not treat the animal, therefore diagnostic tests are usually interpreted as dichotomous outcomes (diseased or non-diseased). In this case, if a diagnostic test is measuring a continuous outcome e.g. antibody titre then a cut-off for classifying animal’s as positive or negative must be selected. At whichever point the cut-off is selected there is usually some overlap between results i.e. some diseased animals will have the same value as non-diseased animals and hence, there will be false-positive and false-negative results.
-[[Category:Veterinary Epidemiology - General Concepts|K]]
+{| class="wikitable" border="1"
+|-
+! Method
+! Summary
+! Advantages
+! Disadvantages
+|-
+| Gaussian (normal) distribution
+| Previously the most common method of selecting a cut-off
+|
+*Easy to use and understand
+|
+*Assumes test results are normally distributed
+*Does not take the prevalence of disease into account<br />
+*No scientific basis for cut-off
+|-
+|Percentile
+|
+|
+*Does not assume normality
+Simple
+|
+*Does not take prevalence into account
+*No scientific basis for cut-off
+|-
+| Risk factor
+|
+|
+*Practical if risk factor is easy to measure
+|
+*Requires knowledge of risk factors
+*PV+ may be low if risk factor is common in population
+|-
+| Therapeutic method
+| An animal is classified as positive at the point where treatment would be administered
+|
+|
+*Requires up to date knowledge of treatment methods
+|-
+| [[Predictive value method]]
+| Currently the most common method of selecting a cut-off
+|
+*Scientific basis for selecting a cut-off
+*Takes charachteristics of the test into account
+*Can be altered depending on objective of testing
+|
+*Most complex method
+|}