Economic considerations for disease testing strategies (Proceedings)

Many veterinarians express frustration when trying to provide their clients with the best advice on which diagnostic tests to recommend for purchased cattle or the resident herd. The goal is to screen apparently healthy cattle to identify carriers of infectious disease that could cause reproductive losses and other health problems in the herd. To determine the economic return for diagnostic testing strategies, veterinarians need information on the accuracy (sensitivity and specificity) of available tests, commonness of the condition in question in the cattle population at large and specified sub-populations, disease dynamics such as reservoir, transmission pattern, incubation period, immune response, treatment efficacy, and negative or unintended consequences of diagnosis or treatment. In addition to the federal eradication programs for brucellosis and tuberculosis, the diseases with long-term carrier states that are most often considered for screening purchased cattle include: Trichomoniasis, Bovine Viral Diarrhea (persistent infection (PI) status), Bovine Leukosis, Anaplasmosis, and Johnes.

The most appropriate method to determine the economic value for diagnostic testing will vary depending on the condition in question, the time frame involved, and how the diagnostic information will be utilized to make decisions. The most straightforward method is by utilizing a partial budget. For rare conditions or events, it may be more appropriate to determine the cost of a negative outcome and the cost of intervention - and working with the client, determine his/her level of risk aversion, and together, determine the value of reducing the risk of a rare event.

Determining Diagnostic Test Usefulness and Diagnostic Strategy

A valid question confronting veterinary practitioners is whether to use available diagnostic tests to screen a particular herd or purchased replacements (bulls, heifers, and cows) for a particular condition. The input needed to arrive at a logical conclusion includes epidemiologic data about the condition or disease, diagnostic test sensitivity and specificity data, disease or condition dynamics, and economic costs of the condition and its treatment.3 Literature review and mathematic aids, such as computer spreadsheets and expert systems, are the tools used to create the necessary outputs. These outputs include post-test predictive values of diagnostic tests, economic value of testing, sensitivity of the decision to the individual inputs, and the importance of individual inputs to the decision. These outputs are used to evaluate alternate diagnostic testing strategies in order to indicate the best testing strategy, and to identify the control points to be monitored for change that can trigger a re-evaluation of the decision.

Sensitivity and Specificity of Diagnostic Tests

Sensitivity and specificity are properties of a diagnostic test that are determined by comparing the test to a "gold standard". The gold standard is considered the true diagnosis, and may be made using a variety of such information as clinical examination, expert opinion, laboratory results, or postmortem results. Sensitivity is the proportion of known positive (gold standard-positive) samples that the test in question identifies as positive. Specificity is the proportion of known negative samples that the test in question identifies as negative. In other words, sensitivity answers the question, "How effective is the test at identifying animals with the condition?" and specificity answers the question, "How effective is the test at identifying animals without the condition?"

Because in almost every situation, there is overlap between the test results of truly negative bulls and truly positive bulls, it is generally impossible to have a test that is 100% accurate. Because diagnostic tests (both laboratory and clinical examination tests) use an arbitrary cut off to separate test-positive and test-negative populations, sensitivity and specificity are inversely related, and placing the cut off is always a trade-off between the impacts of false-negative and false-positive results.

Prevalence (Commonness)

Prevalence is the number of cases of a condition at a given time compared to the population size at that time. Each practitioner's judgment, based on history and clinical examination of both individuals and the population, aided by available prevalence information, is often all we have to estimate disease probability.

For a test with imperfect specificity, an increasing proportion of the test positives will be false positives as prevalence decreases. At low prevalence, the majority of test positives will be false positives, so that for an uncommon condition, even a highly accurate test will render results that must be interpreted with care when applied to the animal population as a whole. In other words, in the case of very rare conditions, most tests for that condition that appear to be positive are actually false-positive.

Post-test Predictive Value

The post-test predictive values of a test are determined, not in the laboratory, but in the field and they tell if a valid test is useful. The positive predictive value is the proportion of animals with a positive test result that are actually positive, and is influenced by test specificity. The negative predictive value is the proportion of animals with a negative test result that are truly negative, and is influenced by test sensitivity. Both positive and negative predictive values of a test are affected by the commonness (prevalence) of the condition. As the prevalence of the condition rises, more animals in the population have the condition, and we have greater confidence that a positive test result is correct. With increasing prevalence, the positive predictive value of the test increases and the negative predictive value decreases, while the reverse is true as prevalence of the condition is decreasing.

It is often impossible to estimate prevalence with any confidence, but one must still consider predictive value in test interpretation. When screening a herd, one often has no data to suggest that an individual animal is in a particularly high-prevalence group. In such a mode with a test with good sensitivity, a negative test result has a high negative predictive value and is useful in striking a rule-out off the list, but a positive test result (which is most likely a false positive) is useful only in keeping a rule-out on the active list and does not mean a the diagnosis has been confirmed.

Diagnostic Testing Strategy

To rule-in a potential diagnosis, many times it is necessary to use more than one test, either in series or in parallel. Running tests in series, where a second test is submitted only after the first test returns a positive result, is used to confirm a positive test with a low positive predictive value (low specificity or low prevalence). A two-test series is interpreted as negative if either test results in a negative response (i.e. BVD PI testing). Running two or more tests in parallel, where they are submitted simultaneously or taken sequentially from essentially the same population, is used to confirm a positive test with low negative predictive value (low sensitivity or high prevalence). Parallel tests are interpreted as positive if any test results in a positive response (i.e. Trich testing).

Determining Economic Benefit of Diagnostic Strategy

The cost effectiveness of alternate diagnostic testing strategies can be compared with a partial budget (Figure 1). In this partial budget, the post-test predictive values, test cost, cost of the negative condition, treatment cost, and cost of false positives are used to calculate the return for true positives, true negatives, false positives, and false negatives. The economic benefit is simply the costs for true negatives, false positives, and false negatives subtracted from the return for true positives.

Figure 1. Diagnostic Testing Partial Budget

The value of a testing strategy, whereby all incoming animals are tested and the true-positive animals are isolated or euthanized, is the value of avoiding disease spread in the population. The cost of true negatives is essentially the cost of doing the diagnostic tests, including laboratory costs, veterinary labor and consulting costs for handling the tests, and labor for handling the animals. The cost for false positives is the cost of isolating or euthanizing an animal that was not truly infected. And, the cost of false negatives is the cost of leaving a positive animal in the herd.

For conditions that are rare (low prevalence), even with an accurate test (but less than 100% specific), many of the positive test results will be false-positive, and the costs of finding true negatives (i.e. testing cost) and the cost of false-positives may be greater than the value of finding the few true-positives. In this situation, a partial budget evaluation may indicate little or no economic benefit for a testing strategy unless the cost of disease is substantial.

Because some of the relatively infrequent negative conditions of interest to veterinarians can have significant production and economic costs when present, the cost of an infrequent but significant condition can be better evaluated as an assessment of risk and cost of risk avoidance. Once the cost of the risk is quantified, the producer and veterinarian can determine the effects such an event would have on a confined period's cash flow, and can evaluate that effect with the cost of risk reduction.

Conclusions

Use of diagnostic tests for biosecurity purposes for purchased cattle offers veterinarians a tool to reduce the cost of disease for our cow-calf clients. Veterinarians should use information about test sensitivity and specificity, disease prevalence, test cost, and the cost of disease to calculate the expected value of testing for biosecurity reasons.

References

Rothman KJ, Greenland S: Causation and causal inference, in K.J. Rothman, S. Greenland (ed): Modern Epidemiology, 2nd ed. Philadelphia, PA, Lippincott Williams & Wilkins, 1998, pp. 7-28.

Larson RL, Pierce VL: Agricultural economics for veterinarians: Partial budgets for beef cow herds. Compend Cont Ed Prac Vet 21(9):S210-S219; 1999

Clemen R.T. (ed)P: Modeling uncertainty - Monte Carlo Simulation, in Making Hard Decisions: An Introduction to Decision Analysis. Belmont, CA, Duxbury Press, 1991, pp. 167

Romatowski J: Problems in interpretation of clinical lab tests. JAVMA 205:1186-1188, 1994.