"Proving the null hypothesis" in clinical trials.
Blackwelder WC. „Proving the null hypothesis” in clinical trials. Control Clin Trials 1982; 3: 345-353
"Proving the null hypothesis" in clinical trials
OBJECTIVE: To review published clinical trials comparing the efficacy of trazodone with that of tricyclic antidepressant medication. DATA SOURCES: MEDLINE was searched for relevant articles published from 1983 to 1991. The bibliography of a review article was searched for further references. STUDY SELECTION: In all, 25 clinical trials were found. Six of these met the methodologic assessment criteria (adapted from the McMaster guidelines for the evaluation of clinical trials), which included the stipulation of a score of 18 or more on the Hamilton depression rating scale and a 50% reduction in that score as an outcome measure. DATA EXTRACTION: All six studies compared trazodone with imipramine. Data describing response to the treatments were extracted, and post-hoc power estimates were calculated. The analysis also involved statistical tests of a modified null hypothesis, the generation of confidence intervals (CIs) and a meta-analysis. DATA SYNTHESIS: All the studies found no significant difference in the efficacy of trazodone and imipramine. However, the statistical power of most of them was less than 50% and often less than 10%; thus there was a low probability that differences would be detected. The results of statistical tests of the modified null hypothesis, inspection of the CIs and the results of the meta-analysis all suggested that trazodone, and imipramine are equally efficacious. CONCLUSION: The application of various techniques for the analysis of equivalence data suggests that trazodone and imipramine are of approximately equivalent efficacy. The data are compatible with small differences in efficacy, but the differences are of a magnitude such that they are unlikely to be of clinical significance.
To avoid misinterpretation of null hypothesis testing, some investigators, including those who contributed to the CONSORT guidelines, favor the confidence interval approach to show noninferiority of two treatments (; ; ; ). The width of the interval signifies the extent of noninferiority, which is a favorable characteristic of this approach. If the confidence interval for the difference between two interventions lies to the right of the noninferiority margin () then noninferiority can be concluded. If the interval crosses the boundary (contains the value of the margin) then noninferiority cannot be claimed. Some investigators prefer the confidence interval approach to examine the precision of noninferiority. Others prefer hypothesis testing. The authors recommend that both confidence interval and p-values be provided to allow the audience to interpret the extent of the findings.
"Proving the Null Hypothesis" in Clinical Trials William C
A radical approach is to change the whole system of statistical hypothesis testing to evaluate the effectiveness of new technologies or to test hypotheses in any field of scientific knowledge. This intends to be a new paradigm that promotes replacing frequentist techniques (as are often called these methods of testing hypotheses we have mentioned) by techniques . Bayesian methods to test hypotheses, are not based on the p-value obtained from a study in which all factors that may affect compliance with the hypothesis, except the one studied, are controlled. The encouraged idea is that the accumulated experience on a hypothesis can and should contribute to its verification. With previous experience (plus probability theory, of course) "a priori probability" is adopted and, with the data provided by a new study the "posterior probability" that would lead to decision making is attained. This is roughly the idea underlying Bayesian techniques that constitute today a broad array struggling to prevail as the main method of statistical inference but, apparently, has not been as successful as it might have been expected. Presciently in 1987, in an article in JAMA, Browner and Newman delved into the similarity between hypothesis testing in research and evaluation of diagnostic tests in the clinic scenario where and probabilities become particularly relevant .
The Department of Defense's office of Congressionally Directed Medical Research Programs has recently funded this 4-year prospective clinical trial evaluating the effectiveness of a telemental health modality, specifically videoteleconferencing, as a means of providing a well validated PTSD specific treatment intervention, Cognitive Processing Therapy (CPT; ) to veterans in remote locations, as compared to the traditional face-to-face modality. In this ongoing study we hypothesize that using a telemental health modality will be non-inferior to the traditional mode of service delivery (face-to-face), on both clinical and process measures, for providing CPT in a group format for veterans with PTSD. In this study we are conducting 9 cohorts of CPT groups over a 3-year period. Each cohort includes a telemental health condition and a traditional face-to-face condition. The noninferiority design was selected as the best way to determine if the novel, less expensive telemental health modality is as good as the traditional face-to-face delivery mode for the delivery of CPT. The prespecified noninferiority margin or “clinically meaningful difference” was based on both clinical and statistical justifications. Both confidence interval and hypothesis testing approaches will be used in the final analysis. If our final data analyses show that the difference in clinical effectiveness (significant change in PTSD symptoms on the Clinician Administered PTSD Scale [CAPS; ] between the two conditions) is less than the prespecified noninferiority margin, we can conclude that telemental health delivery is not inferior to in-person deliver of CPT (i.e., we can reject the null hypothesis of inferiority). With a finding of noninferiority, one could be confident in recommending the novel, more easily accessible telemental health treatment modality in lieu of the standard face-to-face treatment without any significant loss of clinical benefit. It is expected that results from this project can be applied to other sites where specialized PTSD clinical services are needed but unavailable due to geographic barriers.
Proving the Null Hypothesis” in Clinical Trials
The choice of noninferiority margin is another critical decision. Unfortunately, there is no gold standard criterion for determining an appropriate margin (). In fact, the only consistent recommendations from regulators are that the margin is determined in advance and that it should not be greater than the smallest effect size the active drug would be reliably expected to have compared with a placebo (; ). If the margin is too large, rejecting the null hypothesis is meaningless; but if the margin is too small, power to detect noninferiority is dramatically reduced (). Some researchers prefer to derive the margin based on statistical properties. That approach often leads to noninferiority margins that are relative to effect size. Typically it is a fraction, usually one half or less, of the historical effect size of the standard intervention (). It can also be a percentage of the effect of the standard treatment in the current trial. For example, the novel intervention must be at least 80% as effective as the standard intervention. Perhaps the most common approach in treatment outcome studies is to set a margin based on what is considered “clinically unimportant.”
The statistical analysis of noninferiority can be conducted by either using confidence intervals or by applying variations to the analytic strategy of null hypothesis testing. Under conventional hypothesis testing in a comparative study of two interventions, the goal is to reject the null in favor of a difference between the two interventions. If this approach is extended to a noninferiority study, then can noninferiority be claimed when the test fails to reject the null hypothesis of no difference? Some argue that it is acceptable assuming that a strict level of Type II error (failing to reject the null when it is false) is provided (; ). However, others argue that it is logically impossible to conclude noninferiority on the basis of failing to reject the null hypothesis (; ; ). Rather, failure to reject the null means that there is not sufficient evidence to accept the alternative hypothesis. Alternately, if the null hypothesis states that the true difference is greater than or equal to the pre-specified noninferiority margin, then failing to reject it can be interpreted as insufficient evidence to conclude that the difference of the two procedures is less than the pre-specified noninferiority margin.
Proving the Null Hypothesis” in Clinical Trials.
Recommended: Proving the null hypothesis in clinical trials
“Proving the null hypothesis” in clinical trials, Controlled Clinical ..
such as Blackwelder's “Proving the null hypothesis ..
“Proving the Null Hypothesis in Clinical Trials.” Controlled Clinical Trials 3, 345-353
Blackwelder Proving the null hypothesis» in clinical trials
Blackwelder Proving the null hypothesis» in clinical trials.
"I have always been impressed by the quick turnaround and your thoroughness. Easily the most professional essay writing service on the web."
"Your assistance and the first class service is much appreciated. My essay reads so well and without your help I'm sure I would have been marked down again on grammar and syntax."
"Thanks again for your excellent work with my assignments. No doubts you're true experts at what you do and very approachable."
"Very professional, cheap and friendly service. Thanks for writing two important essays for me, I wouldn't have written it myself because of the tight deadline."
"Thanks for your cautious eye, attention to detail and overall superb service. Thanks to you, now I am confident that I can submit my term paper on time."
"Thank you for the GREAT work you have done. Just wanted to tell that I'm very happy with my essay and will get back with more assignments soon."