This meta analysis is designed to exclude most studies. Authors
select a small subset of studies, with a majority of results based on only 1
or 2 studies. Authors split up studies which dilutes the effects and results
in a lack of statistical significance for most outcomes. Authors perform 16+
meta analyses with very few studies in each analysis, and do not combine the
evidence from all studies. However, we can consider the probability of the
observed results across all outcomes.
Authors find positive results for 11 of 12 primary efficacy
outcomes with events, or 16 of 18 including secondary outcomes. One of the
primary outcomes and two of the secondary outcomes show statistically
significant improvements in isolation. If we assume independence, the
probability that 11+ of 12 primary efficacy outcomes were positive for an
ineffective treatment is p = 0.003. For 16+ of 18 outcomes we get
p = 0.0007. This simple analysis does not take into account the
magnitude of positive effects, or the dependence due to some studies
contributing multiple outcomes, however observation suggests that a full
analysis of the combined evidence is likely to show efficacy.
The study is entirely retrospective in the current version. The
protocol is dated April 20, 2021, and the most recent study included is from
March 9, 2021. The protocol was modified after publication in order to include
a close to null result (
[Beltran Gonzalez] "patients discharged without
respiratory deterioration or death at 28 days"), so the current protocol is
dated July 28, 2021.
Authors excluded many studies by requiring results at a
specific time, for example mortality, ventilation, etc. required results at
exactly 28 days. Authors excluded all prophylaxis studies by requiring
results at exactly 14 days.
Studies comparing with other medications were excluded, however
these studies confirm efficacy of ivermectin. The only case where they could
overstate the efficacy of ivermectin is if the other medication was harmful.
There is some evidence of this for excessive dosage/very late stage use,
however that does not apply to any of the studies here.
Studies using combined treatment were excluded, even when it is
known that the other components have minimal or no effect. 3 of 4 RCTs with
combined treatment use doxycycline in addition, which was shown to have no
significant effect in
[Butler]. Other studies were excluded by
requiring PCR confirmation.
Authors are inconsistent regarding active comparators. They
state that hydroxychloroquine “does not work”, yet excluded trials comparing
ivermectin to a drug they hold to be inactive. On the other hand, remdesivir
was an acceptable comparator, although it is considered to be effective
standard of care in some locations
[Fordham].
Authors fail to recognize that Risk of Bias (RoB) domains such
as blinding are far less important for the objective outcome of
mortality.
Authors include
[Beltran Gonzalez] as "moderate"
COVID-19, however patients in this study were in severe condition (baseline
SatO2 83).
[Fordham] summarizes several problems:
•unsupported assertions of adverse
reactions to ivermectin, and the outdated claim that unsafe dosing would be
needed to be effective;
•a demand for PCR or antigen testing,
without analysis of reliability and not universally available even in
developed countries at the start of the pandemic;
•contradictions in the exclusion
criteria, including placebo and approved SoC comparators, but rejecting
hydroxychloroquine, though held to be ineffective (and an approved SoC in
some jurisdictions);
•inclusion of “deemed active”
comparators whilst excluding “potentially active” ones;
•exclusion of combination therapies,
though the norm among practising clinicians;
•the rejection of other than RCTs when
the objective is a “complete evidence profile”;
•arbitrary time-points for outcome
measures, excluding non-compliant trials;
•fragmentation of data by location of
care under varying hospitalisation criteria;
•the resulting focus on a small
fraction of the available clinical evidence, with most comparisons based on
single studies with no meta-analysis possible;
•a resulting inpatient mortality
comparison with fewer patients than a June 2020 confounder-matched
study;
•no conclusion on the headline
mortality outcome, when multiple lines of evidence from elsewhere
(including the WHO) point to significant mortality advantage.
Cochrane was reputable in the past, but is now controlled by
pharmaceutical interests. For example, see the news related to the expulsion
of founder Dr. Gøtzsche and the associated mass resignation of board members
in protest
[blogs.bmj.com, bmj.com, en.x-mol.com].
For another example of bias see
[ebm.bmj.com].
The BiRD group gave the following early comment: "Yesterday’s
Cochrane review surprisingly doesn’t take a pragmatic approach comparing
ivermectin versus no ivermectin, like in the majority of other existing
reviews. It uses a granular approach similar to WHO’s and the flawed Roman et
al paper, splitting studies up and thereby diluting effects. Consequently,
the uncertain conclusions add nothing to the evidence base. A further
obfuscation of the evidence on ivermectin and an example of research waste.
Funding conflicts of interests of the authors and of the journal concerned
should be examined."
For dicussion of issues added in the updated version see
[Popp].
Authors report funding from the German Federal Ministry of
Education and Research, which may be influenced by
[gcgh.grandchallenges.org].
Popp et al., 28 Jul 2021, preprint, 8 authors.
