                    United States Court of Appeals
                         FOR THE EIGHTH CIRCUIT
               ___________

                No. 11-1809
               ___________

Pamela Kuhn,                          *
                                      *
             Appellant,               *
                                      *
      v.                              *
                                      *
Wyeth, Inc., formerly known as        *
American Home Products; Wyeth         *
Pharmaceuticals, Inc., formerly known *
as Wyeth-Ayerst Pharmaceuticals,      *
Inc.,                                 *   Appeals from the United States
                                      *   District Court for the
             Appellees.               *   Western District of Arkansas.
      _______________________         *
                                      *
Arkansas Trial Lawyers Association,   *
                                      *
             Amicus Curiae.           *
               ___________

                No. 11-1815
               ___________

Shirley Davidson,                     *
                                      *
            Appellant,                *
                                      *
      v.                              *
                                      *
Wyeth; Wyeth Pharmaceuticals, Inc.,   *
                                      *
            Appellees.                *
                                    ___________

                              Submitted: January 10, 2012
                                  Filed: July 26, 2012
                                  ___________

Before WOLLMAN, LOKEN and GRUENDER, Circuit Judges.
                          ___________

WOLLMAN, Circuit Judge.

       Pamela Kuhn and Shirley Davidson were prescribed Prempro, a hormone
therapy drug comprised of estrogen and progestin and manufactured by Wyeth.1
Kuhn took Prempro for three years and twenty-eight days; Davidson took it for one
year and nine months. Both women developed breast cancer and filed separate
lawsuits against Wyeth in the Western District of Arkansas. The complaints alleged,
among other things, that the use of Prempro increased the risk of breast cancer and
that Wyeth failed to adequately warn of the drug’s adverse effects. Both cases were
transferred to the Multidistrict Litigation proceedings in the Eastern District of
Arkansas, where Wyeth moved to preclude any expert testimony that Prempro use
increases breast cancer risk when taken for three years or less.

       On behalf of Kuhn and Davidson (collectively, Plaintiffs), Donald F. Austin,
M.D., opined that short-term use of Prempro increases the risk of breast cancer. The
magistrate judge to whom the evidentiary matter was referred concluded that Dr.
Austin’s opinion was not sufficiently reliable to meet the admissibility standard set
forth in Daubert v. Merrell Dow Pharmaceuticals, Inc., 509 U.S. 579 (1993), and thus
granted Wyeth’s motion. Thereafter, the district court overruled Plaintiffs’ objections



      1
       This opinion uses the name “Wyeth” to refer to the named defendants, Wyeth,
Inc.; Wyeth Pharmaceuticals, Inc.; and Wyeth.

                                         -2-
to the Daubert order, granted Wyeth’s motion for summary judgment, entered
judgment dismissing the two cases, and denied Plaintiffs’ post-trial motions.

       Kuhn and Davidson appeal. They argue that the magistrate judge abused his
discretion in granting Wyeth’s motion to preclude expert testimony and that summary
judgment based on the preclusion of expert testimony was inappropriate. Kuhn also
argues that her case did not qualify as a short-term use case because she had taken
Prempro for more than three years. We conclude that the magistrate judge abused his
discretion in precluding Dr. Austin’s expert testimony, and thus we reverse and
remand for further proceedings.

                                   I. Background

                              A. General Background

      Prempro is a combination hormone therapy composed of conjugated equine
estrogen and medroxyprogesterone acetate.2 It is used to treat symptoms of
menopause, including vasomotor symptoms and vaginal atrophy. See Physician’s
Desk Reference 3394 (65th ed. 2011).




      2
        Medroxyprogesterone acetate is a progestin. Dorland’s Illustrated Medical
Dictionary 1137 (31st ed. 2007). Progestin, also called progestogen, is a
progestational agent, meaning that it has effects similar to those of progesterone. Id.
at 39, 1546. Progesterone is “the principal progestational hormone of the body . . .
it prepares the uterus for the reception and development of the fertilized oocyte . . .
and maintains an optimal intrauterine environment for sustaining pregnancy.” Id. at
1546.

                                         -3-
        In the 1990s, the National Institutes of Health began studying the effects of
hormone therapy drugs as part of its Women’s Health Initiative (WHI).3 More than
160,000 postmenopausal women between fifty and seventy-nine years of age enrolled
in a set of clinical trials, including a randomized controlled trial of combined estrogen
and progestin (the WHI study). The WHI study included 16,608 participants and
assessed the risks and benefits of taking Prempro daily, compared to taking a placebo,
in the prevention of certain chronic diseases. Id. at 3400-01. The study was brought
to a premature conclusion, however, after the increased risk of breast cancer and
certain cardiovascular events exceeded the study’s specified benefits. Id. at 3401. As
reported in the Journal of the American Medical Association, the WHI study found
that the use of estrogen plus progestin increases the risk of breast cancer. Jacques E.
Rossouw et al., Risks and Benefits of Estrogen Plus Progestin in Healthy
Postmenopausal Women, 288 JAMA 321 (2002).

       Following the publication of the WHI study results, thousands of women filed
lawsuits against Wyeth and other hormone therapy drug manufacturers, alleging that
hormone therapy increased the risk of breast cancer and that the companies had failed
to warn of that risk. In 2003, the federal lawsuits were consolidated in the Eastern
District of Arkansas.

             B. Procedural Posture of Kuhn’s and Davidson’s Lawsuits

        In late October 2010, the district court selected Kuhn’s and Davidson’s cases
to go to trial. Earlier that month, the district court had directed the parties to provide
“a list of all ‘short-term use’ Plaintiffs who filed cases in either the Eastern District

      3
       “The Women’s Health Initiative (WHI) focuses on defining the risks and
benefits of strategies that could potentially reduce the incidence of heart disease,
breast and colorectal cancer, and fractures in postmenopausal women.” Jacques E.
Rossouw et al., Risks and Benefits of Estrogen Plus Progestin in Healthy
Postmenopausal Women, 288 JAMA 321, 321 (2002).

                                           -4-
or Western District of Arkansas.” Kuhn App. 49. The district court selected
Davidson from the list of plaintiffs who had used Prempro for three years or less. The
court also selected Kuhn, who had taken Prempro for slightly more than three years
and thus was not listed as a “short-term use” plaintiff.

      On October 29, 2010, Wyeth advised the court that a short-term use case was
proceeding to trial in the District of Puerto Rico and that it planned to file a Daubert
challenge to the general causation opinions of the plaintiff’s experts. Because Wyeth
would file similar challenges in both Puerto Rico and Arkansas, Wyeth suggested that
the two courts hear together the Daubert challenge. The courts agreed, and a joint
hearing was scheduled for November 29, 2010, in Puerto Rico.

       Prior to the hearing, Wyeth filed its motion to preclude expert testimony.4
Wyeth argued that there existed no reliable scientific basis for Plaintiffs’ experts to
conclude that taking Prempro for less than three years increases a woman’s risk of
developing breast cancer. Wyeth relied on the WHI study’s report that women who
took Prempro for three years or less had fewer incidents of breast cancer than those
who took the placebo. It argued that the medical and scientific communities had
accepted the WHI study as definitive and that the studies that Plaintiffs would rely
upon were methodologically flawed. Moreover, Wyeth cited evidence that the
experts Plaintiffs likely would call upon had admitted that Prempro does not increase
the risk of breast cancer when taken daily for three years or less.

      In opposition, Plaintiffs set forth their position that estrogen plus progestin
(E+P) is a “growth promoting agent” that causes cancer-susceptible cells to become


      4
       The motion to preclude was filed before case-specific discovery had been
completed and before the parties had designated their experts. Accordingly, Wyeth
did not challenge Plaintiffs’ expert’s opinions and methodologies, but rather tried to
anticipate their position.

                                          -5-
cancerous. Kuhn App. 168. According to Plaintiffs, the promotion effect can be seen
in a matter of months. Hormone therapy plaintiffs typically have relied on the WHI
study to show that use of Prempro causes an increased risk of breast cancer, but Kuhn
and Davidson argued that the study was not powerful5 enough to detect whether
short-term use of Prempro caused an increased risk. Moreover, Plaintiffs argued that
the WHI study failed to account for gap time6 and that subsequent analysis of the
study showed an increased risk of breast cancer after two years of Prempro use.
Plaintiffs cited a number of observational studies that found an increased risk of
breast cancer from E+P use of three years or less.

       In its reply, Wyeth argued that Plaintiffs’ evidence failed to establish a reliable
basis for concluding that short-term use of Prempro increases the risk of breast
cancer. Wyeth argued that Plaintiffs’ criticisms of the WHI study’s findings were
misguided, particularly because hormone therapy litigation experts had relied so
heavily on the study in the past and because the WHI study was a randomized control
study, the “gold standard” of epidemiological studies. Wyeth accused Plaintiffs of
“cherry picking” from the relevant observational studies, relying upon the ones that
showed an increased risk of breast cancer rather than the great weight of the studies


      5
         Statistical power is “the probability of rejecting the null hypothesis in a
statistical test when a particular alternative hypothesis happens to be true.” Merriam-
Webster Collegiate Dictionary 973 (11th ed. 2003). In other words, it is the
probability of observing false negatives. Power analysis can be used to calculate the
likelihood of accurately measuring a risk that manifests itself at a given frequency in
the general population based on the sample size used in a particular study. Such an
analysis is distinguishable from determining which study among several is the most
reliable for evaluating whether a correlative or even a causal relationship exists
between two variables.
      6
       Gap time is the time from menopause onset to treatment initiation. See Hr’g
Tr. 48. Throughout this opinion, Hr’g Tr. refers to the transcript of the January 12,
2011, Daubert hearing.

                                           -6-
that showed no increased risk. Moreover, Wyeth argued that the observational
studies cited by Plaintiffs were inapplicable because they evaluated hormone therapy
formulations other than Prempro and failed to reliably assess participants’ duration
of use. Wyeth also rebutted the promotion theory, noting that even experts for
hormone therapy litigation plaintiffs regarded the theory as insufficient to establish
causation.

                               C. Dr. Austin’s Opinion

      The day before the evidentiary hearing in Puerto Rico, Plaintiffs filed the
declaration of Dr. Austin, who opined that “E+P can, and often does cause clinically
detectable breast cancer in short-term durations including a matter of months.” Kuhn
App. 3905. Dr. Austin set forth in his declaration his standards for reviewing
observational studies, including that he would not rely on “underpowered” studies,
which he defined as studies that were not likely to identify an association or an effect,
if one existed.7 The declaration also listed the following studies as useful in
determining whether a short-term risk existed: The Million Women Study, the
French Teachers Study (Fournier), the Li (2000) study, and the Saxena (2010) study.

      Dr. Austin also opined that the WHI study’s estimate of short-term risk was
“quite poor” due to shortcomings “that diminish the estimate of the effect of short-
term exposure.” Kuhn App. 3902. He explained that the WHI study participants


      7
        Dr. Austin also opined that studies employing “ever used” as the measure of
exposure were unreliable because they included both women who were taking E+P
during the study and those who were not. He explained that because the risk for
women who had ceased using E+P would have returned to a baseline risk, those cases
were not suitable for assessing short-term risk. Dr. Austin also would not rely upon
studies that analyzed conjointly the risk of E+P and E-only because “the causal effect
of estrogen alone on the breast is considerably less than that of E+P.” Kuhn App.
3900.

                                          -7-
were more than ten years past the onset of menopause, meaning that they were much
older than the women who typically start hormone therapy. Moreover, the study
tended to exclude women who were experiencing moderate hot flashes. According
to Dr. Austin, those women were more likely to be estrogen deficient and more likely
to be susceptible to the carcinogenic effects of E+P. Dr. Austin also explained that
the WHI study’s analysis necessarily underestimates the relative risk because
approximately forty percent of the participants dropped out of the study and about
eleven percent of the placebo group began taking E+P.8 The study’s first report was
thus flawed, Dr. Austin opined, because “women were counted as E+P users who
actually were not and vice versa.” Id. Dr. Austin also listed other shortcomings that
led the WHI study to underestimate short-term risk, including that E+P makes breast
cancer detection more difficult and that the study included hormone-negative tumors
in its results.

     D. Daubert Hearings and Order Excluding Dr. Austin’s Expert Opinion

       Given that Dr. Austin’s declaration was filed on November 28, 2010, it was not
fully considered at the November 29, 2010, hearing in Puerto Rico. Because no
witnesses were called to testify, the hearing was limited to counsels’ arguments.

       Shortly after the hearing, the district court asked Plaintiffs and Wyeth to
address the WHI study’s finding that breast cancer incidents were lower among the
participants taking Prempro than those who were taking the placebo during the first
three years. Specifically, the court asked, “With respect to short term use, please give
me a short, pointed, pithy brief explaining what evidence of your’s meets the WHI
findings on this point.” Kuhn App. 3912. Plaintiffs reiterated the arguments they had
made in opposition to Wyeth’s motion to preclude, including that the WHI study was


      8
        During the first three years of the study, twenty-three percent of the
participants dropped out.

                                          -8-
not powerful enough to detect a short-term risk, that it was not designed to do so, and
that other studies established a short-term effect. After reviewing the parties’
submissions, the district court ordered a second Daubert hearing and called for live
testimony from the parties’ experts.

      On January 12, 2011, the magistrate judge held a lengthy Daubert hearing. Dr.
Austin and Wyeth’s expert, Kurt Barnhart, M.D., testified, fielding questions from the
attorneys and the court. During the hearing, Dr. Austin conceded that the Li (2000)
and Saxena (2010) studies should not have been included in his expert report. Dr.
Austin thus based his opinion that short-term use of Prempro causes breast cancer on
the Million Women Study, the Fournier study, and the American Cancer Society
Study (Calle study).9

       The magistrate judge issued an order the following week, granting Wyeth’s
motion to preclude expert testimony.10 The order stated, “The issue before the Court
is whether Dr. Austin, an epidemiologist and Plaintiffs’ general causation expert, can
testify reliably that short-term use of Prempro increases the risk of breast cancer.”
Order of January 19, 2011, at 1 (footnote omitted). The magistrate judge ultimately

      9
        Although the Calle study was not cited in Dr. Austin’s expert report, it was
cited in Plaintiffs’ opposition to Wyeth’s motion to exclude, and Dr. Austin testified
regarding the Calle study during the January 2011 Daubert hearing. When quoting
material from the articles describing these studies, we will cite Kuhn’s Exhibit
Appendix, hereinafter Ex. App. Those articles include the following: Agnes
Fournier et al., Estrogen-Progestagen Menopausal Hormone Therapy and Breast
Cancer: Does Delay From Menopause Onset to Treatment Initiation Influence
Risks?, J. Clinical Oncology (published online ahead of print on Sept. 14, 2009);
Eugenia E. Calle et al., Postmenopausal Hormone Use and Breast Cancer
Associations Differ by Hormone Regimen and Histologic Subtype, Cancer 936 (Mar.
1, 2009); Breast cancer and hormone-replacement therapy in the Million Women
Study, 362 Lancet 419 (Aug. 9, 2003).
      10
           Identical orders were filed in both Kuhn’s and Davidson’s cases.

                                          -9-
excluded the expert evidence because it found that Dr. Austin failed to discredit the
WHI study’s results and failed to base his opinion on epidemiological studies that
“reliably support[ed] his position.” Id. at 21.11 The district court denied Plaintiffs’
objections to the order and their motion to reconsider and later granted Wyeth’s
motion for summary judgment.

                                     II. Discussion

                                  A. Expert Evidence

        We review the decision to exclude expert evidence for an abuse of discretion.
Gen. Elec. Co. v. Joiner, 522 U.S. 136, 143 (1997); Junk v. Terminix Int’l Co., 628
F.3d 439, 447 (8th Cir. 2010). The opinion of a qualified expert witness is admissible
if (1) it is based on sufficient facts or data, (2) it is the product of reliable principles
and methods, and (3) the expert has reliably applied the principles and methods to the
facts of the case. Fed. R. Evid. 702. The expert’s scientific, technical, or specialized
knowledge must also “assist the trier of fact to understand the evidence or determine
a fact in issue.” Id. The district court is thus vested with a gatekeeping function,
ensuring that “any and all scientific testimony or evidence admitted is not only
relevant, but reliable.” Daubert, 509 U.S. at 589. The function also serves “to make
certain that an expert, whether basing testimony upon professional studies or personal
experience, employs in the courtroom the same level of intellectual rigor that
characterizes the practice of an expert in the relevant field.” Kumho Tire Co., Ltd.
v. Carmichael, 526 U.S. 137, 152 (1999).

       “[T]he requirement that an expert’s testimony pertain to ‘scientific knowledge’
establishes a standard of evidentiary reliability.” Daubert, 509 U.S. at 590. The


       11
         Wyeth did not challenge Dr. Austin’s qualifications or the general practice
of relying on epidemiological studies to support an expert opinion.

                                           -10-
Supreme Court explained that evidentiary reliability means trustworthiness. Id. at
591 n.9. “Proposed testimony must be supported by appropriate validation—i.e.,
‘good grounds,’ based on what is known.” Id. at 590. The standard for judging the
evidentiary reliability of expert evidence is “lower than the merits standard of
correctness.” In re Paoli R.R. Yard PCB Litig., 35 F.3d 717, 744 (3d Cir. 1994).
Proponents of expert testimony need not demonstrate that the assessments of their
experts are correct, and trial courts are not empowered “to determine which of several
competing scientific theories has the best provenance.” Milward v. Acuity Specialty
Prods. Grp., 639 F.3d 11, 15 (1st Cir. 2011) (quoting Ruiz-Troche v. Pepsi Cola of
Puerto Rico Bottling Co., 161 F.3d 77, 83 (1998)); see Fed. R. Evid. 702 advisory
committee’s note (discussing 2000 amendments) (“When a trial court, applying this
amendment, rules that an expert’s testimony is reliable, this does not necessarily mean
that contradictory expert testimony is unreliable.”). “Vigorous cross-examination,
presentation of contrary evidence, and careful instruction on the burden of proof are
the traditional and appropriate means of attacking shaky but admissible evidence.”
Daubert, 509 U.S. at 596.

        The Supreme Court identified in Daubert a number of factors that might assist
the district court in determining the admissibility of expert evidence: (1) whether the
theory or technique applied can be tested, (2) whether the theory or technique has
been subject to peer review or publication, (3) the known or potential rate of error,
and (4) whether it is accepted in the relevant discipline. Id. at 593-94. It instructed
district courts to focus on “principles and methodology, not on the conclusions that
they generate.” Id. at 595. The Court later recognized that “conclusions and
methodology are not entirely distinct from one another.” Joiner, 522 U.S. at 146.
Accordingly, a district court’s focus on principles and methodology “need not
completely pretermit judicial consideration of an expert’s conclusions.” Milward,
639 F.3d at 15 (quoting Ruiz-Troche, 161 F.3d at 81). Expert evidence may be
excluded if the court determines “that there is simply too great an analytical gap
between the data and the opinion proffered.” Joiner, 522 U.S. at 146.

                                         -11-
                                   1. WHI Study

       Plaintiffs involved in hormone therapy litigation have long relied on the WHI
study to prove general causation: that is, that use of Prempro increases the risk of
breast cancer. A 2003 article about the WHI study reported:

      For women with no menopausal hormone use before entering the study,
      invasive breast cancer rates were lower for the initial 2 years in the
      estrogen plus progestin group compared with placebo, and similar in the
      third year. In the fourth year and thereafter, invasive breast cancer rates
      were higher in the estrogen plus progestin group . . . .

Ex. App. 55.12

       The magistrate judge found that Dr. Austin failed to meet his burden “to
present reliable science to support his conclusion regarding the unreliability of the
WHI.” Order of Jan. 19, 2011, at 9. Plaintiffs, as the proponents of Dr. Austin’s
testimony, however, do not necessarily have a burden to disprove the WHI study’s
finding that short-term use of Prempro does not increase the risk of breast cancer.
Instead, it is their burden to show that Dr. Austin arrived at his contrary opinion in
a scientifically sound and methodological fashion. If they can meet that burden, the
question becomes one for the jury to decide. See Bonner v. ISP Techs., Inc., 259 F.3d
924, 930 (8th Cir. 2001) (“[Q]uestions of conflicting evidence must be left for the
jury’s determination.”).

      As an initial matter, we note that the magistrate judge did not address Dr.
Austin’s testimony that women who took Prempro had “[l]ower recorded rates of


      12
        Rowan T. Chlebowski et al., Influence of Estrogen Plus Progestin on Breast
Cancer and Mammography in Healthy Postmenopausal Women, 289 JAMA 3243
(2003).

                                         -12-
breast cancer in the first two years, not three years.” Hr’g Tr. 152. Dr. Austin
distinguished between annual incidents of breast cancer and cumulative incidents of
breast cancer, acknowledging that a graph contained in a 2006 article showed that the
placebo group had higher rates of breast cancer during the first three years of the trial.
See Ex. App. 19.13 Dr. Austin explained that the graph measured cumulative
incidents, not annual incidents, and that it did not plot individual occurrences:

      Now, it is true that if you look at the number of cases occurring in a
      year, not cumulative starting from the beginning, but how many in year
      one, there were more placebo cases; year two, more placebo cases; year
      three, there were more Prempro cases. So by year three, the Prempro
      group had caught up and even had surpassed. But if you look at all three
      years, they were still behind because they hadn’t made up for the deficits
      in year one and two.

Hr’g Tr. 154. The magistrate judge’s opinion addresses only Dr. Austin’s testimony
that the cumulative incidents showed no increase in the breast cancer risk in the first
three years. When asked at oral argument before this court whether the magistrate
judge had made a mistake in interpreting the graph, Wyeth’s counsel responded that
he had not because the magistrate judge was interpreting Dr. Austin’s testimony, not
the underlying evidence, and because Dr. Austin previously had testified that the
WHI study shows no increased risk when Prempro is taken for three years or less. Dr.
Austin’s hearing testimony, however, distinguished between the cumulative and
annual incidents, and any conflict with his previous testimony, addressed more fully
below, presents issues of credibility.

      We conclude that the magistrate judge abused his discretion in deciding that
Dr. Austin’s criticisms of the WHI study were unfounded and inconsistent with his

      13
        Garnet L. Anderson et al., Prior hormone therapy and breast cancer risk in
the Women’s Health Initiative randomized trial of estrogen plus progestin, Maturitas
(2006).

                                          -13-
reliance on the study in other hormone therapy cases. When asked by the court how
the WHI could be the “gold standard” to show that E+P is a carcinogen but could not
be relied upon to show that its short-term use does not increase the risk of breast
cancer, Dr. Austin replied that the WHI study was an ideal study design—“the gold
standard for what it was designed for”—but that it was designed to show what effect
E+P had on heart disease. Hr’g Tr. 69. He explained that although the study
monitored incidents of breast cancer, the women were not selected to test whether
Prempro causes breast cancer. According to Dr. Austin, this resulted in a study
population that was at a much lower risk because (1) the study discouraged the
participation of women with moderate menopausal symptoms and (2) the participants
had a longer gap time between menopause and beginning hormone therapy treatment
than women who begin hormone therapy on their own volition. Dr. Austin went on
to explain that, despite its shortcomings, the study showed that E+P causes breast
cancer:

      Now, with all of these disadvantages, it still showed that after five years,
      the E plus P group had a statistically significantly higher risk, and so
      high that they had to stop the study. So with all those things against it,
      it still demonstrated cause, even the way they analyzed it, the fact they
      didn’t use gap time, the fact that they picked women that are not really
      representative of the general population who use E plus P. In spite of all
      those things, they still found it was a cause and demonstrated it was a
      cause.

Id. at 69-70. In light of this testimony and its supporting evidence, Dr. Austin’s
reliance on the WHI study to prove general causation does not foreclose his opinion
that the study did not accurately assess the risk of breast cancer associated with the
short-term use of Prempro.

      The magistrate judge also found that Dr. Austin previously had contradicted
his opinion that the WHI study lacked power to detect an increased risk of breast


                                         -14-
cancer from short-term use of Prempro. The court noted that “previously, Dr. Austin
agreed that WHI, as a clinical trial, had the greatest power when assessing HRT
[hormone therapy] and women’s health.” Order of Jan. 19, 2011, at 7. Dr. Austin
now offers a more nuanced opinion than he has in the past, setting forth the
foundation for his present concerns, for as he explained,

      Well, that’s because there’s two separate concepts. One is, is it a cause
      or not? And the other one is, what is the risk estimate? And
      observational studies have different strengths and weaknesses on those
      two points. They’re not as good for demonstrating cause and effect, but
      they’re much better at estimating the size of the risk. So different study
      designs are used to try to do different things, and in this [WHI], even
      though it wasn’t designed to see what the effect on breast cancer was, it
      did, and the causal nature was demonstrated and it’s irrefutable.

Hr’g Tr. 71-72.

       Wyeth will certainly challenge Dr. Austin’s credibility based on his previous
testimony, including his agreement that the WHI study would be a “good study to
look at in determining whether a certain duration of exposure to Prempro increases
the risk of breast cancer.” Id. at 145. But his previous reliance on and testimony
regarding the WHI study does not render his opinion inadmissible, as it was not his
burden to disprove the WHI’s finding that short-term use of Prempro does not
increase the risk of breast cancer.

      Finally, in determining that Dr. Austin failed to support his criticisms of the
WHI study, the magistrate judge found that Dr. Austin’s opinions were “hastily
formed in an attempt to overcome overwhelmingly reliable evidence to the contrary.”
Order of January 19, 2011, at 9. Dr. Austin testified that the short-term use issue was
presented to him shortly before the Daubert hearing in Puerto Rico. He wrote the
expert report in approximately five hours, with the assistance of Plaintiffs’ counsel.


                                         -15-
The magistrate judge’s concerns about Dr. Austin’s opinions, however, go to his
credibility as a witness, not to the admissibility of his testimony as an expert. Wyeth
argues that, like the expert testimony in In re TMI Litigation, 193 F.3d 613 (3d Cir.
1999), Dr. Austin’s testimony should be excluded based on his reliance on Plaintiffs’
counsel. In that case, however, the expert relied on medical history summaries
prepared at the direction of plaintiffs’ counsel, which the court found unreliable. Id.
at 697-98. Here, Dr. Austin relied on epidemiological studies that, at most, were
compiled for him by Plaintiffs’ counsel. Unlike the medical history summaries, the
supporting observational studies provide adequate foundation for Dr. Austin’s
testimony.

                        2. Supporting Observational Studies

       Dr. Austin ultimately relied on three observational studies to support his
opinion: the Calle study, the Million Women Study, and the Fournier study. The
magistrate judge analyzed each study separately and concluded that the studies failed
to reliably support Dr. Austin’s position. On appeal, Kuhn and Davidson argue that
the magistrate judge usurped the role of factfinder and that any problems with the
studies go to the weight of the evidence, not to the studies’ validity. Wyeth responds
that the magistrate judge properly determined that the studies were insufficient to
support Dr. Austin’s opinion because there existed too great an analytical gap
between the underlying studies and Dr. Austin’s opinion, see Joiner, 522 U.S. at 146-
47, and because in relying on the studies, he failed to employ the rigorous methods
that he uses in other contexts, see Kumho Tire Co., 526 U.S. at 152.

                                   a. Calle Study

       Dr. Austin relied on the Calle study, a study of 67,754 American women that
“examined prospectively the relation between postmenopausal hormone use and the
risk of incident of breast cancer . . . .” Ex. App 32. When the participants enrolled

                                         -16-
in 1992, they completed self-administered mailed questionnaires that collected
information regarding demographic, medical, behavioral, environmental,
occupational, and dietary factors. Follow-up questionnaires were sent in 1997, 1999,
2001, and 2003 “to update exposure information and to ascertain newly diagnosed
cancers.” Id. According to the authors, “[i]nformation regarding postmenopausal
hormone use was collected at the time of enrollment and on all follow-up
questionnaires.” Id. at 33. Women with unknown type or duration of hormone use
were excluded from the analysis. The authors observed no increase in risk with use
of E+P in the first two years of use. “However, we observed a significant increase in
risk for both types of breast cancer at 2 to 3 years of use . . . and all years thereafter.”
Id. at 37.

       The magistrate judge found the study unreliable because “there was no way to
measure how it accounted for previous use [of hormone therapy]” and because it
failed to meet Dr. Austin’s criteria to characterize exposure “carefully and
accurately.” Order of Jan. 19, 2011, at 17. Plaintiffs argue that the magistrate judge
erred in finding that the study failed to monitor exposure and duration of use. We
agree. As set forth above, the study explains that women with unknown type or
duration of hormone use were excluded from the analysis and that the authors
gathered information regarding hormone therapy use at the time of enrollment and
again in 1997, 1999, 2001, and 2003.14 Consistent with the study, Dr. Austin testified
that he believed that information regarding breast cancer occurrence and duration of
hormone therapy use was collected in each follow-up questionnaire.



      14
         The participants returned self-administered questionnaires to report their
hormone use, and Dr. Austin testified that self-reporting lends itself to some error.
We note that the WHI study assessed some breast cancer risk factors and initial
reports of cancer outcomes by self-administered questionnaires. Ex. App. 53. Based
on the reliance on the WHI study by all the parties in this case, we conclude that self-
reporting alone does not render a study unreliable in this context.

                                           -17-
                                 b. Foreign Studies

       Dr. Austin also relied on two foreign studies: the Million Women Study from
England and the Fournier study from France. “The Million Women Study was set up
to investigate the effects of specific types of [hormone therapy] on incident and fatal
breast cancer.” Ex. App. 165. As its name suggests, the study involved more than
one million women between fifty and sixty-four years of age, including more than
20,000 participants who took medroxyprogesterone acetate combined with
conjugated equine estrogen, the same formulation as Prempro. It found that
combined estrogen and progestin therapy increased the risk of breast cancer and that
“[r]esults varied little between specific oestrogens and progestagens or their doses;
or between continuous and sequential regimens.” Id. Specifically, the study reported:

      Other than the substantial difference between the effects of oestrogen-
      only and oestrogen-progestagen combinations, these results suggest no
      large variations between the effects of specific oestrogens (equine
      oestrogen and oestradiol) or between specific progestagens
      (medroxyprogesterone acetate, norgestrel, and norethisterone). They
      also suggest that results on the risk of breast cancer for the specific
      constituents used in the Women’s Health Initiative trial do not differ
      materially from the results for other oestrogen-progestagen
      combinations.

Id. at 172. Upon joining the Million Women Study, the participants reported
information regarding their hormone therapy use including their “ever use; current
use; age at first and last use; total duration of use; and the name of the proprietary
preparation used most recently and duration of its use.” Id. at 165. The participants
thus reported their duration of use when they enrolled, but the study did not measure
their use after enrollment. Instead, it found that “[t]he breast cancers were diagnosed
on average 1.2 years after recruitment” to the study. Id. at 167.




                                         -18-
       A table in the Million Women Study shows an elevated relative risk of breast
cancer among women who reported using estrogen plus progestagen for less than one
year. Because the study measured duration of use at enrollment rather than diagnosis,
the precise duration of use by those women diagnosed with breast cancer was
unknown. Dr. Austin suggested that by adding the average 1.2 years from enrollment
to diagnosis to the less than one year of use, the study supported his short-term use
opinion because it showed an elevated risk in less than 2.2 years. He testified that,
“we have very large numbers here, and small amounts of misclassifications don’t
really make a lot of difference.” Hr’g. Tr. 27.

       Dr. Austin also relied upon the Fournier study, which investigated “whether the
relation between estrogen-progestagen menopausal hormone therapy (EP-MHT) and
breast cancer risk varies according to the delay between menopause onset and
treatment initiation.” Ex. App. 83. More than 50,000 postmenopausal French women
participated in the study. The results suggested that “in recent users of some EP-
MHT, the timing of treatment initiation modulates the risk of breast cancer: short
durations (#2 years) of use were associated with significant increases in risk—with
the exception of MHT containing progesterone—when initiated in the 3-year period
following menopause, but not when initiated later.”15 Id. at 86-87. In other words,
as Dr. Austin explained, the study showed an increased risk of breast cancer in
women taking hormone therapy for less than two years when the women began
hormone therapy shortly after the onset of menopause.

      French women typically use a different type of hormone therapy than American
women. In the Fournier study’s cohort, “the estrogenic component consisted almost
exclusively of estradiol compounds, with progesterone and dydrogesterone as the


      15
       The Fournier study observed that using progesterone may be safer than other
progestagens, but that the finding “needs to be confirmed in other settings.” Ex. App.
88.

                                        -19-
most frequently associated progestagens.” Id. at 87. Although relatively few
participants took formulations comprised of conjugated estrogens combined with
medroxyprogesterone acetate, like Prempro, more than one hundred participants using
hormone therapy comprised of progestagens were diagnosed with breast cancer.

      With more than 100 participants diagnosed with breast cancer among
      those using EP-MHT (containing progestagens other than progesterone
      or dydrogesterone) for 2 years or less, we found significant doubling in
      risk when the MHT began within 3 years of menopause compared with
      MHT never use. For the other durations of use, estimates of the WHI
      and ours are consistent.

      Our results of breast cancer risks increased even with relatively short
      durations of some EP-MHT are consistent with epidemiological studies
      that found significant increases in risk for current or recent short-term
      (<5 years) use of EP-MHT, although some did not find significant
      increases.

Id. In an editorial commenting on the Fournier study, Leslie Bernstein, M.D., thus
wrote, “[I]t will be important for other cohorts that focus on use of HT in United
States populations to evaluate breast cancer risk associated use immediately after
menopause.” Id. at 28.16

        Dr. Austin acknowledged that the Fournier study did not separate Prempro’s
formulation from other hormone therapy formulations. He testified that
medroxyprogesterone acetate is combined only with conjugated estrogens and that
its risk estimates are similar to those of other of synthetic progesterones that are
combined with other estrogens. “Since we look at them separately and they’re
similar, I think no violence is done to the analysis by combining them.” Hr’g Tr. 119.

      16
        Leslie Bernstein, Combined Hormone Therapy at Menopause and Breast
Cancer: A Warning—Short-Term Use Increases Risk, J. Clinical Oncology
(published online ahead of print on Sept. 14, 2009).

                                        -20-
On cross-examination, however, Dr. Austin was confronted with an expert report he
authored in 2007, wherein he opined that estradiol and conjugated equine estrogen
should be considered separately “to determine if they have statistically different long-
term effects.” Id. at 117. When asked if that same analysis should be done to
determine the short-term effects, Dr. Austin responded, “It’s the most ideal, yes. . . .
Fournier did not do that.” Id. Dr. Austin also acknowledged that the Fournier study
provided no separate analysis of Prempro formulation at three years or less.

       The magistrate judge concluded that the foreign studies did not reliably support
Dr. Austin’s opinion because most participants in the studies took hormone therapy
formulations other than Prempro and because the studies did not analyze Prempro
separately from other hormone therapy formulations after three years.17 With respect
to the Million Women Study, the magistrate judge determined that the study’s failure
to accurately assess years of use rendered the study unreliable. The court found Dr.
Austin’s suggestion to add the “average underestimate of 1.2 years” use irreconcilable
with “his position that when looking at short-term use, one must be quite precise in
the analysis.” Order of Jan. 19, 2011, at 16.

      Although the foreign studies have important limitations, they nonetheless
provide support for Dr. Austin’s opinion. As set forth above, the Million Women
Study explained that there exists no material difference in risk among the combined
estrogen and progestin hormone therapy formulations, including Prempro. With this
finding, and its finding that less than one year’s use of combined hormone therapy
increases the risk of breast cancer, the Million Women Study provides support for Dr.
Austin’s opinion regarding short-term use. Likewise, the differences between

      17
         The Million Women Study considered separately participants’ use of Prempro
for five years or less and found that the relative risk of breast cancer was 1.62. The
magistrate judge found this analysis unhelpful because “the issue before the Court
involves usages of around three years or fewer; the MWS did not separately assess
Prempro for around three years or fewer.” Order of Jan. 19, 2011, at 15.

                                         -21-
Prempro and the French combined hormone therapy formulation do not render the
study unreliable to support Dr. Austin’s opinion.

        Citing Glastetter v. Novartis Pharmaceuticals Corp., 252 F.3d 986 (8th Cir.
2001) (per curiam), Wyeth argues that Dr. Austin’s testimony relating the European
formulations to the American formulations constitutes an unsupported assumption.
In Glastetter, the plaintiff’s experts hypothesized that the chemical at issue,
bromocriptine, “may behave like its chemical cousins,” ergot alkaloids, which were
known to cause a condition that was a known risk factor for the plaintiff’s injury. Id.
at 989-90. We concluded that the “generic assumption that bromocriptine behaves
like other ergot alkaloids carries little scientific value.” Id. at 990. Here, Dr. Austin
presented evidence that the Million Women Study considered estrogen plus progestin
hormone therapy formulations separately and found little difference between the
effects of the different formulations.18 His testimony was thus not a “generic
assumption,” but rather was supported by the study’s findings.

      Along with the different formulations, we recognize that the Million Women
Study’s computation of duration of use is imprecise and that the “average
underestimate of 1.2 years” is disputed. Like the formulation differences, the
duration issues, however, do not create so great an analytical gap between the data
and the opinion as to render the opinion inadmissible.

       As set forth above, the magistrate judge relied on Dr. Austin’s testimony in
other cases and his earlier statements to find that the foreign studies did not reliably
support his opinion. Specifically, the court found that the studies failed to meet Dr.


      18
        We note that the magistrate judge did not address the study’s finding that
results varied little between specific estrogen plus progestin hormone therapy
formulations.


                                          -22-
Austin’s criteria of (1) accurately characterizing exposure to the drug, (2) identifying
the specific drug formulations, and (3) analyzing the Prempro formulation separately.
At the hearing, Dr. Austin acknowledged that, ideally, a study would measure
exposure precisely and analyze the Prempro formulation separately, but he did not
concede that the studies on which he based his opinion were unreliable. Instead, he
explained,

      [The foreign studies] give us information that other studies don’t
      because of their large number, and the fact that, like in this one, where
      they were able to eliminate the gap time, they are imperfect in that they
      mix together drugs of similar types so that it loses its specificity. But it
      does gain in the fact that they have large numbers for the things that
      we’re interested in with short-term use. So although there’s no study
      that’s perfect that I’m aware of for answering this specific question, the
      fact that these two studies are consistent in their findings is important.

Hr’g Tr. 51-52. Although the foreign studies are not perfect, they provide useful
information that, along with the Calle study, provides an adequate foundation for Dr.
Austin’s opinion. His prior testimony may well call his credibility into question, but
it does not prove that he failed to follow his own general practice or that he relied on
unfounded assumptions in his analysis. Cf. Junk, 628 F.3d at 448 (holding that the
expert’s “failure to follow his own general practice and his reliance on unfounded
assumptions in his comparative method created ‘too great an analytical gap’ between
his opinion and the data on which it relied” (quoting Joiner, 522 U.S. at 146);
Fireman’s Fund Ins. Co. v. Canon U.S.A., Inc., 394 F.3d 1054, 1059 (8th Cir. 2005)
(noting that the sudden reversal of opinion regarding the meaning of burn pattern
evidence “seriously undermine[d] the reliability of the experts’ opinions”).




                                         -23-
 c. Supporting Observational Studies Sufficient to Support Dr. Austin’s Opinion

       Dr. Austin’s testimony is admissible because the studies upon which he relied
were sufficient to support his opinion that short-term use of Prempro increases the
risk of breast cancer. Taken together, the Calle study and the foreign studies
constitute appropriate validation of and good grounds for Dr. Austin’s opinion. The
studies’ limitations may be presented to the jury, and Dr. Austin’s reliance on the
studies may be tested through the traditional means of cross examination and
presentation of contrary evidence.19

                      3. Wyeth’s “Cherry Picking” Allegation

       Wyeth argues that Dr. Austin “cherry picked” the Calle study and the two
foreign studies from a wealth of studies showing no increased risk of breast cancer
from short-term Prempro use. Wyeth compares this case to Norris v. Baxter
Healthcare Corp., 397 F.3d 878 (10th Cir. 2005), in which the defendant presented
at least seventeen epidemiological studies in support of its contention that silicone
breast implants do not cause disease. The plaintiffs in Norris offered no
epidemiological studies to support their contrary position, instead relying upon
differential diagnosis and case studies. The Norris court concluded that “[p]laintiff’s
experts’ differential diagnoses and case studies are unreliable because they assume
what science has largely shown does not exist-a causal connection between silicone


      19
        In his expert report, Dr. Austin opined that E+P is “both a promoter and a
growth stimulant, meaning its effect is both to cause new cancers to occur, and to
cause [estrogen receptive] tumors to grow more quickly.” Kuhn App. 3904.
Accordingly, he opined that the effect of E+P on cell proliferation is “almost
immediate.” Id. Because we conclude that Dr. Austin’s opinion is adequately
supported by epidemiological studies, we do not address Plaintiffs’ argument that
“basic biology confirms the causal associations shown by several epidemiological
studies.” Kuhn Br. 70.

                                         -24-
breast implants and disease.” Id. at 886. Unlike those experts, Dr. Austin has
presented reliable epidemiological evidence to support his opinion that short-term use
of Prempro increases the risk of breast cancer. There may be several studies
supporting Wyeth’s contrary position, but it is not the province of the court to choose
between the competing theories when both are supported by reliable scientific
evidence.

                               B. Summary Judgment

      The grant of summary judgment in favor of Wyeth was based on the preclusion
of Dr. Austin’s opinion. Because we hold Dr. Austin’s opinion is admissible, we
reverse the order granting summary judgment.

                                   III. Conclusion

       We reverse the orders granting Wyeth’s motions to preclude expert testimony
and for summary judgment. Because our reversal is based on the conclusion that Dr.
Austin’s opinion is admissible, we do not reach the merits of Kuhn’s argument that
the Daubert order did not apply to her because she had used Prempro for more than
three years. The case is remanded for further proceedings.

LOKEN, Circuit Judge, dissenting.

       Early in its lengthy analysis, the district court described the “methodology” of
plaintiffs’ expert, Dr. Donald Austin:

      according to Dr. Austin, his short-term use report was written in about
      five hours at the behest and assistance of Plaintiffs’ counsel. Dr. Austin
      testified that he had never really thought about the short-term use issue
      before Plaintiffs’ counsel presented it to him shortly before the recent
      Daubert challenge in Puerto Rico. He testified, “[i]t had never been

                                         -25-
      raised as an issue to me, is there an important, a biologically important
      and clinically important risk within three years. It was just not an issue
      that I had thought about.”

After thoroughly reviewing the bases for Dr. Austin’s opinion in an analysis the court
now finds statistically faulty in some respects, the district court concluded:

             Dr. Austin contends that short-term use of Prempro causes breast
      cancer, and cites five studies to support his conclusions. However, he
      conceded the following: two of the studies he admitted should not have
      been included in his report (Li (2000) and Saxena (2010)); two of the
      studies primarily involved drug combinations that were not Prempro
      (MWS [Million Women Study], French Teachers Study [Fournier]); and
      one of the studies did not reliably track duration of use, which is
      essential when making a short-term use causation finding (Calle (2009)).
      With no studies to reliably support his position, along with a failed
      effort to discredit WHI results, Dr. Austin’s opinion on short-term use
      causation [is] not sufficiently reliable to be admissible under Daubert.

       Like the district court in General Electric Co. v. Joiner, 522 U.S. 136 (1997),
the district court excluded opinion testimony of plaintiffs’ only expert. In my view,
the district court’s Daubert analysis is supported by the record and properly focused
on the gate-keeping function mandated by the Supreme Court and by Rule 702. We
review that ruling for abuse of discretion and must affirm unless it is “manifestly
erroneous.” Joiner, 522 U.S. at 142. We may not subject the ruling to “a more
searching standard of review” because it was “‘outcome determinative.’” Id. at 142-
43. I conclude that, like our sister circuit the Supreme Court reversed in Joiner, the
court has “appl[ied] an overly ‘stringent’ review to that ruling [and] failed to give the
trial court the deference that is the hallmark of abuse of discretion review.” Id. at
143. Accordingly, I respectfully dissent.
                         ______________________________



                                          -26-
