                  FOR PUBLICATION
  UNITED STATES COURT OF APPEALS
       FOR THE NINTH CIRCUIT

THE LANDS COUNCIL, a Washington        
nonprofit corporation; KOOTENAI
ENVIRONMENTAL ALLIANCE, an
Idaho nonprofit corporation; THE
ECOLOGY CENTER, a Montana
nonprofit corporation; IDAHO                No. 03-35640
SPORTING CONGRESS, INC., an Idaho             D.C. No.
nonprofit corporation,
              Plaintiffs-Appellants,      CV-02-00517-EJL
                                            ORDER AND
                v.                           AMENDED
BRADLEY POWELL, Regional                      OPINION
Forester of Region One of the
U.S. Forest Service; UNITED
STATES FOREST SERVICE, an agency
of the United States,
             Defendants-Appellees.
                                       
        Appeal from the United States District Court
                  for the District of Idaho
         Edward J. Lodge, District Judge, Presiding

                  Argued and Submitted
           March 30, 2004—Seattle, Washington

                  Filed August 13, 2004
                 Amended January 24, 2005

   Before: William C. Canby, Kim McLane Wardlaw, and
            Ronald M. Gould, Circuit Judges.

                  Opinion by Judge Gould


                            1015
1018             LANDS COUNCIL v. POWELL


                       COUNSEL

Thomas J. Woodbury, Forest Defense P.C., Missoula, Mon-
tana, for the plaintiffs-appellants.

Ronald M. Spritzer, Environmental and Natural Resources
Division, United States Department of Justice, Washington,
D.C., for the defendants-appellees.
                   LANDS COUNCIL v. POWELL                   1019
                           ORDER

  The opinion filed on August 13, 2004, and published at 379
F.3d 738, is AMENDED as follows:

   The final sentence in the first paragraph in section III.B.2.
states:

    Because the Final Environmental Impact Statement
    must include cumulative effects discussion for “rea-
    sonably foreseeable projects,” 40 C.F.R. § 1508.7,
    we must determine whether these two potential proj-
    ects are “reasonably foreseeable.”

   The final sentence in the first paragraph in section III.B.2.
is deleted in its entirety and replaced with the following lan-
guage:

    Because the Final Environmental Impact Statement
    must include cumulative effects discussion for “rea-
    sonably foreseeable projects,” 40 C.F.R. § 1508.7,
    the Lands Council asks us to determine whether
    these two potential projects are “reasonably foresee-
    able.”

  The final two paragraphs in section III.B.2. state:

       Our precedent defines “reasonably foreseeable” in
    this context to include only “proposed actions.” Or.
    Natural Res. Council v. Marsh, 832 F.2d 1489, 1498
    (9th Cir. 1987), rev’d on other grounds, 490 U.S.
    360 (1989). This rule makes sense: The agency is
    required to analyze the cumulative effects of projects
    that it is already proposing. For any project that is
    not yet proposed, and is more remote in time, how-
    ever, a cumulative effects analysis would be both
    speculative and premature. By contrast, any future
    project, once proposed, becomes more concrete and
1020              LANDS COUNCIL v. POWELL
    less speculative, and thus, would be subject to
    NEPA’s cumulative effects analysis. Further, at the
    time it was proposed, if it is a major federal action,
    its NEPA assessment would be obligated to include
    all past projects in the cumulative effects analysis.

       Here, of the future projects that the Lands Council
    urges were improperly ignored, none appears to have
    been proposed or scoped at the date of the Final
    Environmental Impact Statement, with one excep-
    tion. Under these circumstances, the Forest Service
    acted within its discretion when it did not analyze
    the projects cited by the Lands Council.

  Footnote 8 states:

    Scoping occurs after there is a proposed project. 40
    C.F.R. § 1501.7. Although the Administrative
    Record is not clear on this matter, the Deerfoot
    Ridge Restoration Project could have been proposed
    at the time of the Final Environmental Impact State-
    ment and Record of Decision because scoping
    occurred so soon after the Record of Decision in this
    case. If such is the case, then the Deerfoot Ridge
    Restoration Project should have been included as a
    reasonably foreseeable activity.

   The final two paragraphs in section III.B.2. and footnote 8
are deleted in their entirety and replaced with the following
language:

       We need not address these issues, however,
    because: (1) the parties have agreed that the Deerfoot
    Ridge Restoration Project was in a different drainage
    and thus need not have been considered, and (2) the
    Lands Council has represented that a determination
    that the IPNF Inland Native Fish Strategy applies —
    a decision that we reach later in this opinion —
                   LANDS COUNCIL v. POWELL                     1021
    moots the need to address the Geographic Assess-
    ment issue here.

   The final sentence in the first paragraph in section IV.A.
states:

    The Forest Service concedes that it did not analyze
    the Project under the fry emergence standard, but
    argues that it did have to do so.

  The final sentence in the first paragraph in section IV.A. is
deleted in its entirety and replaced with the following lan-
guage:

    The Forest Service concedes that it did not analyze
    the Project under the fry emergence standard, but
    argues that it did not have to do so.

   The final sentence in the third paragraph in section IV.A.
states:

    The Lands Council argue to the contrary.

   The final sentence in the third paragraph in section IV.A.
is deleted in its entirety and replaced with the following lan-
guage:

    The Lands Council argues to the contrary.

   The first sentence in the third paragraph in section IV.B.
states:

    The Forest Service concedes that it did not test the
    activity area, but argues that because it tested similar
    soils within the Forest, and similar soils act the same
    way, then the methodology is sound.
1022               LANDS COUNCIL v. POWELL
  The first sentence in the third paragraph in section IV.B. is
deleted in its entirety and replaced with the following lan-
guage:

    The Forest Service concedes that it did not test much
    of the activity area, but argues that because it tested
    similar soils within the Forest, and similar soils act
    the same way, then the methodology is sound.

   The second sentence in the fifth paragraph in section IV.C.
states:

    Problems with the TSMRS database have been rec-
    ognized by another court, which found that the data-
    base overstates old growth by thirty-two percent to
    fifty-six percent. Lands Council, 198 F. Supp. 2d at
    1224.

   The second sentence in the fifth paragraph in section IV.C.
is deleted in its entirety.

  The two sentences in footnote 25 in section IV.C. read:

    Snags are typically dense woody areas created by
    fallen trees or branches, and they are a key habitat
    for the pine marten, one of the Indicator Species.
    The database contains no information about this key
    habitat variable.

  The two sentences in footnote 25 in section IV.C. are
deleted in their entirety and replaced with the following lan-
guage:

    Snags are dead, standing trees, and they are a key
    habitat for the pine marten, one of the Indicator Spe-
    cies. The database contains no information about this
    key habitat variable.
                   LANDS COUNCIL v. POWELL                  1023
   The final sentence in the sixth paragraph in section IV.C.
states:

    The spot surveys do not rehabilitate the proxy on
    proxy method, and, in this case, do not vitiate the
    Forest Service’s reliance on the proxy on proxy
    method as a monitoring of population trends.

   The final sentence in the sixth paragraph in section IV.C.
is deleted in its entirety and replaced with the following lan-
guage:

    The spot surveys do not rehabilitate the proxy on
    proxy method, and, in this case, do not vindicate the
    Forest Service’s reliance on the proxy on proxy
    method as a monitoring of population trends.

  IT IS SO ORDERED.


                          OPINION

GOULD, Circuit Judge:

   Appellants Lands Council, Kootenai Environmental Alli-
ance, Ecology Center, and Idaho Sporting Congress, Inc. (col-
lectively “the Lands Council”) challenge the timber harvest
approved by the United States Forest Service (“Forest Ser-
vice”) as part of a “watershed restoration” project in the Idaho
Panhandle National Forest (“the Forest” or “IPNF”). The dis-
trict court granted summary judgment in favor of the Forest
Service, finding that it had complied with the National Envi-
ronmental Policy Act, 42 U.S.C. § 4321 et seq. (“NEPA”) and
the National Forest Management Act, 16 U.S.C. § 1601 et
seq. (NFMA), and thus did not act arbitrarily and capriciously
by approving the plan. For the reasons discussed below, we
reverse the decision of the district court and grant summary
judgment in favor of Lands Council.
1024                   LANDS COUNCIL v. POWELL
                                     I

   The Lands Council contests a decision of the United States
Forest Service to proceed with Modified Alternative Eight of
the Iron Honey Project (“Project”). The Project area is at the
headwaters of the Little North Fork of the Coeur d’Alene
River, and the fourteen watersheds within the Project area
account for one-fifth of the watershed1 of the Little North
Fork. In the Little North Fork watershed, 39,977 acres of
National Forest have been logged since 1960. As a result of
this intense logging, all but two of the fourteen watersheds
within the Project area either are not functioning or are func-
tioning at risk. The Project is designed to improve the aquatic,
vegetative, and wildlife habitat in the Project area. Stated
another way, this is a project designed to restore nature’s bal-
ance in the watersheds within the Project area.2 This is no
easy task because of past environmental degradation. None-
theless, the Lands Council is troubled that the selected alter-
native allows the logging of 17.5 million board feet of lumber
from 1,408 acres of the Idaho Panhandle National Forest in
order to fund the project.

   The Forest Service began scoping3 the Project in 1996 as a
  1
     A “watershed” is the whole gathering ground of a river system; i.e., the
geographic area from which any river or creek draws its flow. The New
Shorter Oxford English Dictionary 3636 (Thumb Index ed. 1993).
   2
     Unlike other types of federal conservation statutes, the law regulating
the use of national forests embraces concepts of “multiple use” and “sus-
tained yield of products and services.” 16 U.S.C. § 1607. The Forest Ser-
vice is obligated to balance competing demands on national forests,
including timber harvesting, recreational use, and environmental preserva-
tion. 16 U.S.C. §§ 528-31. “The national forests, unlike national parks, are
not wholly dedicated to recreational and environmental values.” Cronin v.
United States Dept. of Ag., 919 F.2d 439, 444 (7th Cir. 1990).
   3
     “Scoping” describes when an agency begins initial consideration of a
project, and identifies the significant issues related to the contemplated
action. See 40 C.F.R. § 1501.7. Scoping typically happens after an agency
decides to complete an Environmental Impact Statement and serves to
identify the major issues the Environmental Impact Statement should
cover. Id. The Forest Service conducts scoping on all proposed actions.
Alaska Ctr. for the Env’t v. United States Forest Serv., 189 F.3d 851, 858
(9th Cir. 1999).
                      LANDS COUNCIL v. POWELL                        1025
watershed restoration project. In April 2000, a Draft Environ-
mental Impact Statement for the Project was released. After
receiving comments, the Final Environmental Impact State-
ment was released in November 2001. The Final Environmen-
tal Impact Statement included several alternatives. In
February 2002, the Supervisor of the IPNF issued a Record of
Decision that selected Modified Alternative Eight. Modified
Alternative Eight anticipates creating 17.5 million board feet
of commercial lumber by shelterwood4 harvesting of 1,408
acres. Modified Alternative Eight would also build 0.2 miles
of new road, 2 miles of temporary road, and reconstruct 29
miles of already existing roads.

  The Lands Council filed an administrative appeal with the
Regional Forester of Region One of the Forest Service. The
appeal was denied on May 15, 2002.

   The Lands Council then commenced this action pursuant to
the Administrative Procedure Act (“APA”), 5 U.S.C. § 701 et
seq., seeking review of the project on grounds that the Project
violates NEPA and NFMA. As part of the action, the Lands
Council attempted to supplement the administrative record by
deposing an employee of the United States Geological Sur-
vey. The Forest Service filed a motion to quash, and follow-
ing briefing on the question, the United States District Court
for the District of Idaho quashed the subpoena and prohibited
the Lands Council from offering evidence outside of the
administrative record. On subsequent cross-motions for sum-
mary judgment, the district court granted summary judgment
to the Forest Service. This appeal timely followed. We have
jurisdiction pursuant to 28 U.S.C. § 1291, and we reverse.
  4
   Shelterwood harvesting cuts the majority, but not all, of the trees in a
given harvesting site. The New Shorter Oxford English Dictionary 2822
(Thumb Index ed. 1993). The version of shelterwood harvesting in the
Project called for seventy percent of the canopy to be removed in the areas
to be logged.
1026                  LANDS COUNCIL v. POWELL
                                    II

   We review the district court’s decision to grant summary
judgment de novo with all facts read in the light most favor-
able to the non-moving party. Covington v. Jefferson County,
358 F.3d 626, 641 n.22 (9th Cir. 2004). Because this is a
record review case, we may direct that summary judgment be
granted to either party based upon our de novo review of the
administrative record. Cf. Sierra Club v. Babbitt, 65 F.3d
1502, 1507 (9th Cir. 1995) (“De novo review of a district
court judgment concerning a decision of an administrative
agency means we review the case from the same position as
the district court.”). Under the APA, we will reverse the
agency action only if the action is arbitrary, capricious, an
abuse of discretion, or otherwise contrary to law. 5 U.S.C.
§ 706(2).5 An agency’s action is arbitrary and capricious if the
agency fails to consider an important aspect of a problem, if
the agency offers an explanation for the decision that is con-
trary to the evidence, if the agency’s decision is so implausi-
ble that it could not be ascribed to a difference in view or be
the product of agency expertise, Motor Vehicle Mfrs. Ass’n v.
State Farm Mut. Auto. Ins. Co., 463 U.S. 29, 43 (1983), or if
the agency’s decision is contrary to the governing law. 5
U.S.C. § 706(2).

                                   III

  The Lands Council first challenges the NEPA analysis con-
ducted by the Forest Service. The Lands Council asserts that
  5
    The Lands Council urges that we apply the “rule of reason” standard,
Idaho Sporting Congress v. Thomas, 137 F.3d 1146, 1149 (9th Cir. 1998),
instead of the arbitrary and capricious standard. However, we have more
recently held that the rule of reason standard does not materially differ
from arbitrary and capricious review. Neighbors of Cuddy Mountain v.
Alexander, 303 F.3d 1059, 1071 (9th Cir. 2002). Because the Supreme
Court has reviewed the adequacy of NEPA analysis under the arbitrary
and capricious standard, e.g., Marsh v. Or. Natural Res. Council, 490 U.S.
360, 377 (1989), we will do the same.
                    LANDS COUNCIL v. POWELL                  1027
the Forest Service did not comply with the requirements of
NEPA when the Forest Service prepared an incomplete Envi-
ronmental Impact Statement. The Lands Council urges error
both in the Forest Service’s cumulative effects analysis of the
Project and in the scientific methodology employed by the
Forest Service.

                                A

   NEPA was passed by Congress to protect the environment
by requiring that federal agencies carefully weigh environ-
mental considerations and consider potential alternatives to
the proposed action before the government launches any
major federal action.

   [1] NEPA imposes procedural requirements, but not sub-
stantive outcomes, on agency action. Marsh v. Or. Natural
Res. Council, 490 U.S. 360, 371 (1989). For any proposed
major federal action, and it is not disputed that the Project
qualifies as such, NEPA requires the agency to prepare an
Environmental Impact Statement. 42 U.S.C. § 4332. “NEPA
requires that a federal agency consider every significant
aspect of the environmental impact of a proposed action . . .
[and] inform the public that it has indeed considered environ-
mental concerns in its decisionmaking process.” Earth Island
Inst. v. United States Forest Serv., 351 F.3d 1291, 1300 (9th
Cir. 2003) (internal quotation marks omitted). “In order to
accomplish this, NEPA imposes procedural requirements
designed to force agencies to take a ‘hard look’ at environ-
mental consequences.” Id.

                                B

  [2] Cumulative effects analysis requires the Final Environ-
mental Impact Statement to analyze the impact of a proposed
project in light of that project’s interaction with the effects of
past, current, and reasonably foreseeable future projects. See
40 C.F.R. § 1508.7. The Lands Council contends that the
1028               LANDS COUNCIL v. POWELL
Final Environmental Impact Statement did not take a “hard
look” at the cumulative effects of the Project in four areas: (1)
prior timber harvests; (2) reasonably foreseeable future timber
harvests; (3) the possibility of toxic sediment transport; and
(4) impact on Westslope Cutthroat Trout. The Forest Service
argues that its review of all issues was sufficient and that it
properly “analyzed the impact of a proposed project in light
of that project’s interaction with the effects of past, current,
and reasonably foreseeable future projects.” See 40 C.F.R.
§ 1508.7. Upon consideration, we conclude that the Forest
Service failed to take its required “hard look” with respect to
prior timber harvests and the impact on the Westslope Cut-
throat Trout.

  1.   Prior Timber Harvests

   The Lands Council first argues that the Final Environmen-
tal Impact Statement section on the cumulative impacts of
past timber harvests is “particularly vague and lacking in any
detailed discussion” because the Forest Service did not note
in detail past timber harvesting projects and the impact of
those projects on the Little North Fork watershed. We agree.
The Final Environmental Impact Statement generally
describes the past timber harvests, gives the total acres cut,
with types of cutting, per decade, and asserts that timber har-
vests have contributed to the environmental problems in the
Project area. But there is no catalog of past projects and no
discussion of how those projects (and differences between the
projects) have harmed the environment. Apart from a map in
the Project file that shows past harvests, with general notes
about total acres cut per watershed, there is no listing of indi-
vidual past timber harvests. Moreover, there is no discussion
of the connection between individual harvests and the prior
environmental harms from those harvests that the Forest Ser-
vice now acknowledges. Instead, the Final Environmental
Impact Statement contains only vague discussion of the gen-
eral impact of prior timber harvesting, and no discussion of
the environmental impact from past projects on an individual
                   LANDS COUNCIL v. POWELL                  1029
basis, which might have informed analysis about alternatives
presented for the current project.

   When we consider the purposes that NEPA was designed
by Congress to serve, what was done here is inadequate. Con-
gress wanted each federal agency spearheading a major fed-
eral project to put on the table, for the deciding agency’s and
for the public’s view, a sufficiently detailed statement of envi-
ronmental impacts and alternatives so as to permit informed
decision making. The purpose of NEPA is to require disclo-
sure of relevant environmental considerations that were given
a “hard look” by the agency, and thereby to permit informed
public comment on proposed action and any choices or alter-
natives that might be pursued with less environmental harm.
To this end, we have previously held that NEPA requires ade-
quate cataloguing of relevant past projects in the area. Muck-
leshoot Indian Tribe v. United States Forest Serv., 177 F.3d
800, 809-10 (9th Cir. 1999) (“[A]n EIS must catalogue ade-
quately the relevant past projects in the area. . . . Detail is
therefore required in describing the cumulative effects of a
proposed action with other proposed actions.” (internal cita-
tion and quotation marks omitted)). Stated differently, the
general rule under NEPA is that, in assessing cumulative
effects, the Environmental Impact Statement must give a suf-
ficiently detailed catalogue of past, present, and future proj-
ects, and provide adequate analysis about how these projects,
and differences between the projects, are thought to have
impacted the environment. See Neighbors of Cuddy Mountain
v. United States Forest Serv., 137 F.3d 1372, 1379-80 (9th
Cir. 1998); City of Caramel-By-The-Sea v. United States
Dept. of Transp., 123 F.3d 1142, 1160-61 (9th Cir. 1997).

   The issue then is whether the description of past timber har-
vests and previous environmental harms caused by these past
timber harvests was set forth in sufficient detail to promote an
informed assessment of environmental considerations and
policy choices by the public and agency personnel upon
review of the Final Environmental Impact Statement.
1030                   LANDS COUNCIL v. POWELL
   [3] Here, while the Final Environmental Impact Statement
discloses tables with types of past harvesting, there was no
inclusion of the specific projects that comprise the totals.
Though the Forest Service asserts that the Final Environmen-
tal Impact Statement had a “comprehensive accounting” of
past timber harvests, in fact the prior harvests from different
projects were not separately discussed, neither as to their
method of harvest, nor as to the consequences of each.
Although the agency acknowledged broad environmental
harms from prior harvesting, the data disclosed would not aid
the public in assessing whether one form or another of harvest
would assist the planned forest restoration with minimal envi-
ronmental harm. For the public and agency personnel to ade-
quately evaluate the cumulative effects of past timber
harvests, the Final Environmental Impact Statement should
have provided adequate data of the time, type, place, and
scale of past timber harvests6 and should have explained in
sufficient detail how different project plans and harvest meth-
ods affected the environment. The Forest Service did not do
this, and NEPA requires otherwise.7 Muckleshoot, 177 F.3d at
809-10.

  6
   This was not difficult data to generate, as is apparent by the Forest Ser-
vice’s response to the Freedom of Information Act request from the Lands
Council.
  7
   The Forest Service contends that by not raising this issue in the com-
ments to the Draft Environmental Impact Statement, the Lands Council
has waived the argument. In our view, however, the Lands Council
expressly and adequately raised and preserved this issue. For example,
Lands Council commented on the Draft Environmental Impact Statement
that the maps supplied did not display all past logging, which was keyed
to method, to assess environmental impacts; the Forest Service replied that
such a list was in the Project Files. This list was not provided in the Final
Environmental Impact Statement, and the Forest Service’s response shows
that the Lands Council raised the issue and the current objection to the
Final Environmental Impact Statement is not waived.
                      LANDS COUNCIL v. POWELL                        1031
  2.    Reasonably Foreseeable Future Timber Harvests

   The Lands Council asserts that while the Final Environ-
mental Impact Statement lists only one future timber harvest
in the “reasonably foreseeable projects” discussion, there are
other future harvests that should have been included in this
section of the cumulative effects analysis. Lands Council
points to the Deerfoot Ridge Restoration Project, scoped six-
teen days after the Record of Decision was issued in this case,
and the Forest’s 1998 Geographic Assessment, which recom-
mends clear-cut harvesting in the Upper Little North Fork
area. Because the Final Environmental Impact Statement must
include cumulative effects discussion for “reasonably foresee-
able projects,” 40 C.F.R. § 1508.7, the Lands Council asks us
to determine whether these two potential projects are “reason-
ably foreseeable.”

   We need not address these issues, however, because: (1)
the parties have agreed that the Deerfoot Ridge Restoration
Project was in a different drainage and thus need not have
been considered, and (2) the Lands Council has represented
that a determination that the IPNF Inland Native Fish Strategy
applies — a decision that we reach later in this opinion —
moots the need to address the Geographic Assessment issue
here.

  3.    Possibility of Toxic Sediment Transport

  Lands Council argues that cumulative effects analysis did
not consider the risk of increased peak flows in the relevant
watersheds caused by “rain-on-snow” events8 that could churn
  8
    “Rain-on-snow” events occur when rain melts a significant amount of
snow that has accumulated in the holes or gaps in a forest. This melting
causes a spike in runoff and water flow in watersheds. Increased water
flows are reflected in greater velocity and volume of water; the water runs
faster and there is more of it. The result is massive sediment deposit in a
watershed. Clearcutting and shelterwood harvesting increase both the like-
1032                  LANDS COUNCIL v. POWELL
up toxic sediments downstream at the confluence of the North
and South Forks of the Coeur d’Alene River. The evidence
proffered to the district court demonstrated the risks of “peak
flows”9 from the North Fork which scoop up toxic sediments
from the heavily contaminated South Fork of the Coeur
d’Alene River and deposit the contamination in Lake Coeur
d’Alene and/or the Spokane River. Further, the proffered evi-
dence showed that the Project would increase the amount and
intensity of such peak flows, increasing the possible risk of
toxic sediment transportation. The Lands Council argues that
the district court erred by refusing to admit new evidence of
this risk, and that had the evidence been admitted, it would
have shown that the Forest Service did not address a substan-
tial environmental risk posed by the Project. The Forest Ser-
vice argues that the administrative record is complete, with no
need of supplementation, and that district court did not err in
limiting consideration to the administrative record.

   The Lands Council’s enthusiastic argument pressing evi-
dence that the Forest Service did not consider stands at odds
with the norms of administrative law and typical judicial
review of agency action.10 As the district court below cor-
rectly noted, the Supreme Court has expressed a general rule
that courts reviewing an agency decision are limited to the
administrative record. Fla. Power & Light Co. v. Lorion, 470
U.S. 729, 743-44 (1985). We have previously stated that
“[j]udicial review of an agency decision typically focuses on

lihood and severity of such events by opening up the necessary gaps in the
forest. See, e.g., Brian Connelly, The Cumulative Effects of Forest Man-
agement on Peak Flows During Rain-on-Snow Events, at
http://depts.washington.edu/cwws/Theses/connelly.html (1992) (abstract
of a theses studying this phenomenon).
   9
     “Peak flows,” often measured in cubic feet (of water) per second, are
the highest volume of water passing a given point at a given time.
   10
      Normally, if an Agency’s administrative record is incomplete, we
would expect litigants to seek to supplement the record in the agency
before seeking to expand the record before the district court.
                      LANDS COUNCIL v. POWELL                       1033
the administrative record in existence at the time of the deci-
sion and does not encompass any part of the record that is
made initially in the reviewing court.” Southwest Ctr. for Bio-
logical Diversity v. United States Forest Serv., 100 F.3d 1443,
1450 (9th Cir. 1996). This general rule derives from our statu-
tory role to review an agency’s action. See Fla. Power &
Light Co., 470 U.S. at 743-44 (“The task of the reviewing
court is to apply the appropriate APA standard of review to
the agency decision based on the record the agency presents
to the reviewing court.”) (internal citation omitted).

   We have, however, crafted narrow exceptions to this gen-
eral rule. In limited circumstances, district courts are permit-
ted to admit extra-record evidence: (1) if admission is
necessary to determine “whether the agency has considered
all relevant factors and has explained its decision,” (2) if “the
agency has relied on documents not in the record,” (3) “when
supplementing the record is necessary to explain technical
terms or complex subject matter,” or (4) “when plaintiffs
make a showing of agency bad faith.” Southwest Ctr., 100
F.3d at 1450 (internal citation and quotation marks omitted).11
These limited exceptions operate to identify and plug holes in
the administrative record. Though widely accepted, these
exceptions are narrowly construed and applied. See, e.g.,
Camp v. Pitts, 411 U.S. 138, 142-43 (1973) (holding that a
reviewing court may require supplementation of the adminis-
trative record if it is incomplete); USA Group Loan Servs.,
Inc. v. Riley, 82 F.3d 708, 715 (7th Cir. 1996) (holding that
a “court is supposed to make its decision on the basis of the
administrative record,” but that “[t]here are exceptions”) (cit-
ing Animal Def. Council v. Hodel, 840 F.2d 1432, 1436 (9th
Cir. 1988), amended by, 867 F.2d 1244 (1989)).

  The scope of these exceptions permitted by our precedent
  11
    A district court’s decision whether to admit extra-record evidence is
reviewed for abuse of discretion. Southwest Ctr. for Biological Diversity,
100 F.3d at 1447.
1034                  LANDS COUNCIL v. POWELL
is constrained, so that the exception does not undermine the
general rule. Were the federal courts routinely or liberally to
admit new evidence when reviewing agency decisions, it
would be obvious that the federal courts would be proceeding,
in effect, de novo rather than with the proper deference to
agency processes, expertise, and decision-making.

   [4] Here, the risks presented by the supplemental evidence
are serious, because the evidence purports to show that the
risk of toxic sediment transport would be increased by the
Project, thereby creating risks to the public downstream. Yet
it is not entirely clear that Lands Council could not have
moved the agency to supplement its record with this evidence.

   [5] Given the difficulty of this issue, we decline to answer
it without necessity. We need not address the extra-record evi-
dence issue because we have determined that there are other
bases for reversing the district court and enjoining the Project.
The Lands Council’s arguments and evidence can be submit-
ted to the Forest Service, and be made part of the administra-
tive record, if and when the Forest Service conducts a new
NEPA analysis of the Project.

  4.    Cumulative Effects on Westslope Cutthroat Trout

   Finally, the Lands Council challenges the cumulative
effects analysis on the Westslope Cutthroat Trout.12 The
Lands Council contests the lack of up-to-date habitat informa-
  12
     “Westslope Cutthroat Trout (Oncorhynchus clarki lewisi, Salmonidae)
are native to the upper Columbia, Missouri, and South Saskatchewan river
drainages of western North America and are at the northern periphery of
their range in southeastern British Columbia, Canada.” E. B. Taylor et al.,
Population Subdivision in Westslope Cutthroat Trout (Oncorhynchus
clarki lewisi) at the Northern Periphery of its Range: Evolutionary Infer-
ences and Conservation Implications, 12 Molecular Ecology 2609, 2609
(2003) (available at http://www.zoology.ubc.ca/~etaylor/wsct2003.pdf).
This sub-species was scientifically described by Lewis and Clark, whose
cohort ate well on these fish.
                      LANDS COUNCIL v. POWELL                        1035
tion in the administrative record because the last survey of the
Westslope Cutthroat Trout’s habitat conditions was taken thir-
teen years ago. The Forest Service counters that it conducted
fish count surveys in 1993, 1994, 1996, and 1997.13

   The evidence of fish count surveys is unavailing. While
these fish count surveys are at least six years old, and reliance
on them is suspect, the Final Environmental Impact Statement
discloses no recent survey of the habitat of the Westslope Cut-
throat Trout. Evidence of the current habitat conditions, and
any degradation or improvement in the last thirteen years, is
relevant evidence in analyzing and determining what, if any,
impact the current Project will have on the cumulative effect
of current and past timber harvesting on trout habitat and on
trout population. Instead, the Forest Service predicted the
Project’s impact on the Westslope Cutthroat Trout (and its
habitat) using stale habitat data.

   [6] We do not suggest that all data relied upon by the
agency be immediate, but here the data about the habitat of
the Westslope Cutthroat Trout was too outdated to carry the
weight assigned to it. We conclude that the lack of up-to-date
evidence on this relevant question prevented the Forest Ser-
vice from making an accurate cumulative impact assessment
of the Project on the habitat and population of the Westslope
Cutthroat Trout. See Seattle Audubon Soc’y v. Espy, 998 F.2d
699, 704-05 (9th Cir. 1993) (overturning an agency decision
when it rested on “stale scientific evidence”).

                                    C

   Lands Council argues that the scientific methodology used
by the Forest Service in conducting its NEPA analysis was
flawed and therefore violated NEPA. We agree that the
  13
    Fish habitat surveys analyze the habitat conditions for potential fish.
Fish surveys count (or estimate) the actual number of fish in a watershed.
1036                  LANDS COUNCIL v. POWELL
WATSED model did contain faulty analysis, but defer deci-
sion as to the Forest Service’s sediment reduction analysis.

  1.   The WATSED Model

   The Lands Council first claims that the Final Environmen-
tal Impact Statement’s cumulative effects analysis of in-
stream sedimentation is arbitrary because the Water and Sedi-
ment Yields (“WATSED”) model14 used by the Forest Service
was incomplete and ignored key variables such as high peak
flow analysis, in-channel and streambank erosion, and “rain-
on-snow” peak flow events. Moreover, Lands Council argues
that the fact that the model was incomplete was never dis-
closed. The Forest Service rejoins that this is a technical dis-
pute and that courts routinely uphold modeling.

   NEPA requires that the Environmental Impact Statement
contain high-quality information and accurate scientific analy-
sis. 40 C.F.R. § 1500.1(b). If there is incomplete or unavail-
able relevant data, the Environmental Impact Statement must
disclose this fact. 40 C.F.R. § 1502.22. The government con-
cedes that the WATSED model does not include relevant
variables in determining total sedimentation of the watershed
and that WATSED does not have variables to predict the
effects of large-scale, high-intensity, short-term peak flows.
Although there are some disclosures of the model’s shortcom-
ings in an appendix to the Final Environmental Impact State-
ment,15 nowhere do the disclosures cover the limitations of
WATSED shown by the Lands Council and now conceded by
the Forest Service.
  14
      “WATSED” is the model the Forest Service used to estimate the
cumulative effects of the Project on water yield, peak flows, and sediment
yield.
   15
      For example, Appendix D to the Final Environmental Impact State-
ment notes that WATSED estimates cumulative effect based on the aver-
age, measured response of the watersheds used to develop the model and
that different watersheds react in different ways.
                       LANDS COUNCIL v. POWELL                        1037
   [7] The Forest Service’s heavy reliance on the WATSED
model in this case does not meet the regulatory requirements
because there was inadequate disclosure that the model’s con-
sideration of relevant variables is incomplete. Moreover, the
Forest Service knew that WATSED had shortcomings, and
yet did not disclose these shortcomings until the agency’s
decision was challenged on the administrative appeal.16 We
hold that this withholding of information violated NEPA,
which requires up-front disclosures of relevant shortcomings
in the data or models. See 40 C.F.R. § 1502.22; Lands Coun-
cil v. Vaught, 198 F. Supp. 2d 1211, 1239 (E.D. Wash. 2002)
(finding the same WATSED shortcomings and holding that
the Environmental Impact Statement failed to disclose such
shortcomings).

  2.    The Forest Service’s Sediment Reduction Analysis

   The Lands Council next argues that the EPA’s methodol-
ogy in calculating the sediment reduction for the Project was
faulty and that the district court erred in excluding evidence
supporting this contention. The Lands Council proffered evi-
dence to the district court that, if admitted, would tend to
show that the Forest Service “overstated” the amount of
reduced sedimentation from the Project’s culvert replacement.17
The Lands Council sought to demonstrate that many of the
Project area’s culverts were not likely to fail; that when the
culverts did fail, 100 percent of the sediment would not be
washed downstream; and that the Forest Service asserted that
it was replacing more culverts than its records indicated
existed in the Project area.
  16
      Unlike the general disclosure language in the appendix to the Final
Environmental Impact Statement, the “Appeal Transmittal Letter,” sent by
Forest Service staff to the Regional Forester in response to the administra-
tive appeal, states: “WATSED is not intended to estimate events, or
instantaneous pulses.”
   17
      One of the major improvements contemplated by the Project is to use
the money generated from timber harvests to replace or upgrade the exist-
ing culverts in the Project area.
1038                  LANDS COUNCIL v. POWELL
   On this issue as above explained in Part III.A.3, the Lands
Council urges us to make an exception to the general rule of
administrative review. Again we need not decide this issue,
because we have already held that NEPA was not satisfied.
The Lands Council may submit its evidence to the administra-
tive record if and when the Forest Service conducts a new
NEPA analysis on this Project.

                                   IV

   The Lands Council next attacks the Project’s compliance
with the NFMA. NFMA requires the Forest Service to create
a comprehensive Forest Plan for each national forest, see 16
U.S.C. §§ 1604(a), (e); Inland Empire Pub. Lands Council v.
United States Forest Serv., 88 F.3d 754, 757 (9th Cir. 1996)
(describing how the Forest Service first develops a Forest
Plan or Land Resource Management Plan consistent with the
requirements of NFMA), and once the Forest Plan is adopted,
NFMA prohibits any site-specific activities that are inconsis-
tent with the Forest Plan. Inland Empire Pub. Lands Council,
88 F.3d at 757 (“[S]ite-specific projects must be consistent
with the stage-one, forest-wide plan.”). The Lands Council
contends that the Project does not comply with the Forest Plan
in three areas: Protection of fisheries; soils impact; and old-
growth species viability. We address each in turn.

                                    A

   The Lands Council first argues that the Forest Service did
not comply with the portion of the IPNF Forest Plan that pro-
tects fisheries within the forest. The Forest Plan incorporates
an 80 percent success rate for fry emergence18 as a measure
of the health of the fisheries. The Lands Council asserts that,
  18
     An eighty percent success rate means that eighty percent of hatched
fish fry can escape the sediment that has settled on top of the eggs during
incubation. If too much sediment has settled on the eggs during incuba-
tion, the fry will suffocate before breaking through the sediment.
                      LANDS COUNCIL v. POWELL                       1039
pursuant to the NFMA, the Forest Service had a duty to ana-
lyze whether the project would meet the fry emergence stan-
dard within the Forest Plan. The Forest Service concedes that
it did not analyze the Project under the fry emergence stan-
dard, but argues that it did not have to do so.

   In 1995, the Forest Service amended the Forest Plan to
incorporate the Inland Native Fish Strategy (“INFISH”).
INFISH creates buffer zones in Riparian Habitat Conservation
Areas where the INFISH standards limit timber harvest and
minimize road construction to lessen sediment delivery to
streams. See Inland Native Fish Strategy: Decision Notice
and Finding of No Significant Impact A-5—A-7 (1995).19
INFISH also stated that “[the INFISH] interim standards and
guidelines replace existing conflicting direction [in Forest
Plans] except where Forest Plan direction provides for more
protection for inland native fish habitat.” Inland Native Fish
Strategy: Decision Notice and Finding of No Significant
Impact (1995); see also Inland Native Fish Strategy, 60 Fed.
Reg. 43758 (Aug. 23, 1995) (publishing the Finding of No
Significant Impact).

   The issue is whether the INFISH amendment supersedes, or
instead supplements, the Forest Plan’s existing fry emergence
standard. If the fry emergence standard is not implicitly super-
seded by INFISH, then the Forest Service’s decision must be
set aside because the fry emergence standard was never evalu-
ated in the Final Environmental Impact Statement. Neighbors
of Cuddy Mountain, 137 F.3d 1372. The Forest Service argues
that the two standards are in conflict, and that the fry emer-
gence rule is less strict, and therefore that INFISH superseded
the fry emergence standard. The Lands Council argues to the
contrary.
  19
    It is unclear from the record whether some or all of the watersheds in
the Project area have been classified as Riparian Habitat Conservation
Areas to which INFISH applies.
1040               LANDS COUNCIL v. POWELL
   INFISH itself describes our framework for analysis. We
must first determine whether INFISH and the fry emergence
standard are in conflict. If they are, then we must determine
if the fry emergence standard “provides for more protection”
than INFISH alone.

   The Lands Council has the better of the argument. The two
standards do not necessarily conflict. The INFISH standard
tries to minimize sediment deposits by limiting where timber
harvest may take place within the National Forest. The emerg-
ing fry standard requires corrective actions if a certain sedi-
mentation threshold is met. There is no explicit rejection of
the fry emergence standard in the INFISH requirements.
There is also no implicit rejection, because both standards can
be met in all cases: The INFISH standard will always be
required and, in addition, the fry emergence standard will
apply when cumulative sedimentation thresholds reach a pre-
scribed level. Because the INFISH standard and the fry emer-
gence standard measure different variables, are triggered by
different conditions, and have different remedies, applying
both to this, or any conceivable, project presents no conflict.

   [8] The Forest Service asserts that we owe its interpretation
deference as a reasonable interpretation of an ambiguity in a
Forest Plan. See Idaho Sporting Cong., 137 F.3d at 1154.
There is no call for deference to the agency’s legal interpreta-
tion of these two standards, however, because neither the
scope nor the effect of the two standards is ambiguous. See,
e.g., The Wilderness Society v. United States Fish & Wildlife
Serv., 353 F.3d 1051, 1059 (9th Cir. 2003) (en banc),
amended by, 360 F.3d 1374 (9th Cir. 2004). Because the fry
emergence standard does not conflict with INFISH, the Forest
Service’s decision should have considered the fry emergence
standard to comply with the Forest Plan under NFMA. That
it did not do so is legal error.
                       LANDS COUNCIL v. POWELL                         1041
                                     B

   The Lands Council next challenges the Forest Service’s
analysis of disturbed soil conditions. Under the Forest Plan
and the applicable Regional Soil Quality Standard, the Forest
Service cannot allow an activity that would create detrimental
soil conditions in fifteen percent of the project area.20 The
Lands Council’s claims that the methodology that the Forest
Service used to calculate the amount of soil that was in a det-
rimental state was insufficiently reliable because the Forest
Service never sampled the soil in the activity area. Instead,
based on samples from throughout the Forest, and aerial pho-
tographs, the Forest Service estimated the quality of the soils
in the Project area using a spreadsheet model.

   [9] This methodology has previously been called into ques-
tion. In a similar case, the United States District Court for the
Eastern District of Washington considered the exact same
methodology and concluded that its use was impermissible:

       The shortcomings in the USFS analysis are all
       directly tied to the fact that they did not take the time
       to walk the areas that they planned to harvest.
       Instead, based on assumptions [from general data
       from the IPNF soils], geological maps, and aerial
       photographs, they estimated the condition of each
       unit, tried to determine which units might exceed
       established standards, and projected potassium
       levels.

Kettle Range Conservation Group v. United States Forest
Serv., 148 F. Supp. 2d 1107, 1127 (E.D. Wash. 2001) (empha-
sis in original). We find this reasoning persuasive. Here, the
  20
     Under the Regional Soil Quality Standard, if fifteen percent or more
of the project area already has detrimental soil conditions, then the Project
will not be permitted to make it worse. Rather, the Project should then aim
to improve the soil conditions.
1042               LANDS COUNCIL v. POWELL
same problem exists: The Forest Service did not walk, much
less test, the land in the activity area.

   The Forest Service concedes that it did not test much of the
activity area, but argues that because it tested similar soils
within the Forest, and similar soils act the same way, then the
methodology is sound. Moreover, the Forest Service argues
that we owe its technical expertise deference. See Marsh, 490
U.S. at 378.

   [10] The Forest Service, granted appropriate deference, still
does not demonstrate the required reliability of the spread-
sheet model. We are asked to trust the Forest Service’s inter-
nal conclusions of the reliability of the spreadsheet model
when the Forest Service did not verify the predictions of the
spreadsheet model. Under the circumstances of this case, the
Forest Service’s basic scientific methodology, to be reliable,
required that the hypothesis and prediction of the model be
verified with observation. The predictions of the model,
which may be reliable across the entire Forest, were not veri-
fied with on the ground analysis. The Forest Service, and con-
sequently the public at large, has no way to know whether the
projection of the Project area’s soils was reliable. Was the
Forest Service “dead on” or “dead wrong?” The Final Envi-
ronmental Impact Statement is inadequate to tell. Our conclu-
sion that such unverified modeling is insufficient is similar to
the holding in Kettle Range, because in that case the court
noted that some of model’s input was based on data about the
soils throughout the Forest. 148 F. Supp. 2d at 1126-27. The
failure of the Forest Service in that case, as well as here, was
that the soils analysis was based entirely on the model with
no on-site inspection or verification. Therefore, we hold that
Forest Service’s reliance on the spreadsheet models, unac-
companied by on-site spot verification of the model’s predic-
tions, violated NFMA.

  The Lands Council also challenges the Forest Service’s
substantive determination that the fifteen percent detrimental
                      LANDS COUNCIL v. POWELL                       1043
soils threshold would not be exceeded by the Project. The
heart of this argument is the Lands Council’s contention that
it was error to categorize “jammer”21 roads as permanent capi-
tal improvements, which do not count for soil compaction.
Instead, Lands Council argues that these are temporary roads,
which must be included in the detrimental soils tally.22 The
Forest Service argues that the jammer roads, while once (and
perhaps still) used for harvesting, now are maintained and
used for other purposes such that, even if they were not ini-
tially permanent, they have now achieved that status.

   This determination is supported by the record and the For-
est Service properly excluded the jammer roads from the soils
analysis. Although we accept the Lands Council’s critique of
the Forest Service’s methodology for evaluating soil condi-
tions, we do not agree with the Lands Council’s argument that
the “jammer” roads should be considered temporary rather
than permanent.

                                    C

   The Lands Council’s final two arguments under the NFMA
relate to the Project’s impact on old growth forests.

   First, the Lands Council argues that the Project would not
allow the Forest Service to reach the ten percent old growth
forest minimum requirement contained in the Forest Plan. The
Plan’s minimum requirement for old growth forest is impor-
tant, both because these forests maintain our connection to a
bygone age and because their well-being is necessary for the
survival of many species. However, because no old growth
  21
     Jammer roads are roads created for the sole purpose of timber harvest
and are usually, but not always, abandoned when the harvest is completed.
  22
     This is a crucial determination, because if the jammer roads are
counted as temporary, then the fifteen percent threshold for detrimental
soil conditions would be exceeded prior to the Project, and the Project
would be required to prevent the conditions from worsening.
1044                   LANDS COUNCIL v. POWELL
forest is to be harvested under the selected alternative, we
reject the contention that the Project will be impermissible if,
thereafter, the “allocated old growth” within the Forest is less
than the Forest Plan requirement. If that requirement would
not be met after this Project, than it must be that the require-
ment is not met now, for the proposed timber harvest cut no
old growth. If we were to accept the Lands Council’s argu-
ment on this score, it would prevent any project from taking
place. We do not think this is a sensible reading of the
NFMA. Because no old growth forest is to be harvested under
the Project, we hold that it cannot be said that the Project
itself violates the IPNF Plan’s requirement to maintain ten
percent of the forest acreage as old growth forest. Thus we
reject the Lands Council’s first contention regarding old
growth forests.

   Second, the Lands Council challenges the old growth forest
analysis as it relates to the population and viability of species
that require old growth habitat, arguing that the Forest Service
was obligated to look deeper than the cumulative effects of
the Project on Management Indicator Species (“Indicator Spe-
cies”) by conducting a long-term viability study of the Indica-
tor Species. NFMA requires that the Forest Service identify
Indicator Species, monitor their population trends, and evalu-
ate each project alternative in terms of the impact on both
Indicator Species habitat and Indicator Species populations.
Idaho Sporting Cong., Inc. v. Rittenhouse, 305 F.3d 957, 971-
74 (9th Cir. 2002).

   We have, in appropriate cases, allowed the Forest Service
to avoid studying the population trends of the Indicator Spe-
cies by using Indicator Species habitat as a proxy for Indicator
Species population trends in a so-called “proxy on proxy”
approach.23 Id. at 972. Crucial to this approach, however, is
  23
     The “proxy on proxy” approach to studying MIS population trends
operates on the assumption that as long as a species’ habitat is maintained,
the species will likewise be maintained. Thus, analysis of trends in the
species habitat is, in essence, an indirect measurement of the species popu-
lation trends.
                      LANDS COUNCIL v. POWELL                       1045
that the methodology for identifying the habitat proxy be
sound. Id. If the habitat trend data is flawed, the proxy on
proxy result, here species population trends, will be equally
flawed.

   Here, there is evidence that the Forest Service’s main tool
for old growth calculation, the timber stand management
reporting system database (“TSMRS”), was inaccurate. The
record here shows that the proffered data is about fifteen years
old, with inaccurate canopy closure estimates, and insufficient
data on snags.24 These concerns leave the results generated by
the proxy on proxy approach unable to satisfy the require-
ments of the NFMA. Rittenhouse, 305 F.3d at 970 n.5 (hold-
ing that if the proxy on proxy method is flawed, then NFMA
is violated because there was no population monitoring as
required by NFMA).

   While the majority of the Forest Service’s analysis is proxy
on proxy, the Forest Service asserts that it does not rely
entirely on the flawed database because it employed field sur-
veys and on-the-ground detection methods. But the spot sur-
veys done for the Final Environmental Impact Statement are
largely irrelevant: They cannot make up for the problems with
the database because different variables are being measured.
Moreover, the surveys do not even begin to qualify as an
accurate monitoring of population trends. The spot surveys do
not rehabilitate the proxy on proxy method, and, in this case,
do not vindicate the Forest Service’s reliance on the proxy on
proxy method as a monitoring of population trends.

   [11] We conclude that, on the record presented in this case,
the Forest Service has not complied with NFMA. The Forest
Service has not ensured that there are no adverse viability
concerns to the relevant MISs because the Forest Service did
  24
   Snags are dead, standing trees, and they are a key habitat for the pine
marten, one of the Indicator Species. The database contains no information
about this key habitat variable.
1046                  LANDS COUNCIL v. POWELL
not monitor MIS population trends, and its proxy on proxy
approach was flawed as applied here.

                                   V

   [12] The decision made by the Forest Service to proceed
with Modified Alternative Eight violates both NEPA and
NFMA.25 We reverse the district court’s summary judgment
granted to the Forest Service, and direct the district court to
enter summary judgment on behalf of the Lands Council,
vacating the agency’s decision. The stay we entered on April
12, 2004, is reaffirmed and “shall remain in full force and
effect until the Forest Service satisfies its NEPA” and NFMA
obligations. Blue Mountains Biodiversity Project v. Black-
wood, 161 F.3d 1208, 1216 (9th Cir. 1998).

  REVERSED and REMANDED with INSTRUCTIONS.




  25
    Although harmless error is not addressed by either party, we conclude
that the errors we have pointed out are not harmless because they pre-
vented a proper, thorough, and public evaluation of the environmental
impact of the Project. See Laguna Greenbelt, Inc. v. United States Dept.
of Transp., 42 F.3d 517, 527 (9th Cir. 1994).
