
537 F.3d 1006 (2008)
NORTHWEST ENVIRONMENTAL ADVOCATES; The Ocean Conservancy, Inc.; Waterkeepers Northern California, Petitioners, and
The States Of New York, Illinois, Michigan, Minnesota, Wisconsin, and the Commonwealth of Pennsylvania, Petitioners-Intervenors,
v.
UNITED STATES ENVIRONMENTAL PROTECTION AGENCY, Respondent.
Northwest Environmental Advocates; The Ocean Conservancy; Santa Monica Baykeeper, dba San Francisco Baykeeper; dba DeltaBaykeeper, Plaintiffs-Appellees, and
The States of New York, Illinois, Michigan, Minnesota, Wisconsin, and the Commonwealth of Pennsylvania, Plaintiffs-Intervenors-Appellees,
v.
United States Environmental Protection Agency, Defendant-Appellant, and
The Shipping Industry Ballast Water Coalition, Defendant-Intervenor-Appellant.
Northwest Environmental Advocates; The Ocean Conservancy; Santa Monica Baykeeper, dba San Francisco Baykeeper; dba DeltaBaykeeper, Plaintiffs-Appellees, and
The States of New York, Illinois, Michigan, Minnesota, Wisconsin, and the Commonwealth of Pennsylvania, Plaintiffs-Intervenors-Appellees,
v.
United States Environmental Protection Agency, Defendant-Appellant, and
The Shipping Industry Ballast Water Coalition, Defendant-Intervenor-Appellant.
Nos. 03-74795, 06-17187, 06-17188.
United States Court of Appeals, Ninth Circuit.
Argued and Submitted August 14, 2007.
Filed July 23, 2008.
*1009 Deborah A. Sivas, Stanford Law School Legal Clinics, Stanford, CA, Melissa Powers, Pacific Environmental Advocacy Center, Portland, OR, Michael R. Lozeau, Alameda, CA, for the petitioners-appellants/plaintiffs-appellees.
Jennifer L. Scheller, Thomas Sansoneti, Martin F. McDermott, U.S. Department of Justice, Washington, D.C., Nancy J. Marvel, U.S. Environmental Protection Agency, San Francisco, CA, for the respondent-appellee/defendant-appellant.
Michael W. Evans; Brian K. McCalmon; James R. Weiss, Kirkpatrick & Lockhart Preston Gates Ellis, Washington, D.C., Tim Walker; Rachel R. Davidson, Kirkpatrick & Lockhart Preston Gates Ellis, San Francisco, CA, for the defendant-intervenor-appellant.
Timothy Hoffman, Office of the Attorney General, Buffalo, NY, Michael A. Cox, Michigan Attorney General, Lansing, MI, Peggy Lautenschlager, Wisconsin Attorney General, Madison, WI, Mike Hatch, Attorney General of Minnesota, St. Paul, MN, Lisa Madigan, Office of the Attorney General, Chicago, IL, Richard P. Mather, Department of Environmental Protection, Harrisburg, PA, Michael W. Evans, Kirkpatrick & Lockhart Preston Gates Ellis, for the plaintiffs-intervenors-appellees.
*1010 Before: HAWKINS, KIM McLANE WARDLAW, and W. FLETCHER, Circuit Judges.
WILLIAM A. FLETCHER, Circuit Judge:
Plaintiffs in this case are Northwest Environmental Advocates, San Francisco Baykeeper, and The Ocean Conservancy. Plaintiffs-intervenors are the States of Illinois, Michigan, Minnesota, New York, Pennsylvania, and Wisconsin. Plaintiffs and plaintiffs-intervenors challenge a regulation originally promulgated by the Environmental Protection Agency ("EPA") in 1973 exempting certain marine discharges from the permitting scheme of sections 301(a) and 402 of the Clean Water Act ("CWA"). That regulation, 40 C.F.R. § 122.3(a), provides that the following vessel discharges into the navigable waters of the United States do not require permits: discharge of effluent from properly functioning marine engines; discharge of laundry, shower, and galley sink wastes from vessels; and any other discharge incidental to the normal operation of a vessel, including the discharge of ballast water.
The district court concluded that the EPA had exceeded its authority under the CWA in exempting these discharges from permitting requirements. The district court vacated § 122.3(a), effective September 30, 2008. We affirm the decision of the district court.

I. Background

A. The CWA and 40 C.F.R. § 122.3(a)
In 1972, Congress enacted sweeping amendments to the Federal Water Pollution Control Act of 1948. After another round of substantial amendments in 1977, the statute became known as the Clean Water Act. The CWA declares a "national goal that the discharge of pollutants into the navigable waters be eliminated by 1985." 33 U.S.C. § 1251(a)(1).
Section 301(a) of the CWA provides that, subject to certain exceptions, "the discharge of any pollutant by any person shall be unlawful." Id. § 1311(a). One of these exceptions is for discharges authorized by a permit granted pursuant to the National Pollutant Discharge Elimination System ("NPDES"), a system set forth in section 402 of the Act. Id. §§ 1311(a), 1342. The combined effect of sections 301(a) and 402 is that "[t]he CWA prohibits the discharge of any pollutant from a point source into navigable waters of the United States without an NPDES permit." N. Plains Res. Council v. Fid. Exploration & Dev. Co., 325 F.3d 1155, 1160 (9th Cir.2003). The EPA administers the NPDES. 33 U.S.C. § 1251(d).
Obtaining a permit under the CWA need not be an onerous process. For example, in appropriate circumstances a discharge may be allowed under a "general permit" requiring only that the discharger submit a "notice of intent" to make the discharge. As we explained in Natural Resources Defense Council v. U.S. EPA, 279 F.3d 1180, 1183 (9th Cir.2002):
NPDES permits come in two varieties: individual and general. An individual permit authorizes a specific entity to discharge a pollutant in a specific place and is issued after an informal agency adjudication process. See 40 C.F.R. §§ 122.21, 124.1-124.21, 124.51-124.66. *1011 General permits, on the other hand, are issued for an entire class of hypothetical dischargers in a given geographical region and are issued pursuant to administrative rulemaking procedures. See id. §§ 122.28, 124.19(a). General permits may appropriately be issued when the dischargers in the geographical area to be covered by the permit are relatively homogenous. See id. § 122.28(a)(2). After a general permit has been issued, an entity that believes it is covered by the general permit submits a "notice of intent" to discharge pursuant to the general permit. Id. § 122.28(b)(2). A general permit can allow discharging to commence upon receipt of the notice of intent, after a waiting period, or after the permit issuer sends out a response agreeing that the discharger is covered by the general permit. Id. § 122.28(b)(2)(iv).
In 1973, the EPA exempted by regulation several categories of vessel discharges from NPDES permitting requirements under the CWA. See NPDES, 38 Fed.Reg. 13,528, 13,530, § 125.4 (May 22, 1973). The regulation provides that "[t]he following discharges do not require NPDES permits":
Any discharge of sewage from vessels, effluent from properly functioning marine engines, laundry, shower, and galley sink wastes, or any other discharge incidental to the normal operation of a vessel. This exclusion does not apply to rubbish, trash, garbage, or other such materials discharged overboard; nor to other discharges when the vessel is operating in a capacity other than as a means of transportation[.]
40 C.F.R. § 122.3(a). The CWA expressly exempts sewage discharges from vessels from the permitting process and regulates these discharges by other means. See 33 U.S.C. §§ 1362(6)(A), 1322. Because § 122.3(a) does not itself exempt sewage discharges but instead merely recognizes the statute's exemption of sewage discharges, the sewage clause in § 122.3(a) is not subject to the ultra vires claim made here. See also Chevron U.S.A., Inc. v. Hammond, 726 F.2d 483, 493 n. 13 (9th Cir.1984) (contrasting the express statutory exemption of sewage with regulation relating to "deballasting" by ships). Therefore, three categories of discharges exempted by 40 C.F.R. § 122.3(a) are at issue in this case: (1) marine engine discharges; (2) graywater discharges ("laundry, shower, and galley sink wastes"); and (3) "any other discharge incidental to the normal operation of a vessel."
The first proposed draft of the regulation would have excluded only marine engine discharges. See NPDES, 38 Fed. Reg. 1362, 1363-64, § 125.4(c) (proposed Jan. 11, 1973). The EPA subsequently added the exclusions for graywater and other discharges incidental to normal vessel operations. When promulgating the final regulation in May 1973, the EPA explained its anticipated effect: "Most discharges from vessels to inland waters are now clearly excluded from the permit requirements." 38 Fed.Reg. at 13,528, (b)(13)(ii). The EPA stated that "[t]his type of discharge generally causes little pollution." Id. The EPA stated, further, that the "exclusion of vessel wastes from the permit requirements will reduce administrative costs drastically." Id. Decades later, an EPA administrator declared that in 1973:
[W]e were faced with many, many other much higher priority situations such as raw sewage being discharged, municipal plants having to be built, very large paper mills or steel mills and the like discharging. At the time we thought that was not an important area to deal with.... Vessels were not important to the overall scheme of things at that time.
*1012 Craig Vogt, EPA, EPA Pub. Meeting # 12227, Ocean Discharge Criteria (Sept. 12, 2000, 1 p.m.). The EPA amended the regulation in 1979 in minor respects that do not affect our analysis. See NPDES, Revision of Regulations, 44 Fed.Reg. 32,854, 32,902, § 122.4 (June 7, 1979); see also NPDES, Revision of Existing Regulations, 43 Fed.Reg. 37,078, 37,079, I(c)(2) (Aug. 21, 1978) (describing the proposed changes).
The text of the CWA does not exempt from NPDES requirements marine engine discharges, graywater discharges, or other discharges incidental to the normal operation of vessels. However, the EPA contended in 1973, and continues to contend, that it has the power to provide these exemptions by regulation. The Administrator of the EPA prefaced the draft January 1973 regulation with a statement that a discharger could discharge lawfully only if the discharger "possesses a valid permit or is excluded from coverage by law or regulation." NPDES, 38 Fed.Reg. at 1362 (emphasis added). The final rules similarly stated that "[a]ll discharges of pollutants ... are unlawful ..., unless the discharger has a permit or is specifically relieved by law or regulation from the obligation of obtaining a permit." NPDES, 38 Fed.Reg. at 13,531, § 125.11(a) (emphasis added).
The first category exempted by § 122.3(a), marine engine discharges, includes unburned fuel and various kinds of oil. The second category, graywater discharges, can include pathogens such as fecal coliform, enterococci, and E. coli and pollutants such as ammonia, arsenic, copper, lead, nickel, and zinc. See, e.g., EPA Draft Cruise Ship Discharge Assessment Report (Dec.2007), available at http:// www.epa.gov/owow/oceans/cruise_ships/ pdf_disch_assess/cruiseship_discharge_assessment_report.pdf. The third category, "any other discharge," includes, among other discharges, ballast water from ships. Cf. 33 U.S.C. § 1322(a)(12)(A)(i) (defining this broad "other discharge" category for purposes of a different CWA section).
Plaintiffs have made clear, both here and in the district court, that their primary environmental concern stems from the discharge of ballast water. We quote a passage from the district court's order granting plaintiffs' motion for permanent injunctive relief that describes the purpose of ballast water and the effects of its discharge:
Ballast water is water that is taken on by cargo ships to compensate for changes in the ship's weight as cargo is loaded or unloaded, and as fuel and supplies are consumed. Ballast water may be used for a number of different purposes, such as maintaining stability, maintaining proper propeller and bow immersion, and to compensate for off-center weights. Thus, ballast water is essential to the proper functioning of cargo ships, as well as to the safety of its crew.
Because ballast water is primarily used to compensate for changes in cargo, it is generally taken in or pumped out at the ports along a ship's route. When a ship takes on ballast water, whether freshwater or saltwater, organisms found in that water are typically taken in as well. These organisms are carried in the ballast tanks of the ship until the ship arrives at its next port, where, due to changes in the distribution of the ship's cargo, they may be released into a new ecosystem. Due to the size of ballast tanks on modern cargo ships, and the speed with which these ships can reach their destinations, organisms are increasingly able to survive the journey to a new ecosystem. All told, "more than 10,000 marine species each day hitch rides around the globe in the ballast water of cargo ships." A number of *1013 these species are released into U.S. waters in the more than 21 billion gallons of ballast water released in the United States each year.
If these foreign organisms manage to survive and reproduce in the new ecosystem, they can cause severe problems in the natural and human environment. For example, zebra mussels, native to the Caspian Sea region of Asia, were brought into the Great Lakes in the ballast water of cargo ships. "Zebra mussels have clogged the water pipes of electric companies and other industries; infestations in the Midwest and Northeast have cost power plants and industrial facilities almost $70 million between 1989 and 1995." As another example, according to a 2001 EPA report,
[a]n introduced strain of cholera bacteria, possibly released in the bilge water of a Chinese freighter, caused the deaths of 10,000 people in Latin America in 1991. This cholera strain was then imported into the United States from Latin America in the ballast tanks of ships that anchored in the port of Mobile, Alabama. Fortunately, cholera bacteria were detected in oyster and finfish samples in Mobile Bay ... and no additional deaths occurred from exposure to this pathogen.
With a lack of natural predators, invasive species can multiply rapidly and quickly take over an ecosystem, threatening native species. Indeed, invasive species "are a major or contributing cause of declines for almost half the endangered species in the United States." Once established, invasive species become almost impossible to remove, leading "[s]cientists, industry officials, and land managers [to] recogniz[e] that invasive species are one of the most serious, yet least appreciated, environmental threats of the 21st century."
In economic terms, invasive species can also have a devastating effect. The Department of Agriculture spends millions of dollars per year to detect and prevent invasive species. One study cited by the [General Accounting Office] concluded that "total annual economic losses and associated control costs [are] about $137 billion a year  more than double the annual economic damage caused by all natural disasters in the United States."
Nw. Envtl. Advocates v. U.S. EPA ("Northwest Environmental Advocates II"), No. 03-05760, 2006 WL 2669042, at *3-4, 2006 U.S. Dist. LEXIS 69476, at *10-12 (N.D.Cal. Sept. 18, 2006) (citations omitted; sixth alteration added).

B. Procedural History
In January 1999, plaintiffs petitioned the EPA, asking that the agency repeal 40 C.F.R. § 122.3(a). See Petition for Repeal of 40 CFR § 122.3(a) (Jan.1999) ("Petition for Rulemaking"). Plaintiffs contended that the regulation was not authorized by the CWA and was thus ultra vires. Plaintiffs sued the EPA a year and a half later, alleging unreasonable delay in responding to their petition. The district court ordered the EPA to respond to the petition, but the EPA obtained a stay from this circuit. Under a subsequent consent decree, the EPA agreed to "grant, deny, or grant in part and deny any remaining part of NWEA's petition" by September 2, 2003. Nw. Envtl. Advocates v. U.S. EPA, 340 F.3d 853, 857 (9th Cir.2003). On the day of the deadline, the EPA denied plaintiffs' petition in its entirety. See EPA, Decision on Petition for Rulemaking To Repeal 40 C.F.R. 122.3(a) (Sept. 2, 2003) ("EPA Decision on Petition"); see also Availability of Decision on Petition for Rulemaking To Repeal Regulation Related to Ballast Water, 68 Fed.Reg. 53,165 (Sept. 9, 2003) (giving notice of the denial).
*1014 Plaintiffs brought suit against the EPA three months later, in December 2003. Their first cause of action alleged that 40 C.F.R. § 122.3(a) is not authorized by the CWA and is thus ultra vires. See 5 U.S.C. § 706(2)(C). Their second cause of action alleged, based on their ultra vires argument, that the 2003 EPA Decision on Petition was "not in accordance with law." See 5 U.S.C. § 706(2)(A). At the same time, as a protective measure in the event that the district court lacked jurisdiction, the plaintiffs filed directly with this court a petition for review of the EPA Decision on Petition, pursuant to jurisdictional provisions contained in 33 U.S.C. § 1369(b)(1).
In March 2005, the district court granted summary judgment to plaintiffs on their first cause of action and ordered the EPA to repeal § 122.3(a). Nw. Envtl. Advocates v. U.S. EPA ("Northwest Environmental Advocates I"), No. 03-05760, 2005 WL 756614, at *7, 2005 U.S. Dist. LEXIS 5373, at *40 (N.D.Cal. Mar. 30, 2005). It is unclear whether the district court reached plaintiffs' second cause of action. Given the court's holding on the plaintiffs' first cause of action, however, it did not need to do so. The district court ordered further proceedings to determine the appropriate remedy. Id. The six states intervened as plaintiffs at the remedy stage "to protect their sovereign, proprietary, regulatory, and economic interest in the States' waters." The Shipping Industry Ballast Water Coalition ("Shipping Coalition") intervened as a defendant. In September 2006, the district court vacated the challenged portions of 40 C.F.R. § 122.3(a) as of September 30, 2008. Nw. Envtl. Advocates II, 2006 WL 2669042, at *1, 2006 U.S. Dist. LEXIS 69476, at *2.
The EPA and the Shipping Coalition (collectively, "the EPA") appealed the district court's decision to this court. We consolidated their appeal with the petition filed directly in this court.

II. Standard of Review
We review de novo questions of subject matter jurisdiction, Ecology Ctr., Inc. v. U.S. Forest Serv., 192 F.3d 922, 924 (9th Cir.1999); the legal question of whether a statute of limitations applies, Sierra Club v. Penfold, 857 F.2d 1307, 1315 (9th Cir.1988) (as amended); a district court's grant of summary judgment, Envtl. Prot. Info. Ctr. v. U.S. Forest Serv., 451 F.3d 1005, 1008 (9th Cir.2006); and the legal question of whether a plaintiff has exhausted the necessary administrative remedies, Great Basin Mine Watch v. Hankins, 456 F.3d 955, 961 (9th Cir.2006).
Under 5 U.S.C. § 706(2)(C), the provision we apply to the plaintiffs' first cause of action, we must "set aside agency action" that is "in excess of statutory jurisdiction, authority, or limitations, or short of statutory right." This standard requires the application of Chevron U.S.A., Inc. v. Natural Resources Defense Council, 467 U.S. 837, 104 S.Ct. 2778, 81 L.Ed.2d 694 (1984). See, e.g., Nat'l Mining Ass'n v. Fowler, 324 F.3d 752, 758 (D.C.Cir.2003); Anna Jacques Hosp. v. Leavitt, 537 F.Supp.2d 24, 29-30 (D.D.C. 2008) ("To determine if the Secretary exceeded his statutory authority in violation of 5 U.S.C. § 706(2)(C), the Court must engage in the two-step inquiry required by Chevron.").
When "reviewing an agency's statutory interpretation under the APA's `not in accordance with law' standard," see 5 U.S.C. § 706(2)(A), the standard applied to the plaintiffs' second cause of action, we also "adhere to the familiar two-step test of Chevron." Holland v. Nat'l Mining Ass'n, 309 F.3d 808, 815 (D.C.Cir.2002); cf. Cleveland v. Ohio, 508 F.3d 827, 838 (6th Cir.2007) ("Agency action is `not in accordance with the law' when it is in conflict with the language of the statute....").
*1015 We review the district court's remedial order for abuse of discretion. Biological Legal Found. v. Badgley, 309 F.3d 1166, 1176 (9th Cir.2002).

III. Discussion
The EPA argues that the district court lacked subject matter jurisdiction over plaintiffs' suit, and that we therefore have subject matter jurisdiction over the appeal only insofar as necessary to order the district court to dismiss the suit for want of jurisdiction. On the alternative assumption that the district court had subject matter jurisdiction, the EPA argues that the statute of limitations bars the ultra vires claim contained in the plaintiffs' first cause of action; that the district court erred on the merits in finding that the CWA did not authorize the exemptions contained in 40 C.F.R. § 122.3(a); and that the district court abused its discretion in choosing its remedy.

A. Subject Matter Jurisdiction
The district court had subject matter jurisdiction over plaintiffs' suit under the general federal question statute, 28 U.S.C. § 1331, unless some other statute divested the district court of jurisdiction. The only statute that could have had that effect is section 509(b)(1) of the CWA. See 33 U.S.C. § 1369(b)(1). Section 509(b)(1) specifies seven categories of agency action for which a challenge must be brought as an original proceeding in a court of appeals rather than in a district court. The EPA contends that plaintiffs' ultra vires challenge to § 122.3(a) falls within two of these seven categories. If the EPA is right as to either category, the district court did not have subject matter jurisdiction, and we must review the EPA's action by means of plaintiffs' petition for review filed directly in this court.
We do not lightly hold that we have jurisdiction under section 509(b)(1). We have "counseled against [its] expansive application." League of Wilderness Defenders/Blue Mountains Biodiversity Project v. Forsgren, 309 F.3d 1181, 1190 n. 8 (9th Cir.2002). "The specificity and precision of section [509], and the sense of it, persuade us that it is designed to exclude" EPA actions that Congress did not specify. Longview Fibre Co. v. Rasmussen, 980 F.2d 1307, 1313 (9th Cir.1992). Indeed, "[n]o sensible person ... would speak" with such detail otherwise. Id.
We address the two potentially relevant categories of section 509(b)(1) in turn, concluding that the agency action falls in neither category. The district court therefore had subject matter jurisdiction over plaintiffs' suit.

1. Section 509(b)(1)(E)
Subsection 509(b)(1)(E) provides for review by a court of appeals of EPA actions "in approving or promulgating any effluent limitation or other limitation under section 1311, 1312, 1316, or 1345 of this title." See 33 U.S.C. § 1369(b)(1)(E) (referring to sections 301, 302, 306, and 405 of the CWA). The CWA defines an "effluent limitation" as "any restriction ... on quantities, rates, and concentrations of chemical, physical, biological, and other constituents which are discharged from point sources...." 33 U.S.C. § 1362(11); see also 40 C.F.R. § 122.2 (same); cf. 33 U.S.C. §§ 1311, 1314(b) (establishing a procedure for adopting effluent limitations guidelines).
Section 509(b)(1)(E) authorizes original court of appeals jurisdiction for challenges to regulations that establish numerical limitations and similar limits. For example, in Natural Resources Defense Council v. U.S. EPA, 673 F.2d 400, 402 (D.C.Cir. 1982) ("NRDC D.C.Cir."), the D.C. Circuit exercised jurisdiction under section 509(b)(1)(E) over a challenge to regulations setting forth "a complex set of procedures for issuing or denying NPDES permits." *1016 In upholding its jurisdiction, the D.C. Circuit characterized the regulations as "restrict[ing] who may take advantage of certain provisions or otherwise guid[ing] the setting of numerical limitations in permits," and as constituting "`a limitation on point sources and permit issuers' and `a restriction on the untrammeled discretion of the industry' that existed before passage of the CWA." Id. at 404-05 (quoting Va. Elec. & Power Co. v. Costle, 566 F.2d 446, 450 (4th Cir.1977)).
The regulation in this case can be characterized as "approving or promulgating any effluent limitation or other limitation" only if those words are understood in a Pickwickian sense. The regulations in NRDC D.C.Cir. established procedures under which limitations on discharges of effluent would be implemented. Unlike the regulations in that case, 40 C.F.R. § 122.3(a) provides no limitation whatsoever.
We conclude that section 509(b)(1)(E) does not authorize original jurisdiction in the court of appeals in this case. Section 122.3(a) does not involve the approval or promulgation of "any effluent limitation or other limitation," but rather creates the categorical and permanent exemptions of three types of discharges from any limit imposed by a permitting requirement.

2. Section 509(b)(1)(F)
Section 509(b)(1)(F) provides for review by a court of appeals of EPA actions "in issuing or denying any permit under section 1342 of this title." See 33 U.S.C. § 1369(b)(1)(F) (referring to section 402 of the CWA).
In Crown Simpson Pulp Co. v. Costle, 445 U.S. 193, 196, 100 S.Ct. 1093, 63 L.Ed.2d 312 (1980), the Supreme Court held that section 509(b)(1)(F) also covers EPA actions "functionally similar" to the denial of permits. The facts of the case make clear that the Court understood functional similarity in a narrow sense. The State of California, which had been delegated permit granting authority under the CWA, proposed granting point source permits to pulp mills discharging pollutants into the ocean. See id. at 194-95, 100 S.Ct. 1093. If California had not been delegated permit granting authority, the EPA would have granted or denied the permits directly. See id. at 196-97, 100 S.Ct. 1093. The EPA vetoed the proposed permits. See id. at 194, 100 S.Ct. 1093. Because the EPA was not the permit-granting entity, the plaintiff contended that the EPA's action was not the issuance or denial of a permit within the meaning of section 509(b)(1)(F). See id. at 195, 100 S.Ct. 1093. The Court concluded instead that the fortuitous circumstance that this case arose in a state with permit-granting authority should not produce a different jurisdictional result from a case involving a state without such authority. Id. at 196-97, 100 S.Ct. 1093; see also Ga.-Pac. Corp. v. U.S. EPA, 671 F.2d 1235, 1238-40 (9th Cir.1982) (exercising original jurisdiction over denial of a variance); Pac. Legal Found. v. Costle, 586 F.2d 650, 655 (9th Cir.1978) (exercising original jurisdiction over extension of a permit), rev'd on other grounds, 445 U.S. 198, 100 S.Ct. 1095, 63 L.Ed.2d 329 (1980).
In American Mining Congress v. U.S. EPA, 965 F.2d 759, 763 (9th Cir.1992), we exercised jurisdiction under section 509(b)(1)(F) over a challenge to an EPA regulation of stormwater discharges from inactive mining operations. The EPA concluded that these discharges required permits because they were "associated with industrial activity" under section 402(p)(2)(B) of the CWA. The challenged regulation therefore required permits for most inactive mines, but, based on temporary permitting delays provided by section 402(p), the regulation exempted "reclaimed" inactive coal mines from the permit *1017 requirement until the expiration of a moratorium. Id. The Mining Congress, representing inactive mines required to obtain permits, challenged the portion of the regulation requiring permits. Id. at 764. The Mining Congress used the exemption of reclaimed mines to argue that other inactive mines should receive the same favorable treatment. Id. at 764-65. American Mining Congress is inapplicable to our case because the Mining Congress challenged the requirement that certain mines obtain a permit, not an exemption.
We have applied section 509(b)(1)(F) in two cases involving challenges to stormwater regulations where those regulations were based in part on exemptions specified in the text of the CWA. In Natural Resources Defense Council v. U.S. EPA ("NRDC 9th Cir. 1992"), 966 F.2d 1292 (9th Cir.1992), we exercised jurisdiction under section 509(b)(1)(F) over a challenge to a complex set of regulations governing discharges from stormwater runoff. See id. at 1296-97. Two statutory provisions formed the basis for the regulations. First, as mentioned above, section 402(p) of the CWA, 33 U.S.C. § 1342(p), "established deadlines by which certain storm water dischargers must apply for permits, the EPA or states must act on permits and dischargers must implement their permits." NRDC 9th Cir.1992, 966 F.2d at 1296. This section required that particularly important categories of discharges be regulated quickly, and exempted less important categories of discharges from regulation until 1992. Id. at 1295-96; see also Envtl. Def. Ctr., Inc. v. U.S. EPA, 344 F.3d 832, 840, 843 (9th Cir.2003) (exercising jurisdiction over challenge to stormwater discharge regulations promulgated under section 402(p)). Second, CWA section 402(l)(2), 33 U.S.C. § 1342(l)(2), exempted certain stormwater discharges by mining, oil, and gas facilities from CWA permitting requirements. See NRDC 9th Cir. 1992, 966 F.2d at 1306-07. The Natural Resources Defense Council argued that the EPA had extended the deadline for storm sewer discharges beyond the dates authorized by CWA section 402(p); that the EPA had defined improperly what constituted certain kinds of storm sewer discharges, so that some discharges were exempted from permitting requirements for a longer period than section 402(p) allowed; and that the EPA had erred in defining what constituted stormwater discharges from mining, oil, and gas facilities, improperly expanding the exemption from the permitting requirement contained in section 402(l)(2). NRDC 9th Cir.1992, 966 F.2d at 1299-1309.
In Natural Resources Defense Council v. U.S. EPA ("NRDC 9th Cir.2008"), 526 F.3d 591 (9th Cir.2008), we exercised jurisdiction under section 509(b)(1)(F) over a challenge to a regulation exempting certain stormwater discharges from mining, oil, and gas facilities. Id. at 601. Two CWA provisions lay behind the regulation. First, section 402(l)(2), mentioned above, exempted discharges from mining, oil, and gas facilities. Second, an amendment to the CWA specified that the exemption contained in section 402(l)(2) included discharges from construction activities at mining, oil, and gas facilities. See Energy Policy Act of 2005, Pub.L. No. 109-58, § 323, 119 Stat. 594, 694; 33 U.S.C. § 1362(24) (codifying the amendment to section 502). The Natural Resources Defense Council challenged the regulation as exempting a broader category of discharges than permitted under CWA sections 402(l)(2) and 502. NRDC 9th Cir.2008, 526 F.3d at 600-01.
In both of these cases in which we exercised jurisdiction under section 509(b)(1)(F), statutory provisions explicitly provided the underlying exemptions. The challenged regulations sought to define more precisely those discharges that came *1018 within statutory exemptions (and thus did not need permits) and those that did not come within statutory exemptions (and thus needed permits). In contrast, the case now before us challenges a regulation providing exemptions not contained in section 402 or in any other section of the CWA. This case thus does not involve the "issuing or denying[of] any permit under Section 402." See Natural Res. Def. Council v. Train, 396 F.Supp. 1393, 1402 (D.D.C.1975), aff'd sub nom. Natural Res. Def. Council v. Costle, 568 F.2d 1369 (D.C.Cir.1977).
We conclude by agreeing with the district court's analysis in a suit very similar to the one before us. In Environmental Protection Information Center v. Pacific Lumber Co., 266 F.Supp.2d 1101, 1108-09 (N.D.Cal.2003), the court addressed an EPA regulation that permanently exempted an entire class of silvicultural discharges from any NPDES permitting requirement. The district court noted that the court of appeals in NRDC D.C.Cir. had upheld original jurisdiction under section 509(b)(1) on the ground that if there were no such jurisdiction, there would be a "`perverse situation' in which the court `will be able to review the grant or denial of the permit, but will be without authority to review directly the regulations on which the permit is based.'" Id. at 1114 (quoting Natural Resources Defense Council, Inc. v. U.S. EPA ("NRDC D.C.Cir."), 656 F.2d 768, 775 (D.C.Cir.1981)). The district court wrote:
Because [plaintiff] challenges a decision that in effect excludes sources from the NPDES program, the circuit courts will never have to confront the issuance or denial of a permit for these sources. The Ninth Circuit, by virtue [of the regulation], will never have to consider on direct review an action involving the denial of an NPDES permit for pollutant discharges [within the exemption provided by the regulation]. Thus, a district court taking jurisdiction over a challenge to the silvicultural regulation does not create the same awkwardness for a circuit court as that described in [NRDC D.C.Cir.].
Id. at 1115-16 (footnote omitted) (upholding its own jurisdiction under 28 U.S.C. § 1331).
We conclude that § 509(b)(1)(F) does not authorize original jurisdiction in the court of appeals in this case. Section 122.3(a) does not involve the issuance or the denial of a permit or a functionally similar action, but rather the permanent exemptions of three types of discharges from any permitting requirement.

B. Statute of Limitations
The applicable statute of limitations provides that "every civil action commenced against the United States shall be barred unless the complaint is filed within six years after the right of action first accrues." See 28 U.S.C. § 2401(a). The EPA promulgated the final version of 40 C.F.R. § 122.3(a) in 1979. The EPA denied plaintiffs' Petition for Rulemaking in September 2003. Plaintiffs filed suit in December 2003. Whether their first cause of action is barred by the statute of limitations depends on whether their right of action first accrued in the 1970s when the EPA promulgated the regulation or in 2003 when the EPA denied plaintiffs' petition. The EPA conceded in the district court and in its brief to this court that the statute of limitations does not bar plaintiffs' second cause of action. See Nw. Envtl. Advocates I, 2005 WL 756614, at *7, 2005 U.S. Dist. LEXIS 5373, at *22.
Our decision in Wind River Mining Corp. v. United States, 946 F.2d 710 (9th Cir.1991), controls our analysis. In Wind River, the Bureau of Land Management ("BLM") had classified certain federal lands as Wilderness Study Areas *1019 ("WSAs") in 1979. Mining was forbidden in a WSA. Id. at 711. In 1986 and 1987, the Wind River Mining Corporation asked the BLM to declare that its decision to create a WSA was invalid. Id. The BLM denied the request, and the Interior Board of Land Appeals denied Wind River's administrative appeal in 1987. Wind River filed suit in 1989 alleging that the BLM's action in creating the WSA was ultra vires. Id. at 712. We held that the six-year statute of limitations began to run with the final administrative action denying Wind River's request:
[A] substantive challenge to an agency decision alleging lack of agency authority may be brought within six years of the agency's application to the specific challenger.... The right to bring a civil suit challenging an agency action accrues "upon the completion of the administrative proceedings." The BLM finally rejected Wind River's attempts to have WSA 243 declared invalid in 1987.... Wind River's complaint for review was filed less than twenty-nine months later, and therefore was timely.
Id. at 716 (citations omitted); see also Legal Envtl. Assistance Found., Inc. v. U.S. EPA, 118 F.3d 1467, 1473 (11th Cir. 1997); Pub. Citizen v. Nuclear Regulatory Comm'n, 901 F.2d 147, 150-53 (D.C.Cir. 1990); NLRB Union v. Fed. Labor Relations Auth., 834 F.2d 191, 194-97 (D.C.Cir. 1987); Natural Res. Def. Council v. Nuclear Regulatory Comm'n, 666 F.2d 595, 601-03 (D.C.Cir.1981); Oppenheim v. Campbell, 571 F.2d 660, 663 (D.C.Cir.1978); cf. Bennett v. Spear, 520 U.S. 154, 177-78, 117 S.Ct. 1154, 137 L.Ed.2d 281 (1997) (defining what constitutes a "final" agency action).
Our case is indistinguishable from Wind River. Plaintiffs asked the EPA to repeal § 122.3(a) in their 1999 Petition for Rulemaking, and the EPA denied that request in 2003. Plaintiffs filed suit a few months after that denial, alleging that EPA had acted ultra vires in promulgating § 122.3(a). The EPA's denial of the Petition for Rulemaking in 2003 was thus an "adverse application" of § 122.3(a) within the meaning of Wind River. See 946 F.2d at 714-16. The date of that decision is the date of first accrual for purposes of the statute of limitations under § 2401(a). We therefore conclude that plaintiffs' suit was timely filed in the district court.

C. Ultra Vires Challenge
In their first cause of action, plaintiffs allege that the CWA does not authorize the exemptions of vessel discharges provided in 40 C.F.R. § 122.3(a). According to plaintiffs, the EPA acted ultra vires in promulgating § 122.3(a). See 5 U.S.C. § 706(2)(C) (covering agency actions "in excess of statutory jurisdiction, authority, or limitations, or short of statutory right"). If plaintiffs are right, the regulation is invalid. In their second cause of action, plaintiffs allege that the EPA did not act "in accordance with law" when the agency denied the 1999 Petition for Rulemaking asking the EPA to repeal § 122.3(a). See 5 U.S.C. § 706(2)(A). As in their first cause of action, the premise of the second cause of action is that the EPA acted ultra vires in promulgating § 122.3(a). Because both causes of action present a question of law, we start at step one of Chevron and apply the same standard of review. See, e.g., Defenders of Wildlife v. Browner, 191 F.3d 1159, 1162 (9th Cir.1999) ("On questions of statutory interpretation, we follow the approach from Chevron.").
The EPA makes three arguments. The first is procedural; the second and third are substantive. First, the EPA argues that the 1999 Petition for Rulemaking challenged only the exclusion for ballast water provided by 40 C.F.R. § 122.3(a). Therefore, the EPA argues, plaintiffs are now limited to challenging only this exclusion. *1020 Second, the EPA argues that the CWA authorized the EPA to promulgate § 122.3(a), or that at least the statute is ambiguous and therefore this court should defer to the agency's interpretation of the statute. Third, the EPA argues that even if the CWA did not authorize the promulgation of § 122.3(a) when the CWA was enacted, Congress has now acquiesced in its promulgation. We consider these arguments in turn.

1. Scope of Plaintiffs'1999 Petition for Rulemaking
The EPA argues that at most we should vacate § 122.3(a) as it applies to ballast water discharges. The agency argues that we should not address the exemptions for marine engine and graywater discharges or discharges incidental to the normal operation of a vessel other than ballast water because plaintiffs did not object to those exemptions in their 1999 Petition for Rulemaking to the EPA. The district court considered and rejected this argument. See, e.g., Nw. Envtl. Advocates II, 2006 WL 2669042, at *6, 2006 U.S. Dist. LEXIS 69476, at *24 ("Plaintiffs have consistently made clear that their overall aim is the repeal of the exemptions contained in 40 C.F.R. § 122.3(a).").
We agree with the district court. It is clear that the plaintiffs always have been most concerned with the environmental effects of ballast water discharges, but it is equally clear that they challenged all three exemptions contained in § 122.3(a) when they petitioned the EPA in 1999. For example, Plaintiffs' Petition for Rulemaking was titled "Petition for Repeal of 40 CFR § 122.3(a)." In that petition, they challenged the exemption in § 122.3(a) of "ballast water discharges and other discharges." In responding to plaintiffs' petition, the EPA stated that its decision addressed a "Petition for Rulemaking to Repeal 40 C.F.R. 122.3(a)." The EPA's denial of plaintiffs' petition quoted the full text of § 122.3(a) and explicitly noted that plaintiffs sought a repeal of the entire regulation. "The record in this case is replete with evidence" that the plaintiffs' position was clear to the EPA. `Ilio`ulaokalani Coal. v. Rumsfeld, 464 F.3d 1083, 1092 (9th Cir.2006) (as amended).

2. Text of the CWA
Our first substantive inquiry is whether § 122.3(a) is invalid under the plain meaning of the CWA. Our inquiry is guided by Chevron. The Court wrote:
When a court reviews an agency's construction of the statute which it administers, it is confronted with two questions. First, always, is the question whether Congress has directly spoken to the precise question at issue. If the intent of Congress is clear, that is the end of the matter; for the court, as well as the agency, must give effect to the unambiguously expressed intent of Congress.
467 U.S. at 842-43, 104 S.Ct. 2778.
Section 301(a) of the CWA mandates that "the discharge of any pollutant by any person shall be unlawful." 33 U.S.C. § 1311(a). This prohibition is "[t]he `cornerstone' and `fundamental premise' of the Clean Water Act." Se. Alaska Conservation Council v. U.S. Army Corps of Eng'rs, 486 F.3d 638, 644 (9th Cir.2007) (citations omitted). Section 402 of the CWA provides that a "point source" can obtain a "permit for the discharge of any pollutant or combination of pollutants." 33 U.S.C. § 1342(a)(1). "[T]he Act categorically prohibits any discharge of a pollutant from a point source without a permit." Comm. to Save Mokelumne River v. E. Bay Mun. Util. Dist., 13 F.3d 305, 309 (9th Cir.1993).
*1021 The text of the statute clearly covers the discharges at issue here. A "discharge of any pollutant" is "any addition of any pollutant to navigable waters from any point source." 33 U.S.C. § 1362(12)(A). A "point source" is "any discernable, confined and discrete conveyance, including... [a] vessel or other floating craft, from which pollutants are or may be discharged." Id. § 1362(14). "[N]avigable waters" are "the waters of the United States, including the territorial seas," which begin near the coast and "extend[ ] seaward a distance of three miles." Id. §§ 1362(7), (8). "Pollutant" is defined as "dredged spoil, solid waste, incinerator residue, sewage, garbage, sewage sludge, munitions, chemical wastes, biological materials, radioactive materials, heat, wrecked or discarded equipment, rock, sand, cellar dirt and industrial, municipal, and agricultural waste discharged into water." 33 U.S.C. § 1362(6). The term "biological materials" includes invasive species. See, e.g., Nat'l Wildlife Fed'n v. Consumers Power Co., 862 F.2d 580, 583 (6th Cir.1988).
The question before us of whether the CWA authorizes the EPA's regulatory exemptions was answered by the D.C. Circuit more than thirty years ago. See Natural Res. Def. Council v. Costle ("Costle"), 568 F.2d 1369 (D.C.Cir.1977). The same year that the EPA issued the regulation in our case, the agency promulgated a different but conceptually identical regulation. Costle addressed an ultra vires challenge to that regulation.
The regulation entirely exempted several categories of point sources from NPDES requirements:
all silviculture point sources; all confined animal feeding operations below a certain size; all irrigation return flows from areas less than 3,000 contiguous acres or 3,000 noncontiguous acres that use the same drainage system; all nonfeedlot, nonirrigation agricultural point sources; and separate storm sewers containing only storm runoff uncontaminated by any industrial or commercial activity.
Id. at 1372. In a unanimous opinion by Judge Leventhal, the D.C. Circuit held that the EPA acted ultra vires in promulgating this regulation. Id. at 1377, 1382-83.
The analysis of the D.C. Circuit in Costle, with which we agree, is dispositive of our case. The only possible textual source of authority for the exemptions at issue in Costle (and in our case) is section 402 of the CWA. In relevant part, that section provides that the EPA Administrator
may, after opportunity for public hearing, issue a permit for the discharge of any pollutant, ... notwithstanding section 301(a), upon condition that such discharge will meet either (A) all applicable requirements under sections 301, 302, 306, 307, 308, and 403 of this Act, or (B) prior to the taking of necessary implementing actions relating to all such requirements, such conditions as the Administrator determines are necessary to carry out the provisions of this Act.
33 U.S.C. § 1342(a)(1).
Section 402 uses the word "may," but only in the context of "issu[ing] a permit for the discharge of any pollutant." The Administrator "may" issue a permit under two circumstances: either on the condition that the discharge meets all of the requirements specified in the section; or, prior to implementation of those statutory requirements, on such conditions "as the Administrator determines are necessary to carry out the provisions of [the] Act." That is, section 402 allows the Administrator to issue a permit, but it does not provide that the Administrator may entirely exempt certain categories of discharges from the permitting requirement. As the D.C. Circuit *1022 concluded, "The use of the word `may' in § 402 means only that the Administrator has discretion either to issue a permit or to leave the discharger subject to the total proscription of § 301. This is the natural reading, and the one that retains the fundamental logic of the statute." Costle, 568 F.2d at 1375.
The D.C. Circuit confirmed the correctness of its reading of the CWA by consulting the legislative history of the Act. It wrote, "[T]he legislative history makes clear that Congress intended the NPDES permit to be the only means by which a discharger from a point source may escape the total prohibition of § 301(a)." Id. at 1374. Because the statutory language is unambiguous, we do not need to revisit the legislative history. Congress's intent was clear: "[T]he EPA Administrator does not have authority to exempt categories of point sources from the permit requirements of § 402." Id. at 1377.
We therefore conclude that Congress expressed "a plain ... intent to require permits in any situation of pollution from point sources." Id. at 1383; see also N. Plains Res. Council, 325 F.3d at 1164; Sierra Club v. U.S. EPA, 118 F.3d 1324, 1327 (9th Cir.1997); NRDC 9th Cir.1992, 966 F.2d at 1305, 1310; Forsgren, 309 F.3d at 1190. In its argument to us, the EPA does not seriously contest this conclusion. Rather, the EPA's central argument is that Congress has acquiesced in the EPA's ultra vires action in promulgating § 122.3(a). We now turn to that argument.

3. Acquiescence by Congress
The EPA argues that even if the CWA as originally enacted did not authorize the EPA to promulgate § 122.3(a), Congress subsequently acquiesced in the agency's interpretation of the CWA. This is a heroic argument, for the standard for a judicial finding of congressional acquiescence is extremely high.
In Solid Waste Agency of Northern Cook County v. U.S. Army Corps of Engineers ("SWANCC"), 531 U.S. 159, 162, 121 S.Ct. 675, 148 L.Ed.2d 576 (2001), the Court considered a challenge to an expansive definition of "navigable waters" under the CWA. The Army Corps of Engineers had promulgated a regulation containing that definition in 1977. The Corps argued that Congress had acquiesced in the regulation's definition. Id. at 168-69, 121 S.Ct. 675. The Court responded, "Although we have recognized congressional acquiescence to administrative interpretations of a statute in some situations, we have done so with extreme care." Id. at 169, 121 S.Ct. 675. The Court continued in a footnote:
In Bob Jones Univ. v. United States, 461 U.S. 574, 595, 600-01, 103 S.Ct. 2017, 76 L.Ed.2d 157 (1983), for example, we upheld an Internal Revenue Service (IRS) Revenue Ruling that revoked the tax-exempt status of private schools practicing racial discrimination because the IRS' interpretation of the relevant statutes was "correct"; because Congress had held "hearings on this precise issue," making it "hardly conceivable that Congress  and in this setting, any Member of Congress  was not abundantly aware of what was going on"; and because "no fewer than 13 bills introduced to overturn the IRS interpretation" had failed. Absent such overwhelming evidence of acquiescence, we are loath to replace the plain text and original understanding of a statute with an amended agency interpretation.

Id. at 169-70 n. 5, 121 S.Ct. 675 (emphasis added); see also Rapanos v. United States, 547 U.S. 715, 749, 126 S.Ct. 2208, 165 L.Ed.2d 159 (2006) (plurality op.) (noting the Court's "oft-expressed skepticism towards reading the tea leaves of congressional *1023 inaction"); Morales-Izquierdo v. Gonzales, 486 F.3d 484, 493 (9th Cir.2007) (en banc) (finding no acquiescence under SWANCC's "overwhelming evidence" standard).
The EPA points to a number of post-1973 statutes in which Congress has addressed the forms of pollution exempted by § 122.3(a), particularly ballast water. According to the EPA, those statutes satisfy the high standard for acquiescence set forth in SWANCC. For the reasons that follow, we disagree and hold that Congress has not acquiesced in § 122.3(a).

a. NDAA and DSHMRA
The EPA relies most heavily on two statutes. The first is the National Defense Authorization Act of 1996 ("NDAA"), Pub.L. No. 104-106, § 325, 110 Stat. 186, 254, codified at 33 U.S.C. §§ 1322(a), (j), (n), 1362(6). The second is the Deep Seabed Hard Mineral Resources Act of 1980 ("DSHMRA"), Pub.L. No. 96-283, 94 Stat. 554, codified at 30 U.S.C. §§ 1419 et seq.
In the NDAA, Congress statutorily exempted discharges incidental to the normal operation of United States military vessels from CWA permitting requirements and established discharge controls specifically tailored to those vessels. Congress was well aware of 40 C.F.R. § 122.3(a) when it enacted the NDAA. Indeed, the statute cited the regulation as a partial aid in defining what the category "discharge incidental to the normal operation of a vessel" did not include. See 33 U.S.C. § 1322(a)(12)(B)(iii).
A Senate Report accompanying the Senate Bill explained that discharges from military vessels, like those from other vessels, already were exempted from NPDES permitting requirements by EPA regulation. But the report went on to explain why, nonetheless, a broader exemption was desirable:
The Navy wishes to clarify the regulatory status of certain non-sewage discharges from Navy vessels. Vessels are point sources of pollution under the Clean Water Act. Any discharge of pollutants from a point source, including a vessel, into the waters of the United States is prohibited unless specifically permitted under section 402 or 404 of the Act....
Although EPA regulations generally exempt nonsewage discharges from vessels from the permit requirements of the Act, some coastal states have imposed regulations or inspection programs that may have application to these types of discharges. A series of events in the waters of several coastal states prompted concern at the Navy as to state authorities to regulate these discharges.
S.Rep. No. 104-113, at 1-2 (1995). The Senate Report explained that § 122.3(a) was the regulatory basis for the exemption of most "non-sewage discharges from vessels." Id. at 7. The report did not, however, endorse or otherwise indicate approval of regulatory exemptions for entire categories of marine discharges. If anything, the report may be read to suggest the contrary. The report indicated that, but for the statutory exemption contained in the NDAA, the CWA permitting process would have applied to marine discharges from military vessels: "The effect of [the NDAA] is to remove the statutory requirement for a permit for these point source discharges[.]" Id. at 3.
The most that can be said, based on the NDAA, is that Congress was well aware of § 122.3(a) and the exemptions it provided. Congress concluded that the existing statutory provisions and exemptions, including the exemptions provided in § 122.3(a), did not fully address the needs of military vessels. It therefore passed a new statute with provisions specifically tailored to military *1024 vessels. In so doing, the NDAA did nothing to endorse § 122.3(a). The NDAA only made § 122.3(a) irrelevant to military vessels except as a definitional tool.
In the DSHMRA, Congress required vessels engaged in deep sea mining and drilling operations to comply with the provisions of the CWA. Congress did so by explicitly extending the CWA's geographical reach over such vessels beyond the otherwise applicable three-mile limit. See 33 U.S.C. § 1362(9), (10), (12)(B). In pertinent part, the DSHMRA provided that:
For purposes of this chapter, any vessel or other floating craft engaged in commercial recovery or exploration shall not be deemed to be "a vessel or other floating craft" under section 502(12)(B) of the Clean Water Act [33 U.S.C. § 1362(12)(B)] and any discharge of a pollutant from such vessel or other floating craft shall be subject to the Clean Water Act.
30 U.S.C. § 1419(e) (alterations in original).
When it enacted the DSHMRA, Congress noted with approval the final sentence of 40 C.F.R. § 122.3(a). This sentence provides that, despite the regulatory exemptions for three categories of marine discharges, CWA permitting requirements would apply to a range of vessels not being used for transportation:
This exclusion does not apply to ... discharges when the vessel is operating in a capacity other than as a means of transportation such as when used as an energy or mining facility, a storage facility or a seafood processing facility, or when secured to a storage facility or a seafood processing facility, or when secured to the bed of the ocean, contiguous zone or waters of the United States for the purpose of mineral or oil exploration or development.
40 C.F.R. § 122.3(a). Plaintiffs do not challenge this part of the regulation because it exempts nothing, but instead recognizes ongoing NPDES requirements. See Nw. Envtl. Advocates II, 2006 WL 2669042, at *1 nn. 1-2, 2006 U.S. Dist. LEXIS 69476, at *2-3 nn. 1-2.
The Senate Report accompanying the DSHMRA noted with approval the refusal of § 122.3(a) to exempt nontransportation vessels from NPDES:
[T]he Environmental Protection Agency has concluded that the Congress did not intend to exempt pollutant discharges into ocean waters by vessels when engaged in such activities as mining or drilling for oil, etc. Relying on this interpretation [of the CWA], the Environmental Protection Agency amended [its regulations] to indicate that vessels engaged in ocean mineral exploration, extraction and processing activities are not exempt from permit requirements under section 402. The Committee concurs in this interpretation.
S.Rep. No. 96-360 at 2-3 (1979); see also id. at 3 (noting that the DSHMRA merely "clarif[ied] the application of section 402" to these vessels). Thus, the most that can be said of the DSHMRA is that Congress was aware of § 122.3(a) and explicitly approved of the EPA's decision not to exempt from the permitting process marine discharges from nontransportation vessels.
We conclude that neither the NDAA nor the DSHMRA comes close to satisfying the SWANCC standard of providing "overwhelming evidence of acquiescence" by Congress in § 122.3(a)'s exemption of three categories of marine discharges.

b. NANPCA, NISA, APPS, and Alaska Cruise Ship Legislation
The EPA also relies on four additional statutes. They are the Nonindigenous Aquatic Nuisance Prevention and Control Act of 1990 ("NANPCA"), Pub.L. No. 101-646, 104 Stat. 4761, codified at 16 U.S.C. *1025 §§ 4701 et seq.; the National Invasive Species Act of 1996 ("NISA"), Pub.L. No. 104-332, 110 Stat. 4073 (amending NANPCA); the Act to Prevent Pollution from Ships ("APPS"), Pub.L. No. 96-478, 94 Stat. 2297 (1980), codified at 33 U.S.C. §§ 1901 et seq.; and a statute regulating discharges by Alaska cruise ships, enacted as part of the Consolidated Appropriations Act of 2001, Pub.L. No. 106-554, § 1(a)(4), 114 Stat. 2763, 2763A-209 (enacting Title XIV of Division B of H.R. 5666, §§ 1401-1414, as introduced Dec. 15, 2000) (see 33 U.S.C. § 1901 Note for the text of the statute).
NANPCA and NISA address the problem of invasive species released in ballast-water-related discharges. For example, these statutes authorize the Coast Guard to develop voluntary guidelines and regulations for a Great Lakes ballast water program. See 16 U.S.C. § 4711(a)-(b). The statutes also require national guidelines for ballast-water-related discharges of nonindigenous species, id. § 4711(c), (f)(2)(A)(ii), and establish an Aquatic Nuisance Species Task Force, of which the EPA is a member, id. § 4721. Savings clauses provide that the Great Lakes regulations "shall .... not affect or supersede any requirements or prohibitions pertaining to the discharge of ballast water" under the CWA, and that the national guidelines "shall .... not affect or supercede any requirements or prohibitions pertaining to the discharge of ballast water" under the CWA. Id. § 4711(b)(2)(C), (c)(2)(J). These statutes do not demonstrate SWANCC's "overwhelming evidence of [congressional] acquiescence" in the exemptions contained in § 122.3(a). They merely demonstrate a congressional intent to address the serious national problem of ballast water discharges of invasive species, and to do so on multiple, nonexclusive fronts. The Supreme Court recently came to similar conclusions regarding Congress's overlapping mandates to combat greenhouse gas emissions. See Massachusetts v. EPA, ___ U.S. ___, 127 S.Ct. 1438, 1448-49, 1460-62, 1461 n. 27, 167 L.Ed.2d 248 (2007).
The APPS implemented the International Convention for the Prevention of Pollution from Ships of 1973 and the Protocol of 1978 (known collectively as "MARPOL 73/78"). The APPS applies to all U.S.-flagged ships worldwide and foreign-flagged ships in the navigable waters of the United States. 33 U.S.C. § 1902(a). The six annexes to MARPOL 73/78 address vessel discharges of oil, noxious bulk liquid substances, harmful packaged substances, sewage, garbage, and air pollution. The APPS's savings clause provides that "requirements of this [Act] supplement and neither amend nor repeal any other provisions of law, except as expressly provided in this [Act]." 33 U.S.C. § 1907(f). The APPS contains no indication of congressional intent to acquiesce in § 122.3(a).
Finally, the Alaska cruise ship legislation authorizes the EPA to regulate sewage and graywater discharges from cruise ships in specified Alaskan waters. A savings clause provides that "[n]othing in this title shall be construed as restricting, affecting, or amending any other law or the authority of any department, instrumentality, or agency of the Unites States." 33 U.S.C. § 1901 Note § 1411(a); see H.R. 5666, § 1411(a). This legislation, too, contains no indication of congressional intent to acquiesce in § 122.3(a).

D. Remedy
After finding that the EPA had acted ultra vires in promulgating § 122.3(a), the district court concluded that the best course was to vacate that regulation, effective September 30, 2008. This date gave the EPA a two-year period during which it could work to promulgate a new regulation. The district court also concluded *1026 that it would be best to leave the EPA free during this period to do its work in the manner the agency thought best. In so concluding, the district court did not provide to plaintiffs everything they had sought. Plaintiffs had asked the district court to provide only an eighteen-month period, and to engage in close supervision of the EPA's progress during that period.
The district court explained its reasons in a careful twenty-one page order. It wrote, inter alia:
[T]he Court is influenced by the fact that the regulation at issue has stood for the past 30 years, and by the fact that the effects of an immediate vacatur would be so dramatic as to make such an option a practical impossibility. Indeed, not even plaintiffs request an immediate vacatur of the challenged regulation. While the practical implications of the Court's order make the Court wary of imposing a deadline on EPA that is too ambitious, the potential harm that ballast waters represent to our nation's ecosystems leads the Court to conclude that there is an urgency to promulgating new regulations that EPA has not, to this point in the litigation, acknowledged. Thus, the Court must decide upon a time frame for vacating the regulation that balances the need for prompt action against the need to allow EPA adequate freedom to address a complicated issue.
The most substantial question confronting the Court is whether to issue injunctive relief ordering EPA to act in accordance with the Court's order by a certain date. In light of the arguments the parties have presented, the Court finds that the preferable route is to give the agency a certain date on which the regulation will be vacated, and to allow the agency freedom to work around that date to find an appropriate solution to the problem of vessel discharges. Indeed, in considering the variety of technical arguments the parties have presented about the appropriate remedy, the Court has been reminded that EPA holds an expertise in this area that the Court cannot approach. Thus, the Court believes that EPA should be given wide latitude, within broad constraints, to address the problem of discharges from vessels. Accordingly, the Court rules as follows: the Court will GRANT plaintiffs' motion for a permanent injunction, and will set aside the challenged regulation as of September 30, 2008. Absent a compelling justification, the Court will not act further to supervise how EPA responds to this order.
Nw. Envtl. Advocates II, 2006 WL 2669042, at *10, 2006 U.S. Dist. LEXIS 69476, at *31-33 (footnotes omitted).
We affirm the district court's decision to vacate the regulation and to remand for further proceedings as a valid exercise of its remedial powers. See, e.g., NRDC 9th. Cir.1992, 966 F.2d at 1305. The district court's order requires the EPA to perform a substantial task  to bring the discharges previously exempted by § 122.3(a) within the permitting process of the CWA. Neither the district court nor this court underestimates the magnitude of the task. But "this ambitious statute is not hospitable to the concept that the appropriate response to a difficult pollution problem is not to try at all." Costle, 568 F.2d at 1380; see also Union Elec. Co. v. EPA, 427 U.S. 246, 268-69, 96 S.Ct. 2518, 49 L.Ed.2d 474 (1976) ("Allowing such [feasibility] claims to be raised ... would frustrate congressional intent.").
The EPA informed this court at oral argument that it has been proceeding in accordance with the district court's order. We anticipate that in formulating a new regulation to replace § 122.3(a) the EPA will take advantage of the flexibility of the NPDES permitting process. For example, we take judicial notice of the fact *1027 that, in its request for comments, the EPA has indicated that "use of general permit(s) would appear to be an attractive possibility." Development of[NPDES] Permits for Discharges Incidental to the Normal Operation of Vessels, 72 Fed.Reg. 34,241, 34,247 (June 21, 2007).
On July 11, 2008, the Department of Justice informed us by letter that on June 17, 2008, the EPA published in the Federal Register draft "General Permits for Discharges Incidental to the Normal Operation of a Vessel," and that the public comment period on the draft is scheduled to close on August 1. See 73 Fed.Reg. 34,296 (June 17, 2008). The letter warns that a final version may not be ready by the September 30, 2008, deadline established by the district court, but the letter stops short of a request to extend the deadline. If the government chooses to request an extension of the deadline, that request should be addressed to the district court.

IV. Petition for Review
Because we hold that the district court had jurisdiction over plaintiffs' suit filed in that court, we hold that we do not have jurisdiction over plaintiffs' petition for review filed directly in this court. We therefore dismiss that petition for lack of subject matter jurisdiction. See, e.g., Appalachian Energy Group v. EPA, 33 F.3d 319, 322-23 (4th Cir.1994) (dismissing the petition for review for lack of subject matter jurisdiction under 33 U.S.C. § 1369(b)(1)); Am. Paper Inst., Inc. v. U.S. EPA, 890 F.2d 869, 878 (7th Cir.1989) (same).

V. Conclusion
We hold that the district court had subject matter jurisdiction over plaintiffs' suit alleging that the EPA acted ultra vires in promulgating § 122.3(a). We affirm the district court, holding that the EPA acted ultra vires in promulgating § 122.3(a) and that EPA's denial of plaintiffs' 1999 petition requesting the repeal of § 122.3(a) was not in accordance with law. We affirm the district court's remedial order as a proper exercise of its discretion. Finally, we dismiss for lack of subject matter jurisdiction plaintiffs' petition for review filed directly with this court.
AFFIRMED and DISMISSED.
