
616 F.3d 844 (2010)
Tyrone WHITE, Appellant,
v.
Lynn DINGLE, Appellee.
No. 09-1415.
United States Court of Appeals, Eighth Circuit.
Submitted: November 19, 2009.
Filed: August 13, 2010.
*845 Jennifer Marie Macaulay, Macaulay Law Offices, argued, St. Paul, MN, for Appellant.
Tyrone White, Minnesota Department of Corrections, Bayport, MN, pro se.
Kimberly Parker, Attorney General's Office, Dale O. Harris, County Attorney's Office, argued, Duluth, MN, for Appellee.
Before WOLLMAN, JOHN R. GIBSON, and SHEPHERD, Circuit Judges.
WOLLMAN, Circuit Judge.
In 2001, Tyrone White was convicted in Minnesota state court of felony murder and attempted first-degree murder and was sentenced to life imprisonment. In a prior appeal we affirmed the district court's dismissal of White's petition for a writ of habeas corpus because the petition contained claims that had not been fully exhausted in state court. See White v. Dingle, 267 Fed.Appx. 489 (8th Cir.2008) (unpublished). White subsequently returned to the district court and attempted to amend his petition to eliminate the unexhausted claims and have his remaining exhausted claims adjudicated on the merits. The district court concluded, however, that White could not proceed with his remaining claims because the statute of limitations had expired, White's original petition had been dismissed without prejudice, and White's amended petition could not relate back to the date when the original petition had been filed. The district court then granted White a certificate of appealability on the latter determination whether an amended habeas petition could relate back to the date of an original petition that was dismissed without prejudice.
We affirm the district court's conclusion on the issue certified for appeal, joining our sister circuits in holding that a second or amended habeas petition cannot relate back to the time at which the original petition was filed. But we also conclude that under the unusual circumstances presented here, White is entitled to equitable tolling of the statute of limitations, and we remand the case with instructions for the district court to consider White's exhausted claims on the merits.

I.
In order to explain the procedural difficulty that White currently faces, it is necessary to lay out in some detail the facts surrounding his attempt to obtain federal habeas review. When White filed his original habeas petition on April 7, 2006, he had more than eight months remaining before the Antiterrorism and Effective Death Penalty Act's (AEDPA) one-year statute of limitations was set to expire on *846 January 1, 2007. In December 2006, roughly one month before the expiration of the limitations period, a magistrate judge issued a report and recommendation that found that White's habeas petition included at least one unexhausted claim and was thus subject to dismissal without prejudice. The magistrate judge also found that White did not qualify for a stay of the limitations period while he pursued his unexhausted claims in state court, and she further observed that the statute of limitations would prevent White from refiling if his petition was dismissed. White appealed the magistrate judge's exhaustion finding to the district court.
Approximately one month after the statute of limitations had expired, the district court adopted the magistrate judge's report and recommendation and dismissed White's petition without prejudice. The district court then granted White a certificate of appealability on the question whether his claims were exhausted, finding the issue to be a debatable question warranting appellate review. The district court additionally found that the unexhausted claims appeared to be at least facially viable.
As mentioned above, we affirmed the district court's conclusion that White's petition included at least one unexhausted claim and that his petition was therefore subject to dismissal without prejudice. Critically, however, our opinion failed to address White's alternative argument that, if we affirmed the district court, we should remand his case with instructions for the district court to allow him to voluntarily dismiss the unexhausted claims and have the remaining exhausted claims adjudicated on the merits. The opinion was silent on the procedure that should govern White's case going forward.
Immediately following our decision, White returned to the district court and filed a motion to amend his habeas petition to dismiss any unexhausted claims. His promptness in taking this action cannot be questioned, as the motion was in fact filed before the mandate had issued from our decision. Recognizing that the statute of limitations was by then long-expired, the district court denied White's motion to amend his petition and certified the question noted above.

II.
Several of the challenges that White has faced in obtaining merits review of his federal habeas petition stem from the interaction of long-standing Supreme Court precedent and the one-year statute of limitations under the AEDPA. In Rose v. Lundy, 455 U.S. 509, 510, 102 S.Ct. 1198, 71 L.Ed.2d 379 (1982), the Supreme Court held that a habeas petitioner cannot proceed in federal court with a "mixed petition," that is, one in which some claims have been fully exhausted in state court and others have not. The Court held that a mixed petition should be dismissed without prejudice and that the petitioner should be given "the choice of returning to state court to exhaust his claims or of amending or resubmitting the habeas petition to present only exhausted claims to the district court." Id. Because Rose v. Lundy was decided before the AEDPA's one-year statute of limitations was established, however, it did not contemplate that a dismissal without prejudice after expiration of the AEDPA limitations period could eliminate access to federal habeas review. The potential peril for the unwary habeas petitioner is exacerbated by the fact that the limitations period is not tolled while a petition is pending in federal court. See Duncan v. Walker, 533 U.S. 167, 181-82, 121 S.Ct. 2120, 150 L.Ed.2d 251 (2001). A petitioner like White, for example, who files his petition well before the expiration of the AEDPA's one-year deadline may find that the limitations period has expired *847 while his case was under consideration. In such a situation, dismissal without prejudice does not provide a petitioner with the options that the Court envisioned in Rose v. Lundy; rather, it leaves the petitioner both out of court and out of time to file another petition.
The Supreme Court considered this issue in Rhines v. Weber, 544 U.S. 269, 278, 125 S.Ct. 1528, 161 L.Ed.2d 440 (2005), and concluded that when dismissal of a mixed petition without prejudice might unreasonably impair a petitioner's opportunity to obtain federal habeas review, "the court should allow the petitioner to delete the unexhausted claims and to proceed with the exhausted claims." In other words, the petitioner should be given the opportunity to have his exhausted claims considered on the merits. White's case presents exactly the situation that was discussed in Rhines v. Weber: the district court's dismissal of his petition without prejudice has placed him in a position where he will be unable to obtain any federal review of his claims.

III.
The issue certified for appeal is whether an amended habeas petition should be allowed to relate back to the date of an original petition that was dismissed without prejudice. We conclude that the answer to that question is no. As the district court noted, every circuit to address this issue has refused to allow an amended petition to relate back. See, e.g., Marsh v. Soares, 223 F.3d 1217, 1220 (10th Cir.2000) ("We therefore join with all the circuit courts which have addressed this issue, and hold that a habeas petition filed after a previous petition has been dismissed without prejudice for failure to exhaust state remedies does not relate back to the earlier petition.").
There are persuasive theoretical and practical justifications for this outcome. Theoretically, there is nothing to which such an amendment can relate back, because the dismissed petition is no longer pending. From a technical standpoint, the original petition no longer exists and it is as if the original petition had never been filed. See Neverson v. Bissonnette, 261 F.3d 120, 126 (1st Cir.2001). From a practical standpoint, permitting relation-back risks "eviscerat[ing] the AEDPA limitations period and thwart[ing] one of AEDPA's principal purposes," which was to expedite federal habeas review. Graham v. Johnson, 168 F.3d 762, 780 (5th Cir. 1999). Courts rightly fear that permitting relation-back would allow petitioners to use an original petition as a placeholder, thereby indefinitely tolling the statute of limitations. The end result of such an approach would be an exception that threatens to swallow the entire rule. Accordingly, the district court properly determined that White's amended habeas petition could not relate back to the date his original petition was filed.
This conclusion does not end our inquiry, however. We have the discretion to consider issues outside the scope of the certificate of appealability, and although we exercise that discretion carefully, we will do so when it is in the interest of justice. See King v. United States, 595 F.3d 844, 854 n. 5 (8th Cir.2010). Under the doctrine of equitable tolling, the AEDPA's statutory limitations period may be tolled if a petitioner can show that (1) he has been diligently pursuing his rights and (2) an extraordinary circumstance stood in his way. Holland v. Florida, ___ U.S. ___, 130 S.Ct. 2549, 2562, 177 L.Ed.2d 130 (2010). Equitable tolling is a flexible procedure that involves both recognition of the role of precedent and an "awareness of the fact that specific circumstances, often hard to predict in advance, could warrant *848 special treatment in an appropriate case." Id. at 2563.
The state does not contend that White has failed to diligently pursue his claims. White filed his habeas petition more than eight months before the statute of limitations expired, and he has been steadily litigating his case ever since, all with the goal of having his claims considered on the merits. After we affirmed the district court's exhaustion determination, White promptly returned to the district court and unsuccessfully sought to amend his petition. Thus, we conclude that White has satisfied the diligence requirement and we turn to the question whether White has demonstrated an extraordinary circumstance that warrants equitable relief.
White's current inability to obtain federal review of his exhausted claims is a direct result of our failure to remand his prior appeal with instructions that the district court consider those claims on the merits.[1] Had we done so, the district court could have given White an opportunity to proceed with his exhausted claims rather than having his entire petition dismissed. There is no indication that our failure to remand White's case was anything other than an oversight. Indeed, such a remand would have been fully consistent with the Supreme Court's directive in Rhines v. Weber to avoid dismissal without prejudice when it would unreasonably jeopardize a petitioner's access to review of his exhausted claims.
We previously endorsed remand under similar circumstances in Underdahl v. Carlson, 381 F.3d 740, 744 (8th Cir.2004), a case in which the petitioner appealed the district court's dismissal of his petition for failure to exhaust state remedies. We observed that "[o]ur case law indicates that a petitioner who presents a mixed petition should be allowed to amend his petition to include only his exhausted claims." Id. Thus, we remanded the case to the district court "with instructions to allow the petitioner to withdraw his unexhausted claims and to proceed to the merits of his exhausted claims." Id.; see also Jackson v. Dormire, 180 F.3d 919, 920 (8th Cir.1999) (per curiam) (affirming the district court's exhaustion determination but remanding to allow petitioner to delete his unexhausted claims); Murray v. Wood, 107 F.3d 629, 632 (8th Cir.1997) (remanding with instructions to afford the petitioner a reasonable opportunity to dismiss his unexhausted claims). Several other circuits have followed a similar course after concluding that the petitioner's appeal involved a mixed petition. See, e.g., Clements v. Maloney, 485 F.3d 158, 168-69 (1st Cir.2007) (remanding a mixed petition to give petitioner the opportunity to dismiss his unexhausted claims); Valerio v. Crawford, 306 F.3d 742, 771 (9th Cir.2002) (en banc) (same); Rockwell v. Yukins, 217 F.3d 421, 425 (6th Cir.2000) (same).
We conclude that our failure to remand White's petition after his first appeal qualifies as an extraordinary circumstance. As noted above, White specifically requested that his petition be remanded in the event that we affirmed the district court, and *849 there was no reason for him to believe that we would not follow that course. The state contends that White is somehow responsible for his current situation because when the magistrate judge initially concluded that White's petition contained unexhausted claims, she alerted White to the fact that he faced a statute of limitations problem. According to the state, the magistrate judge's report and recommendation placed White on notice that he needed to voluntarily dismiss his unexhausted claims before the limitations period expired.
The state essentially maintains that White gambled and lost, and that his misstep eliminates any entitlement to federal review of his exhausted claims. This argument, however, assumes that White was required to decide between accepting the magistrate judge's initial ruling on the exhaustion question or running the risk that, if that ruling was later affirmed, he would forfeit any further federal review. We can see no reason why a petitioner should be required to make that decision in a case like thiswhere the district court later determined that the unexhausted claims were facially viable and that its exhaustion analysis involved a debatable question warranting appellate review. White can hardly be faulted for appealing such an important determination.
Moreover, even if the rule that the state proposes were a desirable one for future cases, there was nothing that would have put White on notice of the risk that he faced in appealing the magistrate judge's report and recommendation. To the contrary, the cases we have discussed support the conclusion that a petitioner in White's situation may seek appellate review of the district court's exhaustion determination without fear that an unsuccessful appeal will preclude any further review. Accordingly, we conclude that White has demonstrated an extraordinary circumstance that warrants tolling of the limitations period.

IV.
We affirm the judgment of the district court on the question certified for appeal, and we remand the case with instructions for the court to allow White to amend his petition and proceed with his exhausted claims.
NOTES
[1]  The state contends that our failure to remand binds us under the law of the case doctrine, arguing that we implicitly denied White the opportunity to amend his petition after his first appeal. We disagree. Our previous decision addressed only the district court's exhaustion determination and was silent on the procedure that should govern White's continued pursuit of habeas relief. Because resolution of that procedural question was not a necessary part of our holding, the law of the case doctrine does not apply. See UniGroup, Inc. v. Winokur, 45 F.3d 1208, 1211 (8th Cir.1995) (observing that the law of the case doctrine applies to matters that have been explicitly or implicitly decided at an earlier stage in the same case).
