
624 F.3d 678 (2010)
Rafael RODRIGUEZ-BARAJAS, Petitioner,
v.
Eric H. HOLDER, Jr., U.S. Attorney General, Respondent.
No. 09-60351 Summary Calendar.
United States Court of Appeals, Fifth Circuit.
October 19, 2010.
*679 Elisabeth S. Brodyaga, Refugio del Rio Grande, San Benito, TX, for Petitioner.
Michael Christopher Heyse, Trial Atty., Tangerlia Cox, U.S. Dept. of Justice, OIL, Washington, DC, for Respondent.
Before DAVIS, SMITH and SOUTHWICK, Circuit Judges.
JERRY E. SMITH, Circuit Judge:
Rafael Rodriguez-Barajas petitions for review of a decision of the Board of Immigration Appeals ("BIA"), arguing that the BIA erred in holding that it lacked jurisdiction to hear his appeal because he had voluntarily left the country while his habeas corpus petition was pending in federal court. We must decide whether an alien subject to removal proceedings who voluntarily departs the United States after the BIA has issued a decision on his appeal, but while his habeas petition is pending, is deemed to have withdrawn his appeal pursuant to 8 C.F.R. § 1003.4 (2010). We conclude that the regulation does not apply to departures occurring after a BIA decision on appeal and while a habeas petition is pending, so the BIA has jurisdiction to review Rodriguez-Barajas's appeal.

I.
In 1990, Rodriguez-Barajas was admitted into the United States as a lawful permanent resident. In 1998, he was convicted in state court of possession of marihuana and sentenced to four years of deferred adjudication. In 2001, he applied for admission as a returning resident alien but was found ineligible for admission because of his conviction. He was served with a notice to appear alleging he was subject to removal for the conviction.
Rodriguez-Barajas appeared with counsel before an immigration judge and conceded the allegations in the notice to appear. After a hearing, the immigration judge ruled that Rodriguez-Barajas was ineligible for cancellation of removal because of his aggravated-felony conviction. The BIA dismissed his appeal for failure to file a brief.
Rodriguez-Barajas filed a habeas petition in federal district court challenging the denial of his application for cancellation of removal. That petition was transferred to this court, which, in 2007, granted the Attorney General's motion to remand to the BIA in light of Lopez v. Gonzales, 549 U.S. 47, 127 S.Ct. 625, 166 L.Ed.2d 462 (2006). On remand, the government submitted documents showing that, during the pendency of his habeas petition, Rodriguez-Barajas had voluntarily removed himself to Mexico. The BIA dismissed the appeal for lack of jurisdiction because, pursuant to 8 C.F.R. § 1003.4, Rodriguez-Barajas's voluntary departure constituted a withdrawal of his appeal. Rodriguez-Barajas filed the instant petition for review in this court.

II.
We review the BIA's rulings of law de novo. Alvarado de Rodriguez v. Holder, 585 F.3d 227, 233 (5th Cir.2009). We may not affirm the BIA's decision except on the basis of the reasons it provided. Galvez-Vergara v. Gonzales, 484 F.3d 798, 803 n. 6 (5th Cir.2007). We must defer to an agency's interpretation of its *680 own regulation "unless an `alternative reading is compelled by the regulation's plain language or by other indications of the Secretary's intent at the time of the regulation's promulgation.'"[1] We grant the BIA's interpretation of its own regulations "considerable legal leeway."[2]
The regulation on which the BIA relied states, in pertinent part:
Departure from the United States of a person who is the subject of deportation or removal proceedings ... subsequent to the taking of an appeal, but prior to a decision thereon, shall constitute a withdrawal of the appeal, and the initial decision in the case shall be final to the same extent as though no appeal had been taken.
8 C.F.R. § 1003.4. Rodriguez-Barajas argues that § 1003.4 is not applicable, because his departure after the BIA's decision on appeal, but while his habeas petition was pending, was not "subsequent to the taking of an appeal, but prior to a decision thereon."
No court appears to have addressed whether the language "subsequent `to the taking of an appeal, but prior to a decision thereon'" covers departures occurring after a decision by the BIA but while a habeas petition is pending. The government argues that this case is controlled by Long v. Gonzales, 420 F.3d 516 (5th Cir. 2005). Long did not, however, address the issue we face, because the petitioner there departed the country before the BIA rendered a decision. Id. at 518. Rather, the issue in Long was whether § 1003.4 covered involuntary as well as voluntary departures. Id. at 520-21. That we considered the petitioner in Long to have waived his appeal, id. at 520, tells us nothing about whether a petitioner does so if his departure follows the BIA's decision.
Without any precedent on point, we turn to the plain language of § 1003.4, which says that if an alien departs "subsequent to the taking of an appeal, but prior to a decision thereon ... the initial decision in the case shall be final to the same extent as though no appeal had been taken." From this language, it is evident that the BIA's decision on appeal is "a decision" under the regulation despite being subject to habeas review. If, according to § 1003.4, the immigration judge's decision counts as a "decision," albeit an "initial" one, despite being subject to BIA review, then surely the BIA's decision to dismiss an appeal counts as "a decision" despite being subject to habeas review. And because the regulation makes a distinction between "initial decision[s]" and those that are "final," it is manifest that a BIA decision on appeal is "a decision" even if it is not "final" in the sense that it is subject to habeas review. Therefore, departure after a BIA decision on appeal, while a habeas petition is pending, is not "prior to a decision" in the appeal.
A contrary interpretation, in which "a decision" would occur only once the case was conclusively decided, would also render the words "prior to a decision thereon" superfluous. This is so because once the case is conclusively decided, there is no longer an appeal to waive. Therefore, any departure subsequent to the taking of an appeal by a person subject to removal proceedings would constitute a waiver of the appeal.
*681 Comparing § 1003.4 to a similar provision, 8 C.F.R. § 1003.2(d) (2010), is also instructive. Under § 1003.2(d), any departure from the United States after filing a motion to reopen or reconsider by a person subject to removal proceedings precludes BIA jurisdiction to consider the motion. The government argues that because the same policies underlie §§ 1003.2(d) and 1003.4, the two sections should be interpreted analogously. But there is a critical difference between the two provisions: Section 1003.2(d) says that "[a]ny departure from the United States... occurring after the filing of a motion to reopen or a motion to reconsider, shall constitute a withdrawal of such a motion." By contrast, § 1003.4 says that a departure constitutes a withdrawal of an appeal only if it occurs "prior to a decision thereon." The presence of such limiting language in § 1003.4, but not in § 1003.2(d), suggests that the former regulation should not be interpreted to mirror the latter.
A contrary holding would also conflict with Nken v. Holder, ___ U.S. ___, 129 S.Ct. 1749, 1761, 173 L.Ed.2d 550 (2009), holding that the burden of removal alone does not by itself constitute irreparable injury for purposes of granting a stay of removal. The Court so held because the Illegal Immigration Reform and Immigrant Responsibility Act of 1996 permitted judicial review of a petition by a removed alien.[3] Without BIA jurisdiction on remand, consideration of a removed alien's habeas petition would be pointless, because the sole remedy available under habeas review in the case of a removed alien is to vacate the removal order.[4] Therefore, contra Nken, removal from the United States would result in irreparable injury, because an alien would not be able to obtain any relief from the BIA on remand after the order was vacated.
The government admits that "there is a tension between this [c]ourt's jurisdiction over petitions for review and the regulation that controls the [BIA's] jurisdiction" and concedes that an alien removed while his habeas petition is pending "could nonetheless pursue his appeal before the [BIA]" on remand. The government argues that the BIA nevertheless does not have jurisdiction over Rodriguez-Barajas's appeal, because his departure from the United States was voluntary. The plain language of § 1003.4, however, makes no distinction between voluntary and involuntary departures[5]neither is covered by § 1003.4 if it does not occur "prior to a decision" on the appeal.
In sum, § 1003.4 unambiguously does not bar the BIA's jurisdiction over the appeal of an alien who departs, whether voluntarily or involuntarily, after the BIA has decided his appeal but while his habeas petition is pending.[6] Therefore, the government's interpretation of the regulation *682 is not entitled to deference. Thomas Jefferson Univ., 512 U.S. at 512, 114 S.Ct. 2381.
The petition for review is GRANTED, the decision of the BIA is VACATED, and this matter is REMANDED for further proceedings as needed.[7]
NOTES
[1]  Thomas Jefferson Univ. v. Shalala, 512 U.S. 504, 512, 114 S.Ct. 2381, 129 L.Ed.2d 405 (1994) (quoting Gardebring v. Jenkins, 485 U.S. 415, 430, 108 S.Ct. 1306, 99 L.Ed.2d 515 (1988)).
[2]  Navarro-Miranda v. Ashcroft, 330 F.3d 672, 675 (5th Cir.2003) (quoting Barnhart v. Walton, 535 U.S. 212, 217, 122 S.Ct. 1265, 152 L.Ed.2d 330 (2002)).
[3]  Nken, 129 S.Ct. at 1761 ("Aliens who are removed may continue to pursue their petitions for review, and those who prevail can be afforded effective relief by facilitation of their return, along with restoration of the immigration status they had upon removal.").
[4]  Zalawadia v. Ashcroft, 371 F.3d 292, 301 (5th Cir.2004) ("[V]acating the deportation order is the beginning and end of the habeas authority we have ....").
[5]  Long, 420 F.3d at 520 (stating that making a distinction between voluntary and involuntary departures "would require us to read into § 1003.4 an exception that it neither expressly nor implicitly provides").
[6]  We need not determine whether § 1003.4 bars BIA jurisdiction in the case of an alien's departure while his remanded appeal is pending before the BIA. We decide only that the BIA has jurisdiction on remand when an alien departs while his habeas petition is pending and before the case has been remanded to the BIA.
[7]  The government argues, in the alternative, that

even assuming the [BIA] had jurisdiction to consider Petitioner's appeal, he is not eligible for any form of relief. Because he relinquished his lawful permanent resident status ..., he is not eligible for cancellation of removal under INA section 240A(a) ..., and he is not eligible for cancellation of removal under INA section 240a(b) ..., because of his controlled substance conviction.
We express no view on that assertion, which the BIA is free to consider on remand. By a post-submission letter filed pursuant to Federal Rule of Appellate Procedure 28(j), Rodriguez-Barajas claims support from Carachuri-Rosendo v. Holder, ___ U.S. ___, 130 S.Ct. 2577, 177 L.Ed.2d 68 (2010), and Dada v. Mukasey, 554 U.S. 1, 128 S.Ct. 2307, 171 L.Ed.2d 178 (2008). We likewise do not opine on the applicability, if any, of those decisions, nor do we limit the matters that the BIA may consider, as appropriate, on remand.
