
618 F.3d 657 (2010)
UNITED STATES of America, Plaintiff-Appellee,
v.
Kerry L. SMITH, Defendant-Appellant.
No. 09-1443.
United States Court of Appeals, Seventh Circuit.
Argued January 19, 2010.
Decided August 19, 2010.
*659 Amanda A. Robertson, Attorney (argued), Office of the United States Attorney, Benton, IL, for Plaintiff-Appellee.
Beau B. Brindley, Attorney (argued), Shapiro & Brindley, Chicago, IL, for Defendant-Appellant.
Before BAUER and WOOD, Circuit Judges, and KENNELLY, District Judge.[*]
KENNELLY, District Judge.
Kerry Smith pled guilty to criminal charges pursuant to a written plea agreement and was sentenced to a ninety-two month prison term. He appeals, contending that the district court erroneously deprived him of his Sixth Amendment right to retain the counsel of his choice. For the reasons stated below, we vacate Smith's conviction and remand with instructions to permit him to withdraw his guilty plea.

I. FACTS
In April 2007, a grand jury indicted Kerry Smith on a charge of conspiracy to distribute marijuana. Smith retained attorney John Rogers to represent him. The district court originally set the case for trial in mid-June 2007 but later continued the trial to September 2007.
In July 2007, the grand jury returned a superseding indictment. The superseding indictment added charges of money laundering, concealment of information from the Social Security Administration, and making false statements. The trial was continued, but the record does not reflect that a new date was set.
Discovery in the case was voluminous. In March 2008, Smith's counsel Rogers filed a motion to suppress evidence and a motion to sever charges. The case was put on hold, however, after Rogers requested and the judge ordered a competency evaluation of Smith. The evaluation was completed in June 2008. The docket does not reflect that a new trial date was set before or after the completion of the evaluation.
On August 25, 2008, Smith filed a motion asking to substitute a new retained attorney, Beau Brindley. The motion cited Smith's constitutional right to the counsel of his choice. The record does not reflect that the court had set a trial date at that point.
Smith's motion to substitute counsel was called for hearing on September 8, 2008. Prior to the hearing, there was still no indication that the district court had set a trial date. At the hearing, the district *660 court first found Smith competent to stand trial.
The court then turned to Smith's motion to substitute Brindley as his defense counsel. The court initially stated that it was inclined to permit substitution of counsel. It stated, however, that "the case is going to jury trial November 4th. I'm going to pick a jury the Friday before and if you are not ready you can't substitute. So it's just that simple." Sept. 8, 2008 Tr. 4. This was the first time the court had made reference to a trial date, and as we have noted the record contains no earlier mention that the court had set the case for trial in November (or, for that matter, any other date). At oral argument before us, the government conceded that no trial date had been set prior to the September 8, 2008 hearing and that there had been no indication that the court intended to try the case in November 2008.
After further discussion, the district court inquired when Brindley had filed the proposed motion substitution of counsel. Brindley said he had filed the motion several weeks earlier, and the court clerk reported that the motion had been filed on August 25. The district court then stated, "This matter has been set for some time. Linda, announce the trial date." The clerk stated, "Jury selection November 3rd; trial will be November 4th at 8:00 a.m."[1]Id. at 6. Brindley told the court that he was scheduled to be on trial in a multiple-defendant racketeering conspiracy case from October 6 through the beginning of December. The court then said, "You don't have any business in this case. You don't have time to do it." Id. at 7.
The following colloquy ensued:
MR. BRINDLEY: Mr. Smith has a constitutional right to choice of counsel.
THE COURT: No, he does not. He had a constitutional right to counsel. He doesn't have a constitutional right to pick any person he wanted. If that was the case, we would have Brendan Sullivan or Edward Bennett Williams or any number of attorneys. You don't necessarily get that.
MR. BRINDLEY: He getsunder the Constitution, I believe, your Honor, he gets whoever he can afford and is willing to represent him.
THE COURT: But he doesn't get his schedule. And that's what we're talking about here. Now, I'm telling you that on November 4th on [sic] this year, this case is going to trial. Now if you tell me you can't do it, you can't do it. But this case is going to trial November 4th. It is not a subject for discussion or a topic up for debate. I have nothing but matters set between now and then. My schedule is what it is. This is the time I've got to try the case.
I'm going to let the three of you discuss it and decide what you are going to do. But your case was already set in Chicago. You are going to have to work around that, or wherever it is. Now, the Court's in recess for five minutes.
Id. Tr. 7-8 (emphasis added).
After the five minute recess, Smith's original retained counsel, Rogers, reported that Smith did not want Rogers to continue to represent him. Rogers indicated that Smith had been opposed to Rogers' earlier request for a competency examination and that their relationship had deteriorated as a result. The court permitted Rogers to withdraw and struck his appearance.
*661 The court, addressing Smith, repeated that the case was set for trial on November 4less than two months hence. The court told Smith that "[i]f your lawyer can be here and represent you that is fine with the Court. If he can't, I will appoint a lawyer for you who will be ready on November 4th. So it's really up to Mr. Brindley whether or not he is going to represent you in the matter." Id. at 9. Brindley repeated that he had long been set for trial in the other case and was therefore unavailable on November 4.
The court denied Smith's motion to substitute counsel and stated, "[t]he Court will appoint counsel for the defendant and you will hear from us shortly." Id. at 10. Smith then asked that his original attorney, Rogers, continue to represent him, but the court denied Smith's request. The court also denied Smith's request (through Brindley) for a continuance. We quote the colloquy from the hearing:
THE COURT: Mr. Smith ... [d]o you have the means and apparently enough means to hire private counsel. So if youI will go ahead and appoint you a lawyer for November 4th and if you can get someone down here of your own choosing before then, that will be fine.
MR. SMITH: If Mr. Brindley can't defend me I still want to go with Rogers then.
THE COURT: Sometimes thatsometimes that's kind of like dating. Once you getonce you get thrown out the front door you might not want to come in
MR. SMITH: I just explained to both of them a second ago. I mean, if he couldn't take over the case I
THE COURT: Well, I'll take another brief recess. I'll be out in five minutes. I don't know how Mr. Rogers feels about that.
He doesn't want to do it. He's out. That's what I thought. So that being the case, we'll
MR. BRINDLEY: Your Honor, just for the record, for purposes of the record, Mr. Smith would request a continuance so that I be allowed to take the case since I am the counsel of his preference.
THE COURT: There's two things about that. You're not in this case.
MR. BRINDLEY: That I understand.
THE COURT: So I'll talk to him about that. Your motion to substitute is denied.
MR. BRINDLEY: Would you inquire of Mr. Smith then.
THE COURT: You are excused.
MR. BRINDLEY: Thank you, your Honor.
THE COURT: You are welcome.
Now, Mr. Smith. I assume thatthe Court doesn't ordinarily get involved in the financial situations, but I would think you have a refund coming. If you want to retain independent counsel you can do that. In the meantime, I'm going to get appointed counsel on your case and I urge you to cooperate with this person. November 4th what you need to know and if you would check with people that have been in my court, this case is going to trial. So just do the best you can, I guess. You have madeI think you have made some decisions that you might find questionable, but you will get a fair trial. And if you can get someone here quickly, the Court would seriously consider letting them substitute in.
Id. at 10-12.
Later that same date, the court entered an order appointing a new attorney, Ronald Jenkins, to represent Smith. Jenkins *662 first appeared before the court on September 12, 2008. He stated that he had first spoken to Smith just before the hearing. Jenkins also indicated that he would be ready to proceed on the trial date the court had set.
On September 29, 2008, Smith entered a guilty plea to all of the charges in the superseding indictment. This was exactly three weeks after the court had denied Smith's request to allow Brindley to represent him, and about two and one-half weeks after Jenkins first appeared.
Smith pled guilty pursuant to a written plea agreement. The plea agreement did not include any written affirmation that Smith was acting voluntarily when he signed the agreement. It did, however, include a statement that Smith "is fully satisfied with the representation he has received from his counsel." Plea Agr. ¶ III.1. The plea agreement also included the following terms:
1. The Defendant understands that by pleading guilty, he is waiving all appellate issues that might have been available if he had exercised his right to trial....
2. The Defendant is aware that Title 18, Title 28, and other provisions of the United States Code afford every defendant limited rights to contest a conviction and/or sentence. Acknowledging all this, and in exchange for the recommendations and concessions made by the Government in this plea agreement, the Defendant knowingly and voluntarily waives his right to contest any aspect of his conviction and sentence that could be contested under Title 18 or Title 28, or under any other provision of federal law, except that if the sentence imposed is in excess of the Sentencing Guidelines as determined by the Court (or any applicable statutory minimum, whichever is greater), the Defendant reserves the right to appeal the reasonableness of the sentence....
3. Defendant's waiver of his right to appeal or bring collateral challenges shall not apply to: 1) any subsequent change in the interpretation of the law by the United States Supreme Court or the United States Court of Appeals for the Seventh Circuit, which is declared retroactive by those Courts, and which renders the Defendant actually innocent of the charges covered herein, and 2) appeals based upon Sentencing Guideline amendments which are made retroactive by the United States Sentencing Commission (see U.S.S.G. § 1B1.10)....
Plea Agr. ¶¶ III.1-3.
During the guilty plea colloquy, attorney Jenkins stated that he had reviewed the file that he had received from Smith's prior counsel and had met with Smith "for a significant period of time one day" to review evidence. Sept. 29, 2008 Tr. 5. Jenkins stated that on the Friday before the guilty plea hearing, he had a phone call with Smith, who was in custody pending trial. Jenkins had met with Smith again the morning of the guilty plea.
After questioning Smith, the court found him competent. The court elicited Smith's agreement that he had, in the court's words, "plenty of time" to discuss the case with Jenkins and that he was satisfied with Jenkins as a lawyer. Id. at 7-8. The court ascertained that Smith understood the rights he would give up by pleading guilty. Id. at 8-10. After discussing the elements of the offenses charged and the possible penalties, id. at 10-14 & 15-17, the court asked Smith whether anyone had threatened or forced him to plead guilty, and Smith said no. Id. at 14. This was the only reference to voluntariness; the court did not ask Smith whether any promises had been made to him and did not directly ask whether he was acting *663 voluntarily or of his free will. See Fed. R.Crim.P. 11(b)(2) (requiring court to "address the defendant personally in open court and determine that the plea is voluntary and did not result from force, threats, or promises (other than promises in a plea agreement))." There was also a brief, though oblique, reference to the appellate waiver, which we will discuss shortly. Id. at 17. The court elicited from Smith a statement that the matters stated in a separately-filed stipulation of facts were true. Id. at 18. After some further discussion, the court found that Smith's plea was knowing and voluntary. Id. at 20.
In February 2009, the district court imposed a prison sentence of ninety-two months. Smith filed a notice of appeal. The government moved to dismiss the appeal. We consolidated the motion with argument on the merits and directed the parties to address in their merits briefs whether Smith's guilty plea was voluntary.

III. DISCUSSION
Smith contends that the district court violated his Sixth Amendment right to counsel when it denied his motion to substitute Brindley as trial counsel. Before addressing the merits, we consider the government's contention that Smith waived his right to appeal.

A. Appellate Waiver
The government has not argued, either in its motion to dismiss or in its brief, that simply by pleading guilty, Smith waived any contention that the district court violated his Sixth Amendment rights. The point is thus likely forfeited.
Forfeiture aside, any such contention would be lacking in merit. It is true that an unconditional guilty plea typically waives non-jurisdictional defects in the proceedings below. See Tollett v. Henderson, 411 U.S. 258, 267, 93 S.Ct. 1602, 36 L.Ed.2d 235 (1973). The Supreme Court has recently held, however, that the erroneous deprivation of the right to counsel of choice in violation of the Sixth Amendment is a "structural error" in a criminal proceeding and is not subject to harmless error analysis. United States v. Gonzalez-Lopez, 548 U.S. 140, 150-52, 126 S.Ct. 2557, 165 L.Ed.2d 409 (2006). The Court stated that
[d]ifferent attorneys will pursue different strategies. And the choice of attorney will affect whether and on what terms the defendant cooperates with the prosecution, plea bargains, or decides instead to go to trial. In light of these myriad aspects of representation, the erroneous denial of counsel bears directly on the framework within which the trial proceeds, or indeed on whether it proceeds at all.... Many counseled decisions, including those involving plea bargains and cooperation with the government, do not even concern the conduct of the trial at all.
Id. at 150, 126 S.Ct. 2557.
Relying on Gonzalez-Lopez, the Fifth Circuit recently held that a defendant's guilty plea does not preclude him from challenging on appeal a denial of his right to counsel of choice. United States v. Sanchez Guerrero, 546 F.3d 328, 332 (5th Cir.2008). We agree. As the Fifth Circuit stated, "it is obvious that the choice of counsel may seriously impact a defendant's decision to plead guilty." Id. "If a defendant is erroneously denied the counsel of his choice, it is a structural error in the trial that brings into question the voluntary and intelligent character of the guilty plea itself." Id. Under the circumstances, Smith's guilty plea does not amount to a waiver of his Sixth Amendment choice-of-counsel claim.
*664 The government argues that Smith waived the right to appeal via the appellate waiver in his plea agreement. There is no question that a defendant may waive his right to a direct appeal as part of a plea agreement. See, e.g., Jones v. United States, 167 F.3d 1142,1144 (7th Cir. 1999). But appellate waivers are not blindly enforced. See, e.g., United States v. Rhodes, 330 F.3d 949, 952 (7th Cir.2003). "To determine if a defendant knew and understood the plea agreement, we must examine the language of the plea agreement itself and also look to the plea colloquy between the defendant and the judge." United States v. Chapa, 602 F.3d 865, 868 (7th Cir.2010) (citing United States v. Sura, 511 F.3d 654, 661 (7th Cir.2007)).
The appellate waiver itself was reasonably clear. That, however, is not the end of the inquiry. If it was, Rule 11(b)'s requirement that the district court address the defendant personally would be a dead letter.
Turning to the district court's colloquy with Smith regarding the appellate waiver, we conclude that it fell short of what the law requires. Federal Rule of Criminal Procedure 11 was amended effective December 1, 1999 "to reflect the increasing practice of including provisions in plea agreements which require the defendant to waive certain appellate rights." Fed. R.Crim.P. 11, 1999 advisory committee notes. The newly-added provision states that the district court "must inform the defendant of, and determine that the defendant understands ... the terms of any plea-agreement provision waiving the right to appeal or to collaterally attack the sentence." Fed.R.Crim.P. 11(b)(1)(N).
The district court erred by failing during the plea colloquy to advise Smith of the terms of the appellate waiver and determine whether he understood that provision. See Sura, 511 F.3d at 661-62. Smith did not argue this omission before the district court, so we review the point for plain error. United States v. Vonn, 535 U.S. 55, 59, 122 S.Ct. 1043, 152 L.Ed.2d 90 (2002); Sura, 511 F.3d at 658. Specifically, Smith must show that his substantial rights were affected and that the error seriously affected the fairness, integrity, or public reputation of the court proceeding. Vonn, 535 U.S. at 62-63, 122 S.Ct. 1043.
We conclude that Smith has made the necessary showing. First, Smith's substantial rights were affected; the violation of his Sixth Amendment rights occurred and was complete (as we will discuss) at the time the district court denied his motion to substitute counsel. Second, the error had a significant effect on the fairness of the proceedings. The right to counsel is one of the bedrock elements protecting the fairness of the adversary process. See, e.g., Strickland v. Washington, 466 U.S. 668, 684, 104 S.Ct. 2052, 80 L.Ed.2d 674 (1984). The Supreme Court made it clear in Gonzalez-Lopez that "the right to select counsel of one's choice ... has been regarded as the root meaning of the constitutional guarantee" contained in the Sixth Amendment. Gonzalez-Lopez, 548 U.S. at 147-48, 126 S.Ct. 2557.
We therefore consider whether the trial court complied with Rule 11(b)(1)(N)'s requirements. During the course of the guilty plea colloquy, the court did not explain, quote, or even summarize the appellate waiver to Smith. Nor did it elicit from Smith a statement that reasonably can be construed as indicating awareness that he was giving up his right to appeal from rulings the court had previously made. Rather, the court referred to the plea agreement's appellate waiver only obliquely, and in a brief question not to *665 Smith himself, but to the attorney the court had appointed to represent him:
THE COURT: Now not having read all of this agreement is there ais there an appeal waiver in here?
MR. JENKINS: Yes, sir.
THE COURT: And the terms of that waiver are what?
MR. JENKINS: Basically, your Honor, that everything is waived with the exception of the reasonableness of the sentence.
THE COURT: All right.
MR. JENKINS: And he can't withdraw his plea.
THE COURT: You understand that?
MR. SMITH: Yes, your Honor.
Sept. 29, 2008 Tr. 17.
This was inadequate, and it did not comport with the requirements of Rule 11(b)(1)(N). All that was stated in the recordby counsel, not the courtwas that "everything is waived with the exception of the reasonableness of the sentence," and that Smith "can't withdraw his plea." The district court, which based on its own comments apparently had not reviewed the terms of the appellate waiver, did not explain the substance of the waiver to Smith, as Rule 11(b)(1)(N) requires. The Rule does not require a detailed description a statement that made it clear Smith was giving up any right he might have to appeal decisions the court had made, his conviction, and his sentence would have sufficedbut the court gave Smith no explanation at all.
The court's failure to provide Smith any explanation or description of the appellate waiver was not cured by its vague question to Smith, "You understand that?" "That" is most reasonably understood as referring to what Smith's counsel had just said that Smith had agreed he "can't withdraw his plea." We cannot reasonably conclude from this brief exchange that Smith understood, or even said he understood, the appellate waiver.
Not every deviation from Rule 11(b)'s requirements renders an appellate waiver unenforceable. In this case, however, the record of the plea colloquy does not reflect that Smith understood the appellate rights he was giving up. More to the point, the record does not reflect an understanding that Smith was giving up his right to appeal from the district court's denial of his defense counsel of choice. Given the court's failure to meet the requirements of Rule 11(b)(1)(N) and the fundamental nature of the underlying right at issue, the plea agreement's appellate waiver does not preclude Smith from challenging on direct appeal the court's alleged denial of his right to counsel of his choice.

B. Denial of Choice of Counsel
Turning to the merits, we conclude that Smith was denied his Sixth Amendment right to his choice of counsel. The Sixth Amendment "guarantees a defendant the right to be represented by an otherwise qualified attorney whom that defendant can afford to hire...." Caplin & Drysdale, Chartered v. United States, 491 U.S. 617, 624, 109 S.Ct. 2646, 105 L.Ed.2d 528 (1989). The constitution "commands... that the accused be defended by the counsel he believes to be the best." Gonzalez-Lopez, 548 U.S. at 146, 126 S.Ct. 2557.
This right, of course, is not without limits. Among other things, trial courts have the authority to establish criteria for admitting lawyers to practice, and a defendant may not demand that a court honor his waiver of conflict-free representation. See id. at 151-52, 126 S.Ct. 2557 (citing Wheat v. United States, 486 U.S. 153, 159-60, 108 S.Ct. 1692, 100 L.Ed.2d *666 140 (1988)). Courts also have an interest in ensuring that trials "are conducted within the ethical standards of the profession and that legal proceedings appear fair to all who observe them." Wheat, 486 U.S. at 160, 108 S.Ct. 1692 (quoted in Gonzalez-Lopez, 548 U.S. at 152, 126 S.Ct. 2557)).
More directly pertinent to Smith's case, the Supreme Court has "recognized a trial court's wide latitude in balancing the right to counsel of choice against the needs of fairness and against the demands of its calendar." Gonzalez-Lopez, 548 U.S. at 152, 126 S.Ct. 2557 (citations omitted). The record in this case, however, does not support a contention that the demands of the district court's calendar justified denying Smith's request to have Brindley represent him. The district court based its ruling entirely on its determination to try the case on November 4, 2008, a date on which Brindley was unavailable due to a previously-set trial date in another court. But the record reflects that the court had not set a November 4 trial date, or for that matter any trial date, prior to Brindley's appearance in court on the motion to substitute. The trial date was not set until the date Brindley appearedafter the motion to substitute his appearance had been on file for two weeks. (As we have indicated, the government stated at oral argument before us that prior to the hearing on the motion to substitute, no trial date had been set.)
We also note that based on the record, there is some question whether the district court correctly understood the scope of Smith's right to counsel of his choice. As quoted earlier, after Brindley argued that "Mr. Smith has a constitutional right to choice of counsel," the court replied, "No, he does not. He had a constitutional right to counsel. He doesn't have a constitutional right to pick any person he wanted." Sept. 8, 2008 Tr. 7. That statement is inconsistent with Gonzalez-Lopez and its Supreme Court forebears. Though the court followed this with a statement alluding to the court's schedule and counsel's availability, the court's comments, taken as a whole, reflect that it did not give sufficient consideration to Smith's Sixth Amendment right to retain the counsel of his choice.
The record also makes clear that the district court was unwilling to entertain any sort of a continuance at all. Specifically, the court stated that "this case is going to trial November 4th. It is not a subject for discussion or a topic up for debate." Sept. 8, 2008 Tr. 8. When Brindley attempted to bring up the topic again, the court declined to consider it, saying that "[y]ou're not in this case" and that "I'll talk to him [Smith] about that." Id. at 11. But when Brindley followed up by asking the court to inquire of Smith, the court did not do so, telling Brindley, "You are excused." Id.
Adherence to a trial date once set is a worthy goal, but a district court's schedule, though a significant consideration, does not automatically trump all other interests. See United States v. Miller, 327 F.3d 598, 601 (7th Cir.2003). In this case, no trial date had been set prior to Brindley's appearance on the motion to substitute. For this reason, allowing him to substitute as Smith's counsel would not have adversely impacted any date the court had previously set. The court's desire to try the case on November 4 does not justify its denial of Smith's right to counsel of his choice.
We also reject the government's argument that Smith was trying to manipulate the schedule by seeking to substitute Brindley's appearance. This argument lacks any support in the record. Again, no trial date had been set prior to the hearing on the motion to substitute. As a result, there is no basis to conclude that Smith *667 was trying to stall the disposition of the case.
We find that the district court erroneously denied Smith his constitutional right to his choice of defense counsel. Under Gonzalez-Lopez, Smith is not required to prove that he was prejudiced by the violation.[2] He is entitled to have his guilty plea vacated.

III. CONCLUSION
For the foregoing reasons, we vacate Smith's conviction and sentence and remand his case to the district court with instructions to permit Smith to withdraw his guilty plea. The government's motion to dismiss the appeal is denied. Circuit Rule 36 shall apply on remand.
NOTES
[*]  The Honorable Matthew F. Kennelly, United States District Court for the Northern District of Illinois, sitting by designation.
[1]  We note that the date the clerk announced for jury selection was a Monday, not "the Friday before" the purported trial date as the district court had earlier stated.
[2]  Although a defendant who has been improperly denied the counsel of his choice need not show prejudice, we note that Smith pled guilty (in a case that had been pending for nearly eighteen months) a mere three weeks after Brindley was denied permission to substitute, Smith's ensuing request to reinstate his original attorney was overruled, and appointed counsel was forced upon him.
