
482 So.2d 501 (1986)
BOARD OF MEDICAL EXAMINERS and Department of Professional Regulation, an Agency of the State of Florida, Appellants,
v.
Hooshang KADIVAR, M.D., Appellee.
No. 85-48.
District Court of Appeal of Florida, Fourth District.
January 22, 1986.
Rehearing Denied February 27, 1986.
Salvatore A. Carpino, Tallahassee, for appellant-Department of Professional Regulation.
Jim Smith, Atty. Gen., Tallahassee, and M. Catherine Lannon, Asst. Atty. Gen., Tallahassee, for appellant-Board of Medical Examiners.
Nancy Little Hoffmann, P.A., Fort Lauderdale, for appellee.
DOWNEY, Judge.
Chronologically, proceedings between The Department of Professional Regulation, Board of Medical Examiners (Agency) and Hooshang Kadivar, M.D. (doctor) began February 24, 1982, when the Agency filed an administrative complaint against the doctor, charging him with violation of four provisions of section 458.331(1), Florida Statutes (1982). After a hearing provided by section 120.57(1), Florida Statutes (1982), the Hearing Officer found that the doctor had violated two of the section's charges involving three different female patients and, furthermore, that he had violated a third section of the statute, subsection (c), by pleading nolo contendere to a crime that directly relates to the practice of medicine. The Hearing Officer recommended suspension of the doctor's license to practice medicine. After entry of the Hearing Officer's Recommended Order, the doctor filed this proceeding in the Circuit Court of the Nineteenth Judicial Circuit, praying for an injunction against the Department. A hearing was immediately scheduled for temporary injunction, since *502 the final Agency hearing was imminent. On February 11, 1983, the circuit court entered a temporary injunction against the agency enjoining them:
from considering, acting upon, or relying upon, in any manner whatsoever, those pleas of nolo contendere or any other aspect of those criminal proceedings currently pending in the case of STATE OF FLORIDA -vs- HOOSHANG KADIVAR, ... in connection with the administrative proceedings currently pending in the case of DEPARTMENT OF PROFESSIONAL REGULATION -vs- HOOSHANG KADIVAR, ... in the Division of Administrative Hearings, State of Florida.
The following day the Agency convened the final hearing and was advised of the circuit court injunction. At the conclusion of the hearing, the Agency approved the Hearing Officer's Recommended Order except for matters pertaining to the nolo plea to the criminal charge and voted to revoke the doctor's license. That final agency order was appealed to this court and affirmed on November 30, 1983, in Kadivar v. Dept. of Professional Regulation, 441 So.2d 1173 (Fla. 4th DCA 1983).
In March, 1983, the Agency filed a motion in the circuit court proceeding to dismiss for lack of jurisdiction and failure to state a cause of action, and a motion to transfer for improper venue. In June, 1984, the trial court ultimately heard the motion to dismiss and the doctor's motion to hold the Agency in contempt for its failure to comply with the court's temporary restraining order. Finally, in December, 1984, the trial court entered the order being reviewed, which denied the motion to dismiss and the motion to transfer venue and found the Agency in contempt of its temporary restraining order. Appellants were given twenty days to file an answer to the complaint and the doctor was given ten days to respond thereto and to advise the court of his desires relative to the imposition of sanctions for the Agency's alleged contemptuous conduct.
The Agency contends on appeal that the trial court erred in exercising jurisdiction and entering the temporary restraining order; in denying appellant's motion to dismiss; in refusing to transfer venue; and in finding the agency in contempt.
Initially, appellee suggests that we have jurisdiction only to consider the venue question and the contempt finding; that we have no jurisdiction at this point to consider the aspects of the non-final order denying the motion to dismiss for lack of subject matter jurisdiction and failure to state a cause of action pursuant to Florida Rule of Appellate Procedure 9.130(a)(3). Since we agree with this suggestion, we decline to treat the agency's points having to do with jurisdiction and failure to state a cause of action.
Turning to the venue question, we find no merit in the Agency's argument that the proper venue of this case is in Leon County where the agency maintains its principal headquarters. We recognize that ordinarily under the common law rule Leon County would be the proper venue for a suit involving the Agency. However, the "sword-wielder" doctrine, set forth in the cases noted in the margin,[1] provides an exception to the common law rule. The exception applies here because the proceedings between the doctor and the Agency commenced when the Agency sought to deprive the doctor of a property right  his license to practice medicine. Therefore, venue of this action in St. Lucie County is proper.
The trial court's temporary injunction was entered the day before the Agency hearing giving rise to the final Agency order revoking the doctor's license. The transcript of that hearing reflects that, at the commencement of the hearing, counsel for the Board advised the Board that they *503 were there to consider the Hearing Officer's Recommended Order and hear argument from the parties. Counsel for the Department of Professional Regulation then advised the Board of the injunction entered the previous day, which enjoined the Board from "going forward on the count in the administrative complaint dealing with the issue of nolo contendere plea and conviction in County Court by Dr. Kadivar." He also apprised the Board that the injunction required the Board to set forth within its final order that the Board did not consider the nolo plea. The attorney for the Department asked counsel for the doctor if he agreed with his understanding of the effect of the injunctive order. The doctor's counsel stated that he did agree to the interpretation but not counsel's advice to proceed on the other aspects of the Recommended Order. Apparently he wanted the proceeding stayed until the criminal proceedings against Dr. Kadivar were resolved. Counsel for the Board then read the restraining order into the record. The chairman then announced that the Board would deny a motion to stay the proceedings and they would consider the Recommended Order, keeping in mind the court's injunction. After everyone, including the doctor, had an opportunity to express himself, a motion was made to "accept the hearing officer's findings of fact, with the exception of the nolo contendere plea, which we will exclude from those findings of fact." The motion carried unanimously. The Board then considered the hearing officer's conclusions, excluding the conclusion that found the doctor guilty of violating the statute and excluding the nolo plea. The Board then voted to revoke the doctor's license.
This court can read the transcript of the Agency hearing as easily as the trial judge did and it is our judgment that both the Board and Counsel made a consciencious effort to fully comply with the injunctive order in all respects. Accordingly, we find no support for the trial judge's conclusion that the Board was guilty of willful contempt.
The Agency contends that the questions remaining in the trial court vis-a-vis the constitutionality of section 458.331(1)(c), Florida Statutes, are res judicata because this court has already ruled on that question in a proceeding between these same parties. However, as of this time that defense has not been raised in the trial court, since it is an affirmative defense and the Agency has not yet filed its answer. We recognize that, in the appeal from the final Agency order in the administrative proceeding, one of the doctor's points on appeal was the unconstitutionality of section 458.331(1)(c) and that this court's opinion expressly stated it found no merit to the points on appeal. However, as we indicated in Frank v. Campbell Property Management, 351 So.2d 364 (Fla. 4th DCA 1977), orderly procedure requires that the matter be pled in the Agency's answer and submitted to the trial court.
Accordingly, we affirm the trial court's order on venue, reverse the order as it pertains to the adjudication of contempt, and remand the cause to the trial court for further proceedings.
AFFIRMED IN PART; REVERSED IN PART; and remanded with directions.
WALDEN, J., concurs.
GLICKSTEIN, J., concurs specially with opinion.
GLICKSTEIN, Judge, concurring specially.
I agree with what I understand the opinion solely to hold; namely, that we have jurisdiction only to consider the adjudication of contempt and denial of the change of venue; and that we are reversing the former and affirming the latter.
NOTES
[1]  Carlisle v. Game & Fresh Water Fish Comm'n, 354 So.2d 362 (Fla. 1978); Graham v. Edwards, 472 So.2d 803 (Fla. 3d DCA 1985); State Dept. of Transportation v. Chothen, 328 So.2d 574 (Fla. 3d DCA 1976); Dept. of Revenue v. First Federal Savings & Loan Ass'n of Ft. Meyers, 256 So.2d 524 (Fla. 2d DCA 1971).
