
634 P.2d 690 (1981)
96 N.M. 668
Richard S. MECHEM and Verna L. Mechem, his wife, Plaintiffs-Appellants,
v.
CITY OF SANTA FE, et al., Defendants-Appellees.
No. 13503.
Supreme Court of New Mexico.
October 5, 1981.
Sommer, Lawler & Scheuer, Joseph G. Lawler, Houston Lee Morrow, Santa Fe, for plaintiffs-appellants.
Coppler & Walter, Frank R. Coppler, Santa Fe, for defendants-appellees.
*691 White, Koch, Kelly & McCarthy, Daniel H. Friedman, Santa Fe, amicus curiae.

OPINION
FEDERICI, Justice.
This is an appeal from the District Court of Santa Fe County. Appellant, Richard S. Mechem (Mechem), sought declaratory and injunctive relief from the effect of a restriction imposed in a special exception by the Board of Adjustment of Santa Fe (City) upon the person and property of Mechem. Mechem alleges that the personal restriction is unconstitutional, illegal, ultra vires and null and void. The parties stipulated to all pertinent facts and to the admissibility of the evidence presented. The trial court denied appellant relief. We reverse.
In 1967, the Santa Fe Board of Adjustment granted a special exception to operate a private tennis club in an R-1 district in Santa Fe. In granting the exception, the City required that the special exception terminate with any change in ownership of the premises. In 1976, the City approved an expansion of the tennis facility. During those proceedings, Mechem questioned the enforceability of the restriction referred to above. Soon thereafter, neighbors of Mechem who opposed the expansion of the facility brought an action in district court in an attempt to prevent the expansion, but were unsuccessful. In 1977, Mechem discovered that a facility similar to his own had been granted a special exception in an R-1 district without imposition of the added restriction at issue here. Mechem at that time again requested that the restriction be lifted, but the City refused to lift it. In 1978, claiming changed circumstances due to marital difficulties, Mechem again requested that the restriction be lifted. The City refused to act upon Mechem's request, even though the request had been placed on the agenda of the City Council for December 13, 1978. This suit was filed on January 5, 1979.
The issues we discuss on appeal are:
I. Whether Mechem is barred from the present action by the statute of limitations;
II. Whether Mechem is barred from the present action by unclean hands;
III. Whether Mechem is barred from the present action by laches; and
IV. Whether the City has the authority to impose a restriction on ownership of property when granting a special exception to a zoning ordinance.

I.
In deciding whether Mechem is barred by the statute of limitations from initiating the present proceedings, we look to the applicable statute, Section 3-21-9, N.M.S.A. 1978. It reads:
A. Any person aggrieved by a decision of the zoning authority, or any officer, department, board or bureau of the zoning authority may present to the district court a petition, duly verified, setting forth that the decision is illegal, in whole or in part, and specifying the grounds of illegality. The petition shall be presented to the court within thirty days after the decision is entered in the records of the clerk of the zoning authority.
The record shows that Mechem did not appeal to the district court following the 1967 and 1976 proceedings between the City and Mechem wherein the restriction was imposed. He may not now directly attack the restriction imposed by the City, Bolin v. City of Portales, 89 N.M. 192, 548 P.2d 1210 (1976), unless the restriction is void. See Nesbit v. City of Albuquerque, 91 N.M. 455, 575 P.2d 1340 (1977).
Mechem contends that he may collaterally attack the prior determination made by the City in 1967 and 1976, and the statute of limitations is therefore inapplicable. The basis of Mechem's collateral attack is that the City acted beyond the scope of its statutory authority and its actions were ultra vires and void. Collateral attack of a city ordinance was upheld in Dale J. Bellamah Corporation v. City of Santa Fe, 88 N.M. 288, 291, 540 P.2d 218, 221 (1975), where the court stated:

*692 Various courts have permitted collateral attacks upon ordinances which are void in the sense that the legislative body had no constitutional or statutory power to pass it or because the ordinance was never legally enacted. State v. Vargas, 6 Conn. Cir. 69, 265 A.2d 345 (1969); Bowling Green-Warren County Airport Bd. v. Long, 364 S.W.2d 167 (Ky. Ct. App. 1962); Simmons v. Holm, 229 Or. 373, 367 P.2d 368 (1961); 6 E. McQuillen, Municipal Corporations § 20.14 (3rd ed. rev. 1969). Since [§ 3-21-9] does not present the exclusive method for attacking invalid ordinances, we hold that a collateral attack upon the ordinance was permissible in the instant case. (Emphasis added.)
Compare Bolin v. City of Portales, supra, and Serna v. Board of Cty. Com'rs of Bernalillo County, 88 N.M. 282, 540 P.2d 212 (1975).
Collateral attack upon judicial proceedings has been permitted where the determinations of judicial bodies are found to be void. Nesbit v. City of Albuquerque, supra. Collateral attack has likewise been permitted to challenge an administrative determination which is void because it was made without express or implied statutory power. See State v. Civil Service Board, 226 Minn. 253, 32 N.W.2d 583 (1948); Foy v. Schechter, 1 N.Y.2d 604, 154 N.Y.S.2d 927, 136 N.E.2d 883 (1956).
In Bischoff v. Hennessy, 251 S.W.2d 582 (Ky. 1952), based upon facts similar to those in this case, the Kentucky court held that a thirty-day time limitation applicable to a zoning action was not exclusive and an action was permitted beyond the thirty-day limitation period, where the zoning authority acted illegally, and vested rights were denied in violation of the law or the constitutional provisions.
We hold that Mechem is entitled to collaterally attack the restriction imposed upon him by the City that made the special exception personal to him.

II.
The City contends that Mechem may not seek equitable relief because he has unclean hands. The key element under this doctrine is that Mechem's misconduct must be related to the transaction giving rise to the claim involved here. "What is material is not that plaintiff's hands are dirty, but that he dirtied them in acquiring the right he now asserts, ... ." Republic Molding Corporation v. B.W. Photo Utilities, 319 F.2d 347 (9th Cir.1963), cited in D.B. Dobbs, Remedies § 2.4 at 46 (1973).
The City argues that the tennis facility is being run as a business enterprise rather than as a private club. It claims that since Mechem has expanded the operation beyond the type of club he represented to City officials he would run, his actions were deceitful and amount to unclean hands. Mechem counters, stating that the record supplies ample evidence of instances of agreement and harmony between the City and himself directly related to the expansion of the club.
The record indicates that the special exception was granted to Mechem in 1967 upon the following conditions: (1) daylight hour operation only; (2) membership was not to exceed a maximum of 100; (3) no liquor was to be sold on the premises; and (4) this special exception was to remain valid only so long as the ownership and operation remained in the name of Mechem. In 1976, Mechem was allowed to expand his operation subject to the following additional conditions: (1) all sales by the Pro Shop were to be limited to members only; (2) the use of guest cottages was to be limited to members only; (3) membership was not to exceed 150; and (4) additional tennis courts were to be permitted north of Camino Corrales and the tract south of Camino Corrales was to be utilized as an off-street parking area only. These facts indicate that Mechem had two major transactions with the City regarding the conditions under which he was to operate his tennis facility. First, in 1967, he acquired the special exception which gave him the right to operate the facility in a residential zone, and second, in 1976, he acquired the right to expand his operation. It was in the 1967 transaction that Mechem became subject to *693 the restriction at issue. Whether or not Mechem's later conduct relating to the expansion of his facility was inequitable, is irrelevant to the determination of whether he was guilty of unclean hands at the time he acquired the exception in 1967. If Mechem had sought an equitable remedy to protect his right to expand his tennis facility, then his acts relating to how he expanded the facility would be relevant. This is not the case here. The record does not show that Mechem intended to expand his facility beyond the scope of the conditions contained in the special exception at the time he acquired it. The right to challenge the restriction at issue was acquired in 1967 when it was granted. The acts of Mechem in 1967 upon which the City relies for its defense of "unclean hands" cannot be used as a defense in the present proceedings.

III.
The City contends that Mechem is barred from bringing the present suit by laches.
The elements of laches are: (1) the City's invasion of Mechem's rights; (2) delay in asserting Mechem's rights, once Mechem had notice and opportunity to take legal action; (3) lack of knowledge by the City that Mechem would assert his rights; and (4) injury or prejudice to the City in the event relief is accorded to Mechem or the suit is not held to be barred. Butcher v. City of Albuquerque, 95 N.M. 242, 620 P.2d 1267 (1980). Whether or not the rule governing laches is to be applied depends upon the circumstances in each particular case. Hart v. Northeastern N.M. Fair Ass'n., 58 N.M. 9, 265 P.2d 341 (1953). Laches is not favored and the rule is applied only in cases where a party is guilty of inexcusable neglect in enforcing his rights. Cain v. Cain, 91 N.M. 423, 575 P.2d 607 (1978).
We do not believe that the rule of laches applies in this case. While Mechem did not assert his rights between 1967 and 1978, the delay alone does not necessarily constitute laches. In Trujillo v. Padilla, 79 N.M. 245, 442 P.2d 203 (1968), the court stated that laches is not necessarily a matter of time, but a question of the inequity of permitting the claim to be enforced. Mechem's inaction from 1967 to 1976 must be considered in the light of all the facts in the case as they relate to the element of laches.
The evidence falls short of proving that the City was prejudiced. The City has not materially changed its position to its detriment during this period, it has pointed to no evidence that has become unavailable, nor has it expended money or incurred new obligations in reliance upon Mechem's inaction. The City cannot claim prejudice because of an expanding commercial enterprise in a neighborhood when the City itself approved the enterprise. If Mechem has expanded the enterprise beyond the permitted special exception, that issue is not properly addressed in this lawsuit, nor is it a basis for a claim of laches by the City.
Absent a showing of prejudice, the doctrine of laches is not available to the City.

IV.
Having disposed of the above preliminary issues, we now turn to the merits of Mechem's claim, that the restriction upon personal ownership is ultra vires and void.
The City obtains its authority to zone from Sections 3-21-1 through 3-21-26, N.M.S.A. 1978 (Orig. Pamp. and Cum.Supp. 1981). It has no zoning authority beyond that provided by statute. City of Santa Fe v. Gamble-Skogmo, Inc., 73 N.M. 410, 389 P.2d 13 (1964). Section 3-21-1, N.M.S.A. 1978 limits the regulations and restrictions the City may impose when zoning:
A. For the purpose of promoting health, safety, morals or the general welfare, a ... municipality .. . may regulate and restrict within its jurisdiction the:
(1) height, number of stories and size of buildings and other structures;
(2) percentage of a lot that may be occupied;
(3) size of yards, courts and other open space;
(4) density of population; and

*694 (5) location and use of buildings, structures and land for trade, industry, residence or other purposes.
Section 3-21-8, N.M.S.A. 1978 (Cum.Supp. 1981), allows the City to grant special exceptions in certain situations:
C. . . . [T]he zoning authority by a majority vote of all its members may:
(1) authorize, in appropriate cases and subject to appropriate conditions and safeguards, special exceptions to the terms of the zoning ordinance or resolution:
(a) which are not contrary to the public interest;
(b) where, owing to special conditions, a literal enforcement of the zoning ordinance will result in unnecessary hardship; and
(c) so that the spirit of the zoning ordinance is observed and substantial justice done... .
If the City has authority to terminate a special exception upon a change in ownership, it must be found in the above statutes. The statutes do not expressly provide for regulation of land by making a special exception personal to a particular owner. Any power to do so must be by necessary implication and must reasonably relate to the objectives of zoning. Otherwise the regulation is ultra vires and unenforceable. See Vlahos v. Little Boar's Head District, 101 N.H. 460, 146 A.2d 257 (1958); Olevson v. Zoning Board of Review, 71 R.I. 303, 44 A.2d 720 (1945).
In Olevson, a restriction similar to that involved in this case was held to be invalid. The court reasoned that the restriction went beyond the zoning function of regulating real estate and attempted to regulate ownership.
The City contends that even under Olevson the conditions of the special exceptions now in issue do not restrict ownership because Mechem can sell his property at any time, and then the burden is upon the new owners to apply for a renewal of the special exception. However, the basis of the court's decision in Olevson was that the zoning authority is limited to regulating matters relating to the real estate itself and not the person who owns or occupies it. A restriction upon ownership, the court held, amounts to a mere license or privilege to an individual and is not related to the use of the property. Our Court has also previously stated that zoning concerns regulation of the uses of land and buildings. See Bd. of Cty. Com'rs., Etc. v. City of Las Vegas, 95 N.M. 387, 622 P.2d 695 (1980).
We hold that it is not within the proper function of the zoning authority to condition an exception to the use of real property upon personal rights of ownership rather than use.
The City points out that an agreement was negotiated between the parties. It argues that regardless of any authority the City may have, Mechem is bound by the agreement. In Edmonds v. Los Angeles County, 40 Cal.2d 642, 255 P.2d 772 (1953), the court applied promissory estoppel to prevent a plaintiff from circumventing an agreement he made with zoning officials. In Bringle v. Board of Supervisors of County of Orange, 54 Cal.2d 86, 4 Cal. Rptr. 493, 351 P.2d 765 (1960), a landowner was unsuccessful in revoking an agreement he made with zoning officials which committed him to the grant of an easement in exchange for a variance. Even so-called "contract zoning" has been upheld under certain circumstances. See R. Anderson, American Law of Zoning 2d § 9.21 (1976). We have no quarrel with the rule announced in those cases. The question here, however, is one of limits. As the court in Olevson, supra, stated:
It seems clear, speaking generally, that under the terms of the statute and of the ordinance applicable in this cause the respondent zoning board of review is given broad discretion in fixing conditions and safeguards when variances or exceptions are permitted. That discretion, however, is not unlimited.
Id., 44 A.2d at 722. The cases cited by the City do not involve the type of restriction involved in this case. While it is true that Mechem is not specifically restricted from *695 selling his property, the effect of the condition expressed in the exception and variance is to do just that. Mechem cannot sell the property and a purchaser cannot buy it without subjecting themselves to the probability of substantial and costly changes in the character of the property together with significant diminution in value of the property.
Even if the restriction at issue were negotiated, it is not enforceable because it is ultra vires. A zoning authority may not impose conditions upon a special exception whether it is negotiated or not if it has no power to impose the conditions. See Olevson, supra.
The trial court is reversed and directed to enter judgment in accordance with this opinion.
We do not express an opinion on whether Mechem is now in compliance with the other special conditions originally imposed by the City. We further express no opinion on their applicability to any prospective successor in interest to Mechem's property.
IT IS SO ORDERED.
RIORDAN, J., and STOWERS, District Judge, concur.
