SCHOONOVER, Chief Judge.

The appellant, George L. Wilson, Trustee, challenges a partial summary judgment entered against him in an action arising out of a real estate transaction. We find that the trial court erred by determining that no material issue of fact existed and that the appellee, Mark J. Woodward, was entitled to a partial summary judgment as a matter of law. We, accordingly, reverse and remand for further proceedings.

On or about March 12, 1989, Mr. Wilson entered into a contract to sell certain real estate. The buyer's rights under this contract were subsequently assigned to Mr. Woodward. The written contract required that all deposits were to be placed with the escrow agent, Re/Max Realty Group. The sum of $55,000 was placed in escrow pursuant to this requirement.

Although the agreement contained a fixed closing date, it also provided that Mr. Woodward had the right to extend that date by two months if he paid, prior to the closing date, an extension fee in an amount equal to one per cent of the purchase price to the escrow agent. He was also allowed a second two month extension under the same terms and conditions. Mr. Woodward exercised his right to both extensions. On each occasion he delivered a check in the amount of $34,662.50 to Mr. Wilson's attorneys. The checks were made payable to the attorneys' escrow account and were placed in a separate interest bearing account.

After it became apparent to Mr. Wilson's attorneys that the sale was not going to close, the extension fees, together with interest thereon, were first forwarded to Re/Max and later deposited in the registry of the court.

When the sale of the property was not completed because of Mr. Woodward's default, Mr. Wilson filed an action against Mr. Woodward seeking a judgment in the amount of the deposit made to Re/Max and the extension fees paid to his attorneys. Mr. Woodward filed a motion for partial summary judgment seeking the return of the extension fees he had deposited in escrow with Mr. Wilson's attorneys. He contended that since these funds were not deposited with the escrow agent named in the contract, they were not subject to Mr. Wilson's claim.

Mr. Wilson's attorney did not appear at the summary judgment hearing because of a secretarial error in the attorney's office. An affidavit in opposition to the motion for partial summary judgment was filed the day of the hearing. Although Mr. Wilson's attorney did not appear at the hearing, the court granted Mr. Woodward's motion and entered an order the same day. A week later Mr. Wilson, pursuant to Florida Rule of Civil Procedure 1.530 and 1.540(b), moved for relief from the judgment and for a rehearing. The motion alleged that Mr. Wilson's attorney was not present at the hearing and an affidavit in opposition to the motion was not timely filed because the attorney's secretary made a mistake and did not calendar the hearing pursuant to the notice of hearing. The trial court denied the motion without stating whether it was denying the motion for relief from judgment, or if it had granted that part of the motion and then, after considering the affidavit and the rest of the record, was denying the motion for rehearing. This timely appeal followed.

Mr. Wilson's motion should have been granted on both of the grounds it set forth. Although the court has the authority to discipline counsel for failure to comply with the rules of civil procedure, ordinarily any punishment should be imposed upon the attorney and not the litigant. Beasley v. Girten, 61 So.2d 179 (Fla. 1952); Anthony v. Schmitt, 557 So.2d 656 (Fla. 2d DCA 1990), approved, Del Duca v. Anthony, 587 So.2d 1306 (Fla. 1991). In this case, the attorney presented uncontroverted evidence that he failed to appear at the hearing because of a mistake and not because of any wilful and flagrant act. Mr. Wilson's right to present a defense to Mr. Woodward's claim to the extension fees should not be taken from him because of this mistake. Beasley; Anthony. Pursuant to the well recognized principle that rule 1.540(b) should be liberally construed, Mr. Wilson's motion for relief from judgment should have been granted. See Florida Aviation Academy, Dewkat Aviation, Inc. v. Charter Air Center, Inc., 449 So.2d 350 (Fla. 1st DCA 1984).

We also find that the trial court should have granted the motion for rehearing and after considering the entire record, set aside the partial summary judgment. A motion for summary judgment may only be granted if the pleadings, depositions, answers to interrogatories, and admissions on file together with affidavits, if any, show that there is no genuine issue as to any material fact and that the moving party is entitled to a judgment as a matter of law. Fla.R.Civ.P. 1.510(c). The burden, furthermore, is on the movant to demonstrate conclusively that the nonmoving party cannot prevail. Gomes v. Stevens, 548 So.2d 1163 (Fla. 2d DCA 1989). If the record reflects the existence of any genuine issue of material fact, or the possibility of any issue, or if the record raises even the slightest doubt that an issue might exist, summary judgment should not be granted. Gomes; Snyder v. Cheezem Dev. Corp., 373 So.2d 719 (Fla. 2d DCA 1979). Mr. Woodward did not carry this heavy burden in this case.

The contract provided that in the event of a default the "deposit and the Extension Fee (but only to the extent the same shall have been then deposited with Escrow Agent) shall be paid over by the Escrow Agent to the Seller and retained by the Seller. . . ." The contract also provided that it could not be amended, modified, altered, or changed in any respect whatsoever except by a further duly executed written agreement.

Mr. Woodward contends that since the extension fees were not paid to the escrow agent named in the agreement, they were not placed at risk, and the court was therefore correct in ordering the funds returned to him. When a contract provides that if a buyer defaults, the deposit paid under the contract can be retained by seller, the seller can only recover a deposit actually made. Stewart v. Mehrlust, 409 So.2d 1085 (Fla. 2d DCA 1982). Under this principle of law, it can be argued that because the extension fees were not placed with the escrow agent named in the agreement they were not actually made. However, under certain circumstances, written contracts can be modified by a subsequent oral agreement of the parties even though the written contract purports to prohibit such modification. Crosslands Properties, Inc. v. Univest Crossland Trace, Ltd., 516 So.2d 320 (Fla. 2d DCA 1987); see also The Race, Inc. v. Lake River Recreational Properties, Inc., 573 So.2d 409 (Fla. 1st DCA 1991); King Partitions Drywall, Inc. v. Donner Enterprises, Inc., 464 So.2d 715 (Fla. 4th DCA 1985). In addition to modifying a contract by oral agreement under certain circumstances, a contract can also be modified by conduct which would render it a fraud upon one party for the other to refuse to perform the alleged oral modification. Pan American Engineering Co., Inc. v. Poncho's Constr. Co., 387 So.2d 1052 (Fla. 5th DCA 1980); Vitra-Spray, Inc. v. Gumenick, 144 So.2d 533 (Fla. 3d DCA 1962).

Mr. Wilson's affidavit in opposition to the motion for summary judgment, together with the attachments, presents a question of material fact concerning the modification of the contract to provide that the extension fees could properly be deposited with his attorney rather than with Re/Max. The trial court, accordingly, erred by entering a summary judgment.

Reversed and remanded for proceedings consistent herewith.

LEHAN and PATTERSON, JJ., concur.