Advance Amount For Property

My question is for my brother. He likes to sell his land for some financial crises. A man has made an advance amount without any contract. At first, he told that within 3 months he will settle all the outstanding amount and register. But he failed and again gave some more amount, my brother was used that for his loans and expenses and like this 3 times, the buyer has given about Rs. 4.5 Lakhs after a complete year passed but without any contract or Sale deed / agreement. At last, the land was sold after 3 years lower than the actual amount of the first buyer agreed amount. Now my brother has some amount like 1 lakhs remaining and has to return 4.5L. The buyer is forcing him to settle his amount. My brother is in heavy depression, how to return and he is not refusing but asking for some time to return. Please advice.