To make the mind austere is to detach it from sense gratification. It should be so trained that it can be always thinking of doing good for others. The best training for the mind is gravity in thought. One should not deviate from Kṛṣṇa consciousness and must always avoid sense gratification. To purify one's nature is to become Kṛṣṇa conscious. Satisfaction of the mind can be obtained only by taking the mind away from thoughts of sense enjoyment. The more we think of sense enjoyment, the more the mind becomes dissatisfied. In the present age we unnecessarily engage the mind in so many different ways for sense gratification, and so there is no possibility of the mind's becoming satisfied. The best course is to divert the mind to the Vedic literature, which is full of satisfying stories, as in the Purāṇas and the Mahābhārata. One can take advantage of this knowledge and thus become purified. The mind should be devoid of duplicity, and one should think of the welfare of all. Silence means that one is always thinking of self-realization. The person in Kṛṣṇa consciousness observes perfect silence in this sense. Control of the mind means detaching the mind from sense enjoyment. One should be straightforward in his dealings and thereby purify his existence. All these qualities together constitute austerity in mental activities.
(Bhagavad-Gita--17:16--purport).

It is also said that Vasudeva and Devakī, in their previous birth as Sutapā and Pṛśni, underwent a severe type of penance to get the Lord as their son, and as a result of such austerities the Lord appeared as their son. It is already declared in the Bhagavad-gītā that the Lord appears for the welfare of all people of the world and to vanquish the asuras, or the materialistic atheists.
(Srimad Bhagavatam--1:8:33--purport).

As the Lord advises in the Bhagavad-gītā, "Whatever you do, do it for Me; whatever you eat, first offer it to Me; whatever charity you want to give, first give it to Me; and whatever austerities and penances you want to perform, perform them for Me." In this way the karmīs are directed gradually to develop Kṛṣṇa consciousness. Similarly, Kṛṣṇa directs the philosophers to approach Him gradually by discriminating between Brahman and māyā, for at last, when one is mature in knowledge, he surrenders unto Kṛṣṇa. As Kṛṣṇa says in the Bhagavad-gītā, "After many, many births, the wise philosopher surrenders unto Me."
(Krsna Book).

When Dhruva Mahārāja was in the forest executing his austerities, his father, Uttānapāda, heard everything about his very wonderful activities. Although Dhruva Mahārāja was the son of a king and was only five years old, he went to the forest and executed devotional service under strict austerity. Therefore his acts were all wonderful, and when he came back home, naturally, because of his spiritual qualifications, he became very popular amongst the citizens.
(Srimad Bhagavatam--4:9:65--purport).

A devotee is not concerned with enjoyment or distress; he simply desires to execute devotional service. It is said in the Śrīmad-Bhāgavatam that devotional service should be apratihatā, unchecked by the material conditions of happiness or distress. The devotee undergoes processes of austerity such as observing Ekādaśī and similar other fasting days and refraining from illicit sex life, intoxication, gambling and meat-eating. Thus he becomes purified from the reactions of his past impious life, and because he engages in devotional service, which is the most pious activity, he enjoys life without separate endeavor.
(Srimad Bhagavatam--4:12:13).