Formerly, there was a Brähmana hailing from the family of Kasyapa in that holy spot. He was well-known as Upädhyäya. He was devoted to the Vedic lore. He begot a son later in his advancing age even as he strictly adhered to regular study of the Vedas. He acquired ample wealth. When only five years old that son expired causing great sorrow to his father and mother. On knowing that his son had gone to the abode of Dharmaräja, the Brähmaoa became angry with Vaivasvata (Dharmaräja). In his height of grief, he took some water in his hand. With mental concentration and personal cleanliness, he then pronounced a terrible curse on Dharmaräja. "Since I have been made a sonless person by that wicked-souled one, Yama, of defiled soul, he will also become sonless. Further people in this world will not perform worship to him nor glorify his name as in the case of other heaven dwellers. If anyone gets up early in the morning and recalls his name, there shall be obstacle in his auspicious activity."
Dharmaraja then pleads Lord Brahma to mitigate the curse. Lord Brahma creates Ailments to take lives of people after their destined life span.
27-30. Lord Brahma said: "O Yama, all these Ailments have been employed by me. They will always render help to you in all matters. Now if anyone reaches the end of his life-span these should be sent always by you to take his life. Thereby, the ignominy arising from the death of people shall devolve upon these alone on the earth and not on you. Therefore, at my behest, go to your own abode and be engaged in your duty. Undoubtedly, you will never incur a fault (sin)."
Then the Son of Ravi took all those Ailments to his world and spoke to them respectfully:
32-34. "You shall regularly ask Citragupta and go to the earth for taking >away the life of people when the proper time arrives.
Lord Yama now returns the Brahmana son to Brahmana and son requests his father to take back curse.
The Brähmara said:
O my son, my utterance can never be untrue even if it is jokingly made. It is all the more so if uttered by a grief stricken' one.
Hence, O wise one, he will not get a son from the divine womb. due to my curse.
He will have another son (one son will be born from Sudra women known as Vidura which is explained in previous chapter) born of the womb of a human female by means of Räjasüya and Asvamedha. He will redeem him.
Of what avail is a son bom to one if he is not capable of redeeming the members of his father's family by performing excellent rites on the earth?
As regards the curse uttered by me in regard to his worship earlier, O dear son, listen to what I say:
His worship with various kinds of Mantras mentioned in the Vedas will not find a place on the earth at all, dear son.
His worship shall be with Mantras composed by humans. It will be different from that of the other Devas. It is the truth that has been uttered by me.
This son is none other than the Yudishtara, the eldest Pandava.
Regarding the story of Vidura, Dharmaraja was cursed by Sage Madavya to take birth from Sudra women. Sage Mandavya was leper and due to severe penance of his chaste wife Dhirgika, he gets relieved of Leprosy. He then inquires Lord Yama the reason for his leprosy. Lord Yama says that mandava pierced a crane with tip of spear in one of his previous births and as a result of it he was born as leper. On hearing the words of Yama, Mandava gets infuriated as he considers punishment of leprosy is unfair for such a small deed. He curses Yama to born from womb of a Shudra. Later Yama prays to Shiva and Shiva grants boon to Yama that though he will be born from womb of Shudra (Vidura), he will be man of righteous temperament.
Son of Brahmin composes a mantra for Yama beginning with "Sugam nah panthah". This story happened at Dharmeswara linga and people who pray Yama statue, installed by son of Brahmana, at this place with the above Mantra will never die prematurely.
So, due to above curse, worship of Yama is rare like Brahma.
Chapter 139 Greatness of Dharmarajeswara Book 6 Nagarakanda of Skanda Purana.
https://en.wikipedia.org/wiki/Nagarkanda_Upazila
https://freeglobaluniversity.blogspot.com/search/label/Skanda