After Ashoka's death, a series of less-skilled emperors came to power. These emperors changed some of their policies. They were less accepting of other religions, and some did not respect Hinduism. The empire was divided, and local leaders took more control. Invaders from northern kingdoms attacked the divided empire, further weakening it. By 180 BCE, just 50 years after Ashoka's death, the empire had broken into many smaller, independent kingdoms.
Google: “The right to travel is part of the Liberty of which a citizen cannot deprived without due process of law under the Fifth Amendment.” So try the 5th!