Answer:
You'd want to go back around to 500 BC
Explanation:
Bhudda would've been born around 500-400 BC if you traveled back any further you wouldn't be able to meet him.
Personally, I think it's because it was simple and free entertainment. Given that there wasn't a taboo around death and watching others die, and it actually being encouraged, like how it was in Anciient Rome, I'd say a lot of people would actually at least try out visiting a gladiatorial contest.