Classical music is not universal because in its European form, it is clearly an European cultural phenomenon, and in its beginnings, it was mostly a Central-European cultural product, because it mostly came out of Italy and Germany.
It is also not universal because it was elitist, produced mostly for wealthy people. The peasants who were the vast majority of the people, mostly listened to other types of music, in what can be referred to under the umbrella of "folk music".