I think most of it is just cultural conditioning, we play major key songs when we're happy and minor key songs when we're sad (though this is changing in modern music). Another part of it could be that minor chords have their midtones clash a bit more with there tonic notes than major chords, it sounds slightly "wrong" to our ears and takes away from the happy effect.
If I recall correctly, it's the center (the center of the piece, where all the perspective reverts back to)
